-- Logs begin at Tue 2022-09-13 02:40:44 UTC, end at Tue 2022-09-13 03:42:29 UTC. -- Sep 13 03:06:28.456940 dv-cinder-857075-2-iscsi systemd[1]: Starting Devstack devstack@keystone.service... Sep 13 03:06:28.464257 dv-cinder-857075-2-iscsi devstack@keystone.service[81385]: [uWSGI] getting INI configuration from /etc/keystone/keystone-uwsgi-public.ini Sep 13 03:06:28.466246 dv-cinder-857075-2-iscsi devstack@keystone.service[81385]: *** Starting uWSGI 2.0.18-debian (64bit) on [Tue Sep 13 03:06:28 2022] *** Sep 13 03:06:28.466246 dv-cinder-857075-2-iscsi devstack@keystone.service[81385]: compiled with version: 10.0.1 20200405 (experimental) [master revision 0be9efad938:fcb98e4978a:705510a708d3642c9c962beb663c476167e4e8a4] on 11 April 2020 11:15:55 Sep 13 03:06:28.466246 dv-cinder-857075-2-iscsi devstack@keystone.service[81385]: os: Linux-5.4.0-72-generic #80-Ubuntu SMP Mon Apr 12 17:35:00 UTC 2021 Sep 13 03:06:28.466246 dv-cinder-857075-2-iscsi devstack@keystone.service[81385]: nodename: dv-cinder-857075-2-iscsi Sep 13 03:06:28.466246 dv-cinder-857075-2-iscsi devstack@keystone.service[81385]: machine: x86_64 Sep 13 03:06:28.466246 dv-cinder-857075-2-iscsi devstack@keystone.service[81385]: clock source: unix Sep 13 03:06:28.466246 dv-cinder-857075-2-iscsi devstack@keystone.service[81385]: pcre jit disabled Sep 13 03:06:28.466741 dv-cinder-857075-2-iscsi devstack@keystone.service[81385]: detected number of CPU cores: 4 Sep 13 03:06:28.466741 dv-cinder-857075-2-iscsi devstack@keystone.service[81385]: current working directory: / Sep 13 03:06:28.466741 dv-cinder-857075-2-iscsi devstack@keystone.service[81385]: detected binary path: /usr/bin/uwsgi-core Sep 13 03:06:28.466741 dv-cinder-857075-2-iscsi devstack@keystone.service[81385]: your processes number limit is 47680 Sep 13 03:06:28.466741 dv-cinder-857075-2-iscsi devstack@keystone.service[81385]: your memory page size is 4096 bytes Sep 13 03:06:28.466741 dv-cinder-857075-2-iscsi devstack@keystone.service[81385]: detected max file descriptor number: 2048 Sep 13 03:06:28.466741 dv-cinder-857075-2-iscsi devstack@keystone.service[81385]: lock engine: pthread robust mutexes Sep 13 03:06:28.466741 dv-cinder-857075-2-iscsi devstack@keystone.service[81385]: thunder lock: enabled Sep 13 03:06:28.467679 dv-cinder-857075-2-iscsi devstack@keystone.service[81385]: uwsgi socket 0 bound to UNIX address /var/run/uwsgi/keystone-wsgi-public.socket fd 4 Sep 13 03:06:28.467773 dv-cinder-857075-2-iscsi devstack@keystone.service[81385]: Python version: 3.8.10 (default, Jun 22 2022, 20:18:18) [GCC 9.4.0] Sep 13 03:06:28.506857 dv-cinder-857075-2-iscsi devstack@keystone.service[81385]: Python main interpreter initialized at 0x55ed3432b9c0 Sep 13 03:06:28.506857 dv-cinder-857075-2-iscsi devstack@keystone.service[81385]: python threads support enabled Sep 13 03:06:28.506857 dv-cinder-857075-2-iscsi devstack@keystone.service[81385]: your server socket listen backlog is limited to 100 connections Sep 13 03:06:28.506857 dv-cinder-857075-2-iscsi devstack@keystone.service[81385]: your mercy for graceful operations on workers is 90 seconds Sep 13 03:06:28.507268 dv-cinder-857075-2-iscsi devstack@keystone.service[81385]: mapped 403077 bytes (393 KB) for 2 cores Sep 13 03:06:28.507352 dv-cinder-857075-2-iscsi devstack@keystone.service[81385]: *** Operational MODE: preforking *** Sep 13 03:06:28.507581 dv-cinder-857075-2-iscsi devstack@keystone.service[81385]: *** uWSGI is running in multiple interpreter mode *** Sep 13 03:06:28.507581 dv-cinder-857075-2-iscsi devstack@keystone.service[81385]: spawned uWSGI master process (pid: 81385) Sep 13 03:06:28.507781 dv-cinder-857075-2-iscsi systemd[1]: Started Devstack devstack@keystone.service. Sep 13 03:06:28.508861 dv-cinder-857075-2-iscsi devstack@keystone.service[81385]: spawned uWSGI worker 1 (pid: 81399, cores: 1) Sep 13 03:06:28.508861 dv-cinder-857075-2-iscsi devstack@keystone.service[81385]: spawned uWSGI worker 2 (pid: 81400, cores: 1) Sep 13 03:06:28.508861 dv-cinder-857075-2-iscsi devstack@keystone.service[81385]: running "unix_signal:15 gracefully_kill_them_all" (master-start)... Sep 13 03:06:32.790005 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Sep 13 03:06:32.790005 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: from cryptography.utils import int_from_bytes Sep 13 03:06:32.790840 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Sep 13 03:06:32.790840 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: from cryptography.utils import int_from_bytes Sep 13 03:06:32.790964 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Sep 13 03:06:32.790964 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: from cryptography.utils import int_from_bytes Sep 13 03:06:32.791370 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Sep 13 03:06:32.791370 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: from cryptography.utils import int_from_bytes Sep 13 03:06:32.840397 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Sep 13 03:06:32.840588 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Sep 13 03:06:32.840683 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Sep 13 03:06:32.840770 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] command line args: [] {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Sep 13 03:06:32.840851 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Sep 13 03:06:32.840982 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] command line args: [] {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Sep 13 03:06:32.841066 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Sep 13 03:06:32.841188 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] ================================================================================ {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Sep 13 03:06:32.841274 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Sep 13 03:06:32.841414 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] ================================================================================ {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Sep 13 03:06:32.841500 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] admin_token = **** {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 03:06:32.841667 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] config_dir = [] {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 03:06:32.841758 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] admin_token = **** {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 03:06:32.841858 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 03:06:32.841944 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] config_dir = [] {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 03:06:32.842042 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] config_source = [] {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 03:06:32.842128 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 03:06:32.842221 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] debug = True {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 03:06:32.842312 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] config_source = [] {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 03:06:32.842417 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] debug = True {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 03:06:32.842496 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 03:06:32.842758 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 03:06:32.842903 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 03:06:32.843006 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 03:06:32.843109 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] insecure_debug = False {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 03:06:32.843221 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] insecure_debug = False {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 03:06:32.843325 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 03:06:32.843422 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 03:06:32.843529 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 03:06:32.843627 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 03:06:32.843736 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] list_limit = None {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 03:06:32.843832 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] list_limit = None {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 03:06:32.843934 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] log_config_append = None {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 03:06:32.844032 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] log_config_append = None {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 03:06:32.844144 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 03:06:32.844247 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 03:06:32.844349 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] log_dir = None {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 03:06:32.844445 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] log_dir = None {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 03:06:32.844552 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] log_file = None {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 03:06:32.844654 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] log_file = None {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 03:06:32.844781 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 03:06:32.844859 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 03:06:32.844982 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 03:06:32.845059 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 03:06:32.845181 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 03:06:32.845258 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 03:06:32.845382 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 03:06:32.845461 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 03:06:32.845586 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 03:06:32.845664 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 03:06:32.845780 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 03:06:32.845869 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 03:06:32.845948 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 03:06:32.846027 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 03:06:32.846144 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 03:06:32.846220 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 03:06:32.846348 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 03:06:32.846433 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 03:06:32.846555 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 03:06:32.847218 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 03:06:32.847353 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 03:06:32.847431 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 03:06:32.847548 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 03:06:32.847645 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 03:06:32.847720 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 03:06:32.847845 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 03:06:32.847921 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] notification_format = cadf {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 03:06:32.847993 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] notification_format = cadf {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 03:06:32.848148 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 03:06:32.848237 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 03:06:32.848632 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.18/identity {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 03:06:32.848718 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.18/identity {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 03:06:32.848803 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] publish_errors = False {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 03:06:32.848882 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] publish_errors = False {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 03:06:32.849034 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 03:06:32.849117 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 03:06:32.849261 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 03:06:32.849348 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 03:06:32.849427 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 03:06:32.849508 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 03:06:32.849668 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 03:06:32.849760 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 03:06:32.849899 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 03:06:32.849993 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 03:06:32.850130 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] standard_threads = False {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 03:06:32.850214 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] standard_threads = False {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 03:06:32.850357 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] strict_password_check = False {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 03:06:32.850447 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] strict_password_check = False {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 03:06:32.850595 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 03:06:32.850691 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 03:06:32.850774 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] use_eventlog = False {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 03:06:32.850853 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] use_eventlog = False {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 03:06:32.850954 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] use_journal = False {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 03:06:32.851083 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] use_journal = False {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 03:06:32.851159 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] use_json = False {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 03:06:32.851236 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] use_json = False {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 03:06:32.851314 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] use_stderr = False {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 03:06:32.851386 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] use_stderr = False {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 03:06:32.851457 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] use_syslog = False {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 03:06:32.851539 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] use_syslog = False {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 03:06:32.851658 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] watch_log_file = False {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 03:06:32.851737 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] watch_log_file = False {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 03:06:32.851907 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 03:06:32.851984 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 03:06:32.852144 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 03:06:32.852234 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 03:06:32.852396 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 03:06:32.852473 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 03:06:32.852630 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 03:06:32.852708 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 03:06:32.852872 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 03:06:32.852957 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 03:06:32.853105 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 03:06:32.853182 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 03:06:32.853347 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 03:06:32.853430 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 03:06:32.853578 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 03:06:32.853667 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 03:06:32.853819 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 03:06:32.853896 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 03:06:32.854060 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 03:06:32.854137 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 03:06:32.854303 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 03:06:32.854392 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 03:06:32.854533 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 03:06:32.854634 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 03:06:32.854806 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 03:06:32.854891 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 03:06:32.855038 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 03:06:32.855135 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 03:06:32.855280 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 03:06:32.855363 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 03:06:32.855522 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 03:06:32.855601 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 03:06:32.855751 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 03:06:32.855843 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 03:06:32.855999 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 03:06:32.856083 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 03:06:32.856246 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 03:06:32.856322 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 03:06:32.856484 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 03:06:32.856643 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 03:06:32.856719 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 03:06:32.856796 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 03:06:32.856952 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 03:06:32.857037 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 03:06:32.857209 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 03:06:32.857287 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 03:06:32.857455 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 03:06:32.857538 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 03:06:32.857698 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 03:06:32.857775 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 03:06:32.857925 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 03:06:32.858003 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 03:06:32.858150 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 03:06:32.858226 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 03:06:32.858376 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 03:06:32.858458 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 03:06:32.858623 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 03:06:32.858714 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 03:06:32.858872 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 03:06:32.858950 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 03:06:32.859113 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 03:06:32.859191 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 03:06:32.859342 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 03:06:32.859488 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 03:06:32.859565 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 03:06:32.859642 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 03:06:32.859800 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 03:06:32.859879 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 03:06:32.860044 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 03:06:32.860122 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 03:06:32.860275 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 03:06:32.860352 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 03:06:32.860522 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 03:06:32.860598 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 03:06:32.860748 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 03:06:32.860826 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 03:06:32.860979 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 03:06:32.861056 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 03:06:32.861209 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 03:06:32.861286 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 03:06:32.861436 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 03:06:32.861514 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 03:06:32.861661 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 03:06:32.861737 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 03:06:32.861888 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 03:06:32.861964 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 03:06:32.862113 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 03:06:32.862188 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 03:06:32.862367 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 03:06:32.862453 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 03:06:32.862625 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 03:06:32.862746 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 03:06:32.862860 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 03:06:32.863093 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 03:06:32.863321 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 03:06:32.863552 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 03:06:32.863783 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 03:06:32.864028 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 03:06:32.864255 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 03:06:32.864487 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 03:06:32.864719 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 03:06:32.864951 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 03:06:32.865182 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 03:06:32.865257 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 03:06:32.865413 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 03:06:32.865522 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 03:06:32.865640 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 03:06:32.865884 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 03:06:32.866200 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 03:06:32.866346 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 03:06:32.866570 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 03:06:32.866850 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 03:06:32.867085 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 03:06:32.867313 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 03:06:32.867547 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 03:06:32.867781 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 03:06:32.867876 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 03:06:32.868046 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 03:06:32.868134 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 03:06:32.868281 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 03:06:32.868525 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 03:06:32.868767 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 03:06:32.869001 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 03:06:32.869177 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 03:06:32.869261 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 03:06:32.869446 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 03:06:32.869526 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 03:06:32.869686 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] cache.enabled = True {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 03:06:32.869764 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 03:06:32.869921 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 03:06:32.870006 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 03:06:32.870159 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 03:06:32.870291 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 03:06:32.870408 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 03:06:32.870506 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 03:06:32.870649 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 03:06:32.870801 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 03:06:32.870928 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 03:06:32.871056 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] cache.memcache_password = {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 03:06:32.871181 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 03:06:32.871312 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 03:06:32.871450 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 03:06:32.871575 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 03:06:32.871715 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 03:06:32.871842 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 03:06:32.871976 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 03:06:32.872103 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 03:06:32.872252 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] cache.memcache_sasl_enabled = False {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 03:06:32.872377 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 03:06:32.872505 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 03:06:32.872642 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 03:06:32.872771 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 03:06:32.872899 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 03:06:32.872899 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 03:06:32.873151 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] cache.memcache_username = {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 03:06:32.873278 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 03:06:32.873406 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 03:06:32.873544 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 03:06:32.873670 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 03:06:32.873670 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 03:06:32.873919 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 03:06:32.874046 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 03:06:32.874172 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 03:06:32.874298 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 03:06:32.874426 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 03:06:32.874561 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 03:06:32.874759 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 03:06:32.874896 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 03:06:32.875024 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 03:06:32.875152 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 03:06:32.875293 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 03:06:32.875430 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 03:06:32.875558 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 03:06:32.875686 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 03:06:32.875812 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] cache.enabled = True {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 03:06:32.875941 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 03:06:32.875941 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 03:06:32.876199 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 03:06:32.876324 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 03:06:32.876454 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 03:06:32.876583 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 03:06:32.876716 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 03:06:32.876843 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 03:06:32.877048 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 03:06:32.877189 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 03:06:32.877317 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] cache.memcache_password = {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 03:06:32.877444 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 03:06:32.877573 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 03:06:32.877709 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 03:06:32.877850 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 03:06:32.877986 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] auth.external = None {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 03:06:32.878113 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 03:06:32.878240 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] auth.mapped = None {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 03:06:32.878368 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 03:06:32.878500 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 03:06:32.878651 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] cache.memcache_sasl_enabled = False {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 03:06:32.878796 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 03:06:32.878935 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 03:06:32.879064 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] auth.password = None {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 03:06:32.879190 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 03:06:32.879317 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] auth.token = None {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 03:06:32.879444 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] cache.memcache_username = {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 03:06:32.879581 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 03:06:32.879709 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 03:06:32.879836 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] catalog.caching = True {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 03:06:32.879970 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 03:06:32.880097 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 03:06:32.880180 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 03:06:32.880358 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 03:06:32.880485 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 03:06:32.880612 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 03:06:32.880749 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 03:06:32.880876 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 03:06:32.881031 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 03:06:32.881171 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 03:06:32.881299 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 03:06:32.881428 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] credential.caching = True {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 03:06:32.881555 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 03:06:32.881696 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] credential.driver = sql {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 03:06:32.881826 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 03:06:32.881959 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 03:06:32.882088 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 03:06:32.882224 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 03:06:32.882353 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 03:06:32.882480 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 03:06:32.882661 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 03:06:32.882661 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 03:06:32.882905 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 03:06:32.882905 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 03:06:32.883160 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 03:06:32.883160 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 03:06:32.883400 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 03:06:32.883400 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 03:06:32.883642 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 03:06:32.883781 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 03:06:32.883911 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 03:06:32.884039 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 03:06:32.884167 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 03:06:32.884167 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] auth.external = None {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 03:06:32.884427 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 03:06:32.884563 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] auth.mapped = None {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 03:06:32.884692 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 03:06:32.884819 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 03:06:32.884950 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 03:06:32.885085 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 03:06:32.885214 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 03:06:32.885350 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] auth.password = None {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 03:06:32.885478 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 03:06:32.885605 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] auth.token = None {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 03:06:32.885733 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] federation.caching = True {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 03:06:32.885860 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 03:06:32.885988 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 03:06:32.886139 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] catalog.caching = True {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 03:06:32.886269 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] federation.driver = sql {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 03:06:32.886397 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 03:06:32.886524 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 03:06:32.886665 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 03:06:32.886791 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 03:06:32.886934 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 03:06:32.887066 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 03:06:32.887205 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 03:06:32.887334 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 03:06:32.887464 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 03:06:32.887593 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 03:06:32.887731 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] credential.caching = True {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 03:06:32.887868 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 03:06:32.887998 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] credential.driver = sql {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 03:06:32.888128 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 03:06:32.888257 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 03:06:32.888388 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 03:06:32.888523 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 03:06:32.888651 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 03:06:32.888779 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 03:06:32.888908 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] identity.caching = True {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 03:06:32.889037 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 03:06:32.889167 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 03:06:32.889304 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 03:06:32.889433 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 03:06:32.889561 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 03:06:32.889700 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 03:06:32.889828 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 03:06:32.889965 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 03:06:32.890094 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 03:06:32.890224 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] identity.driver = sql {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 03:06:32.890349 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 03:06:32.890476 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 03:06:32.890601 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 03:06:32.890767 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 03:06:32.890896 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 03:06:32.891026 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 03:06:32.891156 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 03:06:32.891284 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 03:06:32.891428 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 03:06:32.891568 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 03:06:32.891568 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 03:06:32.891810 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 03:06:32.891945 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 03:06:32.892083 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] federation.caching = True {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 03:06:32.892212 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 03:06:32.892417 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 03:06:32.892547 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 03:06:32.892681 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] federation.driver = sql {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 03:06:32.892810 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 03:06:32.892937 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 03:06:32.892937 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 03:06:32.893186 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 03:06:32.893326 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 03:06:32.893957 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 03:06:32.894085 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 03:06:32.894216 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 03:06:32.894347 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 03:06:32.894484 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 03:06:32.894632 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 03:06:32.894758 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 03:06:32.894887 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 03:06:32.895017 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 03:06:32.895145 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 03:06:32.895285 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 03:06:32.895416 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 03:06:32.895566 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 03:06:32.895696 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] identity.caching = True {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 03:06:32.895816 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 03:06:32.895942 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 03:06:32.896076 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 03:06:32.896207 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 03:06:32.896333 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 03:06:32.896459 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 03:06:32.896590 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 03:06:32.896714 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 03:06:32.896853 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 03:06:32.896983 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] identity.driver = sql {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 03:06:32.897107 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 03:06:32.897237 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 03:06:32.897373 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 03:06:32.897499 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 03:06:32.897631 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 03:06:32.897762 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 03:06:32.897887 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 03:06:32.898012 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 03:06:32.898142 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 03:06:32.898266 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 03:06:32.898400 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 03:06:32.898532 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 03:06:32.898674 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 03:06:32.898798 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 03:06:32.898928 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] ldap.password = **** {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 03:06:32.899057 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 03:06:32.899210 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 03:06:32.899341 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 03:06:32.899471 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 03:06:32.899608 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 03:06:32.899738 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 03:06:32.899867 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 03:06:32.900006 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 03:06:32.900136 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 03:06:32.900266 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 03:06:32.900382 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 03:06:32.900509 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 03:06:32.900637 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 03:06:32.900771 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 03:06:32.900907 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 03:06:32.901034 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 03:06:32.901160 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 03:06:32.901285 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 03:06:32.901419 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 03:06:32.901546 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 03:06:32.901672 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 03:06:32.901798 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 03:06:32.901924 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 03:06:32.902049 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 03:06:32.902183 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 03:06:32.902309 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 03:06:32.902434 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 03:06:32.902560 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 03:06:32.902737 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 03:06:32.902867 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] ldap.user = None {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 03:06:32.903007 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 03:06:32.903136 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 03:06:32.903265 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 03:06:32.903394 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 03:06:32.903522 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 03:06:32.903650 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 03:06:32.903788 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 03:06:32.903917 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 03:06:32.904045 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 03:06:32.904173 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 03:06:32.904308 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 03:06:32.904446 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 03:06:32.904582 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 03:06:32.904711 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 03:06:32.904841 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 03:06:32.904968 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 03:06:32.905096 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] ldap.password = **** {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 03:06:32.905225 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 03:06:32.905361 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 03:06:32.905489 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 03:06:32.905616 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 03:06:32.905751 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 03:06:32.905879 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 03:06:32.906007 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 03:06:32.906146 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 03:06:32.906285 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 03:06:32.906413 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 03:06:32.906541 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 03:06:32.906691 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 03:06:32.906821 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 03:06:32.906961 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 03:06:32.907092 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 03:06:32.907221 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 03:06:32.907351 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 03:06:32.907479 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 03:06:32.907608 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 03:06:32.907746 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 03:06:32.907874 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 03:06:32.908005 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 03:06:32.908145 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 03:06:32.908274 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 03:06:32.908403 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 03:06:32.908540 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 03:06:32.908668 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 03:06:32.908798 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 03:06:32.908928 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 03:06:32.909064 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] ldap.user = None {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 03:06:32.909755 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 03:06:32.909894 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 03:06:32.910042 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 03:06:32.910173 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 03:06:32.910303 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 03:06:32.910459 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 03:06:32.910587 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] policy.driver = sql {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 03:06:32.910778 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 03:06:32.910908 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 03:06:32.911036 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 03:06:32.911167 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 03:06:32.911294 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 03:06:32.911422 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 03:06:32.911557 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 03:06:32.911688 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] receipt.caching = True {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 03:06:32.911815 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 03:06:32.911944 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 03:06:32.912071 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 03:06:32.912201 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 03:06:32.912345 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 03:06:32.912476 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 03:06:32.912603 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 03:06:32.912731 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 03:06:32.912859 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 03:06:32.912986 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 03:06:32.913123 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 03:06:32.913253 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] resource.caching = True {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 03:06:32.913380 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 03:06:32.913507 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 03:06:32.913634 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 03:06:32.913762 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] resource.driver = sql {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 03:06:32.913900 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 03:06:32.914028 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 03:06:32.914165 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 03:06:32.914295 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 03:06:32.914422 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 03:06:32.914547 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 03:06:32.914700 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 03:06:32.914828 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] revoke.caching = True {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 03:06:32.914959 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 03:06:32.915084 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 03:06:32.915214 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 03:06:32.915340 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 03:06:32.915479 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 03:06:32.915609 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] role.cache_time = None {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 03:06:32.915734 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 03:06:32.915865 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] role.caching = True {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 03:06:32.916000 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 03:06:32.916131 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] role.driver = sql {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 03:06:32.916269 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 03:06:32.916396 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] role.list_limit = None {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 03:06:32.916526 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 03:06:32.916651 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 03:06:32.916801 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] policy.driver = sql {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 03:06:32.916930 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 03:06:32.917019 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 03:06:32.917096 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 03:06:32.917171 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 03:06:32.917244 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 03:06:32.917317 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 03:06:32.917398 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 03:06:32.917474 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] receipt.caching = True {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 03:06:32.917547 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 03:06:32.917620 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 03:06:32.917693 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 03:06:32.917766 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 03:06:32.917916 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 03:06:32.917995 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 03:06:32.918069 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 03:06:32.918142 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 03:06:32.918216 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 03:06:32.918289 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 03:06:32.918374 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 03:06:32.918451 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] resource.caching = True {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 03:06:32.918524 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 03:06:32.918598 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 03:06:32.918719 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 03:06:32.918793 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] resource.driver = sql {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 03:06:32.918793 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 03:06:32.918940 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 03:06:32.919014 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 03:06:32.919087 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 03:06:32.919159 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 03:06:32.919231 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 03:06:32.919312 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] revoke.caching = True {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 03:06:32.919388 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 03:06:32.919461 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 03:06:32.919534 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 03:06:32.919607 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 03:06:32.919680 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 03:06:32.919761 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] role.cache_time = None {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 03:06:32.919836 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 03:06:32.919910 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] role.caching = True {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 03:06:32.919982 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 03:06:32.920055 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] role.driver = sql {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 03:06:32.920128 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 03:06:32.920210 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] role.list_limit = None {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 03:06:32.920285 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 03:06:32.920359 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 03:06:32.920431 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 03:06:32.920552 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 03:06:32.920631 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 03:06:32.920703 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 03:06:32.920814 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 03:06:32.920895 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 03:06:32.920969 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 03:06:32.921042 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 03:06:32.921116 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 03:06:32.921198 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 03:06:32.921273 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 03:06:32.921347 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 03:06:32.921420 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 03:06:32.921493 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 03:06:32.921566 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 03:06:32.921647 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 03:06:32.921722 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] token.cache_time = None {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 03:06:32.921795 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 03:06:32.921867 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] token.caching = True {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 03:06:32.921940 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 03:06:32.922022 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 03:06:32.922098 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 03:06:32.922171 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] token.provider = fernet {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 03:06:32.922244 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 03:06:32.922317 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 03:06:32.922317 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 03:06:32.922457 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 03:06:32.922532 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 03:06:32.922624 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 03:06:32.922701 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 03:06:32.922774 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 03:06:32.922848 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 03:06:32.922933 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 03:06:32.923009 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 03:06:32.923082 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 03:06:32.923155 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] trust.driver = sql {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 03:06:32.923227 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 03:06:32.923300 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 03:06:32.923381 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 03:06:32.923457 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 03:06:32.923530 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 03:06:32.923603 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 03:06:32.923676 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 03:06:32.923748 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 03:06:32.923829 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 03:06:32.923904 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 03:06:32.923977 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 03:06:32.924050 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 03:06:32.924136 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 03:06:32.924219 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 03:06:32.924294 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 03:06:32.924375 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 03:06:32.924450 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 03:06:32.924523 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] database.connection = **** {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 03:06:32.924595 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 03:06:32.924689 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 03:06:32.924766 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 03:06:32.924848 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 03:06:32.924922 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 03:06:32.924995 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 03:06:32.925068 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 03:06:32.925141 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 03:06:32.925214 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] token.cache_time = None {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 03:06:32.925295 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 03:06:32.925370 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] token.caching = True {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 03:06:32.925444 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 03:06:32.925517 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 03:06:32.925589 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 03:06:32.925662 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] token.provider = fernet {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 03:06:32.925744 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 03:06:32.925819 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 03:06:32.925893 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 03:06:32.925955 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 03:06:32.926015 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 03:06:32.926075 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 03:06:32.926142 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 03:06:32.926211 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 03:06:32.926326 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 03:06:32.926408 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 03:06:32.926481 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 03:06:32.926554 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 03:06:32.926654 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 03:06:32.926733 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] trust.driver = sql {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 03:06:32.926807 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 03:06:32.926881 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 03:06:32.926954 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 03:06:32.927027 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 03:06:32.927109 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 03:06:32.927184 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 03:06:32.927257 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 03:06:32.927330 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 03:06:32.927403 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Sep 13 03:06:32.927476 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 03:06:32.927476 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 03:06:32.927476 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 03:06:32.927476 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 03:06:32.927476 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] database.connection = **** {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 03:06:32.927813 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 03:06:32.927813 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 03:06:32.928008 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 03:06:32.928244 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 03:06:32.928476 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 03:06:32.928708 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 03:06:32.928939 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 03:06:32.929170 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 03:06:32.929409 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 03:06:32.929639 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 03:06:32.929880 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 03:06:32.930111 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 03:06:32.930353 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 03:06:32.930584 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 03:06:32.930847 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 03:06:32.931075 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 03:06:32.931311 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 03:06:32.931542 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 03:06:32.931710 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Sep 13 03:06:32.955127 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=81400) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 13 03:06:32.955362 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=81400) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 13 03:06:32.955587 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=81400) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 13 03:06:32.956687 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=81399) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 13 03:06:32.956913 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=81399) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 13 03:06:32.957130 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=81399) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 13 03:06:32.964507 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=81400) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 13 03:06:32.965366 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=81399) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 13 03:06:32.993233 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=81400) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 13 03:06:32.993458 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=81399) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 13 03:06:32.993545 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=81400) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 13 03:06:32.993689 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=81399) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 13 03:06:33.047370 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=81399) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 13 03:06:33.048104 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=81400) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 13 03:06:33.096299 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=81399) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 13 03:06:33.096549 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=81399) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 13 03:06:33.097155 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=81400) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 13 03:06:33.097395 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=81400) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 13 03:06:33.252729 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81399) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 13 03:06:33.253411 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81400) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 13 03:06:33.253517 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=81399) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 13 03:06:33.253596 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=81400) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 13 03:06:33.253697 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=81399) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 13 03:06:33.253777 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=81400) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 13 03:06:33.253852 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=81399) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 13 03:06:33.253928 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=81400) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 13 03:06:33.254070 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=81399) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 13 03:06:33.254156 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=81400) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 13 03:06:33.254282 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=81399) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 13 03:06:33.254366 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=81400) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 13 03:06:33.278335 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81400) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 13 03:06:33.278566 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=81400) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 13 03:06:33.278716 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81399) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 13 03:06:33.278819 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=81400) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 13 03:06:33.278951 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=81399) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 13 03:06:33.279044 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=81400) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 13 03:06:33.279171 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=81399) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 13 03:06:33.279255 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=81400) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 13 03:06:33.279387 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=81399) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 13 03:06:33.279471 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=81400) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 13 03:06:33.279596 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=81399) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 13 03:06:33.279679 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81400) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 13 03:06:33.279803 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=81399) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 13 03:06:33.280010 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81399) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 13 03:06:33.290700 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=81400) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 13 03:06:33.290935 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=81399) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 13 03:06:33.303405 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81400) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 13 03:06:33.303585 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=81400) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 13 03:06:33.303672 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81399) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 13 03:06:33.303815 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=81400) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 13 03:06:33.303943 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=81399) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 13 03:06:33.304028 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=81400) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 13 03:06:33.304104 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=81399) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 13 03:06:33.304245 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=81400) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 13 03:06:33.304338 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=81399) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 13 03:06:33.304461 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=81400) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 13 03:06:33.304545 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=81399) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 13 03:06:33.304676 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81400) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 13 03:06:33.304759 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=81399) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 13 03:06:33.304908 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81399) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 13 03:06:33.321939 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=81400) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 03:06:33.322144 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=81400) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Sep 13 03:06:33.322488 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=81400) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 03:06:33.322649 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=81399) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 03:06:33.322736 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=81400) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Sep 13 03:06:33.322862 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=81399) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Sep 13 03:06:33.322979 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=81400) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 03:06:33.323201 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=81399) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 03:06:33.323283 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=81400) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 03:06:33.323400 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=81399) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Sep 13 03:06:33.323491 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=81400) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 03:06:33.323676 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=81399) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 03:06:33.323759 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=81400) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 03:06:33.323941 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=81399) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 03:06:33.324202 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=81399) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 03:06:33.324457 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=81399) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 03:06:33.333869 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=81400) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 03:06:33.334158 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=81400) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 03:06:33.334431 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=81400) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 03:06:33.334590 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=81399) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 03:06:33.334746 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=81400) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 03:06:33.334888 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=81399) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 03:06:33.335161 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=81399) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 03:06:33.335447 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=81399) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 03:06:33.341704 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `type` (API Prefix: ) [/credentials, /credentials/] {{(pid=81400) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 13 03:06:33.342373 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `type` (API Prefix: ) [/credentials, /credentials/] {{(pid=81399) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 13 03:06:33.344635 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `type` (API Prefix: ) [/domains, /domains/] {{(pid=81400) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 13 03:06:33.344923 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=81400) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 03:06:33.345209 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=81400) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 03:06:33.345294 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `type` (API Prefix: ) [/domains, /domains/] {{(pid=81399) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 13 03:06:33.345497 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=81400) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 03:06:33.345579 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=81399) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 03:06:33.345782 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=81400) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 03:06:33.345866 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=81399) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 03:06:33.346045 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=81400) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 03:06:33.346128 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=81399) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 03:06:33.346317 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=81400) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 03:06:33.346409 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=81399) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 03:06:33.346638 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=81400) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 03:06:33.346723 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=81399) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 03:06:33.346912 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=81400) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 03:06:33.346994 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=81399) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 03:06:33.347186 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=81399) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 03:06:33.347269 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=81400) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 03:06:33.347456 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=81400) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 03:06:33.347537 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=81399) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 03:06:33.347726 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=81399) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 03:06:33.347994 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=81399) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 03:06:33.370093 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=81400) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 03:06:33.370330 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=81399) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 03:06:33.371618 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `type` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=81400) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 13 03:06:33.371822 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `type` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=81399) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 13 03:06:33.371906 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=81400) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 03:06:33.372109 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=81399) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 03:06:33.376641 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `type` (API Prefix: ) [/groups, /groups/] {{(pid=81400) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 13 03:06:33.376733 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `type` (API Prefix: ) [/groups, /groups/] {{(pid=81399) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 13 03:06:33.376927 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=81400) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 03:06:33.377011 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=81399) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 03:06:33.377213 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=81400) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 03:06:33.377297 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=81399) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 03:06:33.384064 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `type` (API Prefix: ) [/limits, /limits/] {{(pid=81399) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 13 03:06:33.384156 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `type` (API Prefix: ) [/limits, /limits/] {{(pid=81400) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 13 03:06:33.384351 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=81399) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 03:06:33.384434 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=81400) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 03:06:33.389881 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `type` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=81399) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 13 03:06:33.389971 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `type` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=81400) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 13 03:06:33.390172 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=81399) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 03:06:33.390255 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=81400) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 03:06:33.390458 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=81399) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 03:06:33.390552 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=81400) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 03:06:33.390803 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=81399) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 03:06:33.390943 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=81400) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 03:06:33.391074 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=81399) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 03:06:33.391203 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=81400) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 03:06:33.391362 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=81399) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 03:06:33.391493 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=81400) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 03:06:33.391623 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=81399) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 03:06:33.391775 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=81400) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 03:06:33.391906 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=81399) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 03:06:33.392035 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=81400) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 03:06:33.407721 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=81400) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 03:06:33.407861 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=81399) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 03:06:33.407991 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=81399) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 03:06:33.408119 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=81400) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 03:06:33.411848 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `type` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=81399) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 13 03:06:33.412005 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `type` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=81400) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 13 03:06:33.415062 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=81400) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 03:06:33.415197 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=81399) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 03:06:33.415360 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=81399) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 03:06:33.415489 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=81400) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 03:06:33.419502 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `type` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=81400) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 13 03:06:33.419635 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `type` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=81399) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 13 03:06:33.422589 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `type` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=81399) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 13 03:06:33.422745 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `type` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=81400) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 13 03:06:33.425880 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=81399) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 03:06:33.425984 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=81400) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 03:06:33.426180 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=81399) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 03:06:33.426263 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=81400) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 03:06:33.426467 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=81399) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 03:06:33.426548 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=81400) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 03:06:33.426777 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=81399) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 03:06:33.426862 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=81400) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 03:06:33.427067 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=81400) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 03:06:33.427149 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=81399) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 03:06:33.427356 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=81400) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 03:06:33.427448 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=81399) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 03:06:33.443741 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `type` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=81399) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 13 03:06:33.443834 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `type` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=81400) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 13 03:06:33.444025 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=81399) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 03:06:33.444108 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=81400) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 03:06:33.444290 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=81399) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 03:06:33.444386 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=81400) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 03:06:33.444553 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=81399) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 03:06:33.444635 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=81400) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 03:06:33.451378 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH2: ['/token' {}] {{(pid=81399) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 03:06:33.451471 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH2: ['/token' {}] {{(pid=81400) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 03:06:33.452910 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=81399) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 03:06:33.453047 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=81400) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 03:06:33.454433 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=81399) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 03:06:33.454515 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=81400) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 03:06:33.454735 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=81400) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 03:06:33.454818 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=81399) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 03:06:33.457960 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `type` (API Prefix: ) [/policies, /policies/] {{(pid=81400) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 13 03:06:33.458087 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `type` (API Prefix: ) [/policies, /policies/] {{(pid=81399) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 13 03:06:33.458255 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=81400) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 03:06:33.458372 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=81399) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 03:06:33.458542 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=81400) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 03:06:33.458680 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=81399) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 03:06:33.458855 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=81400) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 03:06:33.458966 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=81399) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 03:06:33.459241 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=81400) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 03:06:33.459422 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=81399) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 03:06:33.470285 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `type` (API Prefix: ) [/projects, /projects/] {{(pid=81400) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 13 03:06:33.470432 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `type` (API Prefix: ) [/projects, /projects/] {{(pid=81399) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 13 03:06:33.470573 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=81400) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 03:06:33.470741 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=81399) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 03:06:33.470881 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=81400) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 03:06:33.471024 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=81399) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 03:06:33.471169 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=81400) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 03:06:33.471309 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=81399) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 03:06:33.471599 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=81400) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 03:06:33.471712 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=81399) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 03:06:33.471888 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=81400) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 03:06:33.471993 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=81399) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 03:06:33.472180 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=81400) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 03:06:33.472262 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=81399) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 03:06:33.487467 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `type` (API Prefix: ) [/regions, /regions/] {{(pid=81400) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 13 03:06:33.487573 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `type` (API Prefix: ) [/regions, /regions/] {{(pid=81399) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 13 03:06:33.490380 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `type` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=81400) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 13 03:06:33.490465 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `type` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=81399) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 13 03:06:33.493287 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=81400) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 03:06:33.493374 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=81399) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 03:06:33.494796 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=81400) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 03:06:33.495081 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=81399) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 03:06:33.496272 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `type` (API Prefix: ) [/roles, /roles/] {{(pid=81400) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 13 03:06:33.496566 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=81400) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 03:06:33.496657 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `type` (API Prefix: ) [/roles, /roles/] {{(pid=81399) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 13 03:06:33.496924 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=81399) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 03:06:33.497015 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=81400) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 03:06:33.497339 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=81399) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 03:06:33.503857 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=81400) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 03:06:33.504221 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=81399) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 03:06:33.505326 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `type` (API Prefix: ) [/services, /services/] {{(pid=81400) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 13 03:06:33.505667 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `type` (API Prefix: ) [/services, /services/] {{(pid=81399) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 13 03:06:33.508259 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=81400) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 03:06:33.508576 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=81399) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 03:06:33.508658 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=81400) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 03:06:33.508937 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=81400) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 03:06:33.509020 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=81399) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 03:06:33.509220 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=81400) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 03:06:33.509377 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=81399) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 03:06:33.509511 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=81399) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 03:06:33.517006 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `type` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=81400) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 13 03:06:33.517275 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `type` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=81399) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 13 03:06:33.517372 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=81400) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 03:06:33.517565 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=81399) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 03:06:33.517647 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=81400) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 03:06:33.517860 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=81399) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 03:06:33.524986 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `type` (API Prefix: ) [/users, /users/] {{(pid=81400) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 13 03:06:33.525274 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=81400) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 03:06:33.525358 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `type` (API Prefix: ) [/users, /users/] {{(pid=81399) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 13 03:06:33.525562 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=81400) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 03:06:33.525646 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=81399) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 03:06:33.525842 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=81399) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 03:06:33.525935 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=81400) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 03:06:33.526121 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=81400) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 03:06:33.526205 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=81399) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 03:06:33.526392 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=81399) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 03:06:33.526474 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=81400) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 03:06:33.526690 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=81399) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 03:06:33.526788 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=81400) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 03:06:33.526966 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=81400) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 03:06:33.527058 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=81399) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 03:06:33.527238 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=81400) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 03:06:33.527331 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=81399) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 03:06:33.527506 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=81399) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 03:06:33.527587 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=81400) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 03:06:33.527787 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=81399) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 03:06:33.527869 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=81400) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 03:06:33.528061 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=81400) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 03:06:33.528143 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=81399) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 03:06:33.528330 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=81399) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 03:06:33.528410 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=81400) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 03:06:33.528604 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=81399) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 03:06:33.528686 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=81400) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 03:06:33.528872 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=81399) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 03:06:33.563117 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: WSGI app 0 (mountpoint='') ready in 5 seconds on interpreter 0x55ed3432b9c0 pid: 81400 (default app) Sep 13 03:06:33.563218 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: WSGI app 0 (mountpoint='') ready in 5 seconds on interpreter 0x55ed3432b9c0 pid: 81399 (default app) Sep 13 03:06:33.566907 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8e02795-f8ba-4b90-bb86-97585a1e7735 None None] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:06:33.567203 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8e02795-f8ba-4b90-bb86-97585a1e7735 None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:06:33.567476 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8e02795-f8ba-4b90-bb86-97585a1e7735 None None] PATH_INFO: `/v3` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:06:33.571637 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 1/1] 10.222.0.18 () {50 vars in 791 bytes} [Tue Sep 13 03:06:33 2022] GET /identity/v3/ => generated 255 bytes in 8 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 03:06:40.312603 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d4ef90-41cd-4b66-b9da-e934dd524b40 None None] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:06:40.313111 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d4ef90-41cd-4b66-b9da-e934dd524b40 None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:06:40.313213 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d4ef90-41cd-4b66-b9da-e934dd524b40 None None] PATH_INFO: `/` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:06:40.318047 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 1/2] 10.222.0.18 () {54 vars in 927 bytes} [Tue Sep 13 03:06:40 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 13 03:06:40.323636 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a157e34e-063a-41d0-ae0e-be9a07c17c03 None None] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:06:40.324057 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a157e34e-063a-41d0-ae0e-be9a07c17c03 None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:06:40.324202 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a157e34e-063a-41d0-ae0e-be9a07c17c03 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:06:40.393369 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: INFO dbcounter [None req-a157e34e-063a-41d0-ae0e-be9a07c17c03 None None] Registered counter for database keystone Sep 13 03:06:40.412116 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG oslo_db.sqlalchemy.engines [None req-a157e34e-063a-41d0-ae0e-be9a07c17c03 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=81400) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Sep 13 03:06:40.418237 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writer thread running {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Sep 13 03:06:40.595059 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG passlib.handlers.bcrypt [None req-a157e34e-063a-41d0-ae0e-be9a07c17c03 None None] detected 'bcrypt' backend, version '4.0.0' {{(pid=81400) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Sep 13 03:06:40.595363 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG passlib.handlers.bcrypt [None req-a157e34e-063a-41d0-ae0e-be9a07c17c03 None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=81400) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Sep 13 03:06:40.654598 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.auth.core [None req-a157e34e-063a-41d0-ae0e-be9a07c17c03 None None] MFA Rules not processed for user `49f280ab086b4610855c4ffa378517d9`. Rule list: `[]` (Enabled: `True`). {{(pid=81400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 03:06:40.656215 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-a157e34e-063a-41d0-ae0e-be9a07c17c03 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:06:40.785041 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 2/3] 10.222.0.18 () {58 vars in 1024 bytes} [Tue Sep 13 03:06:40 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 462 msecs (HTTP/1.1 201) 6 headers in 384 bytes (2 switches on core 0) Sep 13 03:06:41.232692 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00d4ef90-41cd-4b66-b9da-e934dd524b40 None None] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:06:41.301213 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: INFO dbcounter [None req-00d4ef90-41cd-4b66-b9da-e934dd524b40 None None] Registered counter for database keystone Sep 13 03:06:41.323183 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG oslo_db.sqlalchemy.engines [None req-00d4ef90-41cd-4b66-b9da-e934dd524b40 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=81399) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Sep 13 03:06:41.329411 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writer thread running {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Sep 13 03:06:41.447228 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00d4ef90-41cd-4b66-b9da-e934dd524b40 None None] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:06:41.451507 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22962abb-9c46-48b9-b9ea-d2b0fe582028 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76c5181fc7be45279ac00017bb4640b3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:06:41.452127 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22962abb-9c46-48b9-b9ea-d2b0fe582028 admin admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:06:41.452409 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22962abb-9c46-48b9-b9ea-d2b0fe582028 admin admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:06:41.452680 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22962abb-9c46-48b9-b9ea-d2b0fe582028 admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:06:41.460814 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22962abb-9c46-48b9-b9ea-d2b0fe582028 admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:06:41.512050 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: WARNING keystone.common.rbac_enforcer.enforcer [None req-22962abb-9c46-48b9-b9ea-d2b0fe582028 admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Sep 13 03:06:41.512399 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22962abb-9c46-48b9-b9ea-d2b0fe582028 admin admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:06:41.521690 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 2/4] 10.222.0.18 () {58 vars in 1324 bytes} [Tue Sep 13 03:06:41 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 290 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Sep 13 03:06:41.527610 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a157e34e-063a-41d0-ae0e-be9a07c17c03 None None] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:06:41.544602 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a157e34e-063a-41d0-ae0e-be9a07c17c03 None None] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:06:41.549025 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e7fa7de-b856-4a90-9dd4-415fcf969734 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76c5181fc7be45279ac00017bb4640b3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:06:41.550755 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e7fa7de-b856-4a90-9dd4-415fcf969734 admin admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:06:41.551040 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e7fa7de-b856-4a90-9dd4-415fcf969734 admin admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:06:41.551314 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e7fa7de-b856-4a90-9dd4-415fcf969734 admin admin] PATH_INFO: `/v3/projects/76c5181fc7be45279ac00017bb4640b3` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:06:41.552910 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e7fa7de-b856-4a90-9dd4-415fcf969734 admin admin] RBAC: Authorizing `identity:get_project(project_id=76c5181fc7be45279ac00017bb4640b3)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:06:41.604901 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: WARNING keystone.common.rbac_enforcer.enforcer [None req-9e7fa7de-b856-4a90-9dd4-415fcf969734 admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Sep 13 03:06:41.605190 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e7fa7de-b856-4a90-9dd4-415fcf969734 admin admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:06:41.609723 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 3/5] 10.222.0.18 () {56 vars in 1207 bytes} [Tue Sep 13 03:06:41 2022] GET /identity/v3/projects/76c5181fc7be45279ac00017bb4640b3 => generated 344 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 03:06:43.731113 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-606faa1e-da34-4468-b4a7-76e76ad6adc9 None None] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:06:43.731670 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-606faa1e-da34-4468-b4a7-76e76ad6adc9 None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:06:43.731670 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-606faa1e-da34-4468-b4a7-76e76ad6adc9 None None] PATH_INFO: `/` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:06:43.735748 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 3/6] 10.222.0.18 () {54 vars in 927 bytes} [Tue Sep 13 03:06:43 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 03:06:43.741126 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74bf636f-3db3-46db-b982-e7930db763d4 None None] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:06:43.741525 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74bf636f-3db3-46db-b982-e7930db763d4 None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:06:43.741675 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74bf636f-3db3-46db-b982-e7930db763d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:06:43.761772 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.auth.core [None req-74bf636f-3db3-46db-b982-e7930db763d4 None None] MFA Rules not processed for user `49f280ab086b4610855c4ffa378517d9`. Rule list: `[]` (Enabled: `True`). {{(pid=81400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 03:06:43.763306 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-74bf636f-3db3-46db-b982-e7930db763d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:06:43.771627 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 4/7] 10.222.0.18 () {58 vars in 1024 bytes} [Tue Sep 13 03:06:43 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 384 bytes (2 switches on core 0) Sep 13 03:06:44.044904 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-606faa1e-da34-4468-b4a7-76e76ad6adc9 None None] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:06:44.057626 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-606faa1e-da34-4468-b4a7-76e76ad6adc9 None None] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:06:44.061810 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dd54a65-b9dd-4679-973f-5a121010c35e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76c5181fc7be45279ac00017bb4640b3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:06:44.062406 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dd54a65-b9dd-4679-973f-5a121010c35e admin admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:06:44.062716 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dd54a65-b9dd-4679-973f-5a121010c35e admin admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:06:44.062992 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dd54a65-b9dd-4679-973f-5a121010c35e admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:06:44.071448 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dd54a65-b9dd-4679-973f-5a121010c35e admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:06:44.073249 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dd54a65-b9dd-4679-973f-5a121010c35e admin admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:06:44.082050 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 4/8] 10.222.0.18 () {58 vars in 1324 bytes} [Tue Sep 13 03:06:44 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Sep 13 03:06:44.087587 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74bf636f-3db3-46db-b982-e7930db763d4 None None] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:06:44.096584 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74bf636f-3db3-46db-b982-e7930db763d4 None None] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:06:44.100948 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29d26bb5-8a6f-4e4b-b59d-d43d99aae930 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76c5181fc7be45279ac00017bb4640b3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:06:44.101558 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d26bb5-8a6f-4e4b-b59d-d43d99aae930 admin admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:06:44.101834 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d26bb5-8a6f-4e4b-b59d-d43d99aae930 admin admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:06:44.102105 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d26bb5-8a6f-4e4b-b59d-d43d99aae930 admin admin] PATH_INFO: `/v3/users/49f280ab086b4610855c4ffa378517d9` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:06:44.103432 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29d26bb5-8a6f-4e4b-b59d-d43d99aae930 admin admin] RBAC: Authorizing `identity:get_user(user_id=49f280ab086b4610855c4ffa378517d9)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:06:44.105255 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29d26bb5-8a6f-4e4b-b59d-d43d99aae930 admin admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:06:44.106803 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 5/9] 10.222.0.18 () {56 vars in 1198 bytes} [Tue Sep 13 03:06:44 2022] GET /identity/v3/users/49f280ab086b4610855c4ffa378517d9 => generated 247 bytes in 20 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 03:06:46.297594 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c312f606-6752-4cb1-b5cd-2434ce215c5d None None] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:06:46.298121 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c312f606-6752-4cb1-b5cd-2434ce215c5d None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:06:46.298121 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c312f606-6752-4cb1-b5cd-2434ce215c5d None None] PATH_INFO: `/` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:06:46.299540 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 5/10] 10.222.0.18 () {54 vars in 927 bytes} [Tue Sep 13 03:06:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 03:06:46.304748 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88cca5cd-767b-4e85-8910-a19199be6a05 None None] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:06:46.305135 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88cca5cd-767b-4e85-8910-a19199be6a05 None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:06:46.305289 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88cca5cd-767b-4e85-8910-a19199be6a05 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:06:46.323280 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.auth.core [None req-88cca5cd-767b-4e85-8910-a19199be6a05 None None] MFA Rules not processed for user `49f280ab086b4610855c4ffa378517d9`. Rule list: `[]` (Enabled: `True`). {{(pid=81400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 03:06:46.324375 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-88cca5cd-767b-4e85-8910-a19199be6a05 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:06:46.338675 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09947a7c-495b-4e6d-9478-6d1e75276c61 None None] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:06:46.338999 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09947a7c-495b-4e6d-9478-6d1e75276c61 None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:06:46.339280 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09947a7c-495b-4e6d-9478-6d1e75276c61 None None] PATH_INFO: `/` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:06:46.340612 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 6/11] 10.222.0.18 () {54 vars in 927 bytes} [Tue Sep 13 03:06:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 13 03:06:46.345572 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74598fe0-bb64-4a5d-8ac0-8cb2db525ed4 None None] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:06:46.345848 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74598fe0-bb64-4a5d-8ac0-8cb2db525ed4 None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:06:46.346759 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74598fe0-bb64-4a5d-8ac0-8cb2db525ed4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:06:46.387728 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG passlib.handlers.bcrypt [None req-74598fe0-bb64-4a5d-8ac0-8cb2db525ed4 None None] detected 'bcrypt' backend, version '4.0.0' {{(pid=81399) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Sep 13 03:06:46.388115 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG passlib.handlers.bcrypt [None req-74598fe0-bb64-4a5d-8ac0-8cb2db525ed4 None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=81399) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Sep 13 03:06:46.404908 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 6/12] 10.222.0.18 () {58 vars in 1024 bytes} [Tue Sep 13 03:06:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 13 03:06:46.437663 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.auth.core [None req-74598fe0-bb64-4a5d-8ac0-8cb2db525ed4 None None] MFA Rules not processed for user `49f280ab086b4610855c4ffa378517d9`. Rule list: `[]` (Enabled: `True`). {{(pid=81399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 03:06:46.439279 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-74598fe0-bb64-4a5d-8ac0-8cb2db525ed4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:06:46.537753 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 7/13] 10.222.0.18 () {58 vars in 1024 bytes} [Tue Sep 13 03:06:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 13 03:06:46.672334 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88cca5cd-767b-4e85-8910-a19199be6a05 None None] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:06:46.696524 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:06:46.780053 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88cca5cd-767b-4e85-8910-a19199be6a05 None None] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:06:46.800590 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74598fe0-bb64-4a5d-8ac0-8cb2db525ed4 None None] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:06:46.812701 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0f7cde4-1f51-4db0-9940-3120c127cf70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:06:46.813328 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0f7cde4-1f51-4db0-9940-3120c127cf70 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:06:46.813604 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0f7cde4-1f51-4db0-9940-3120c127cf70 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:06:46.813874 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0f7cde4-1f51-4db0-9940-3120c127cf70 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:06:46.825642 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0f7cde4-1f51-4db0-9940-3120c127cf70 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 03:06:46.835205 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application [None req-b0f7cde4-1f51-4db0-9940-3120c127cf70 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 13 03:06:46.835205 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 03:06:46.835205 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 03:06:46.835205 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 03:06:46.835205 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 03:06:46.835205 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 03:06:46.835205 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 03:06:46.835205 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 03:06:46.835205 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 03:06:46.835205 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 03:06:46.835205 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 03:06:46.835205 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 03:06:46.835205 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 03:06:46.835205 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 03:06:46.835205 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 03:06:46.835205 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 03:06:46.836147 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 03:06:46.836147 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 03:06:46.836147 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 03:06:46.836147 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 03:06:46.836147 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 03:06:46.836147 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 03:06:46.836147 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 03:06:46.836147 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 03:06:46.836147 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 03:06:46.836147 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 03:06:46.836147 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 03:06:46.836147 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application with Lock( Sep 13 03:06:46.836147 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 03:06:46.836147 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._enter() Sep 13 03:06:46.836147 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 03:06:46.836147 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 03:06:46.836982 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 03:06:46.836982 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.creator() Sep 13 03:06:46.836982 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 03:06:46.836982 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application created_value = creator( Sep 13 03:06:46.836982 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 03:06:46.836982 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 03:06:46.836982 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 03:06:46.836982 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 03:06:46.836982 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 03:06:46.836982 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 03:06:46.836982 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 13 03:06:46.836982 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Sep 13 03:06:46.836982 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 7/14] 10.222.0.18 () {56 vars in 1096 bytes} [Tue Sep 13 03:06:46 2022] GET /identity/v3/roles/admin => generated 83 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 03:06:46.842063 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0f7cde4-1f51-4db0-9940-3120c127cf70 None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:06:46.899223 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74598fe0-bb64-4a5d-8ac0-8cb2db525ed4 None None] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:06:46.901448 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:06:46.922655 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:06:46.935846 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2c97d61-ea73-4176-a288-349d70f64544 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:06:46.936713 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2c97d61-ea73-4176-a288-349d70f64544 None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:06:46.936994 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2c97d61-ea73-4176-a288-349d70f64544 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:06:46.937266 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2c97d61-ea73-4176-a288-349d70f64544 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:06:46.941598 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0f7cde4-1f51-4db0-9940-3120c127cf70 None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:06:46.974853 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-940fb32a-b38a-4ee7-a81e-3c762f02d329 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:06:46.975445 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-940fb32a-b38a-4ee7-a81e-3c762f02d329 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:06:46.976031 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-940fb32a-b38a-4ee7-a81e-3c762f02d329 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:06:46.976031 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-940fb32a-b38a-4ee7-a81e-3c762f02d329 None admin] PATH_INFO: `/v3/roles` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:06:46.976625 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-940fb32a-b38a-4ee7-a81e-3c762f02d329 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 03:06:46.976925 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-940fb32a-b38a-4ee7-a81e-3c762f02d329 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:06:46.978311 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-940fb32a-b38a-4ee7-a81e-3c762f02d329 None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:06:46.990062 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 8/15] 10.222.0.18 () {56 vars in 1099 bytes} [Tue Sep 13 03:06:46 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 03:06:46.996274 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-940fb32a-b38a-4ee7-a81e-3c762f02d329 None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:06:47.006373 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2c97d61-ea73-4176-a288-349d70f64544 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:06:47.007955 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2c97d61-ea73-4176-a288-349d70f64544 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:06:47.095687 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-940fb32a-b38a-4ee7-a81e-3c762f02d329 None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:06:47.104160 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 8/16] 10.222.0.18 () {58 vars in 1282 bytes} [Tue Sep 13 03:06:46 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 304 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Sep 13 03:06:47.109568 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2c97d61-ea73-4176-a288-349d70f64544 None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:06:47.113039 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:06:47.129513 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c32e1ede-9809-4790-b8b2-8f5d04961240 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:06:47.130143 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c32e1ede-9809-4790-b8b2-8f5d04961240 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:06:47.130421 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c32e1ede-9809-4790-b8b2-8f5d04961240 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:06:47.130726 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c32e1ede-9809-4790-b8b2-8f5d04961240 None admin] PATH_INFO: `/v3/users/49f280ab086b4610855c4ffa378517d9` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:06:47.131844 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c32e1ede-9809-4790-b8b2-8f5d04961240 None admin] RBAC: Authorizing `identity:get_user(user_id=49f280ab086b4610855c4ffa378517d9)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:06:47.133299 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c32e1ede-9809-4790-b8b2-8f5d04961240 None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:06:47.134908 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 9/17] 10.222.0.18 () {56 vars in 1177 bytes} [Tue Sep 13 03:06:46 2022] GET /identity/v3/users/49f280ab086b4610855c4ffa378517d9 => generated 247 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 03:06:47.140164 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c32e1ede-9809-4790-b8b2-8f5d04961240 None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:06:47.154440 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:06:47.206794 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2c97d61-ea73-4176-a288-349d70f64544 None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:06:47.238210 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c32e1ede-9809-4790-b8b2-8f5d04961240 None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:06:47.239589 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0379bd85-f56f-43f9-ac6c-7d6732625e64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:06:47.240439 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0379bd85-f56f-43f9-ac6c-7d6732625e64 None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:06:47.240719 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0379bd85-f56f-43f9-ac6c-7d6732625e64 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:06:47.240993 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0379bd85-f56f-43f9-ac6c-7d6732625e64 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:06:47.259931 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0379bd85-f56f-43f9-ac6c-7d6732625e64 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:06:47.261407 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0379bd85-f56f-43f9-ac6c-7d6732625e64 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:06:47.272719 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e9b8a69-83f0-405e-8df9-0f3ac311b189 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:06:47.273329 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e9b8a69-83f0-405e-8df9-0f3ac311b189 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:06:47.273603 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e9b8a69-83f0-405e-8df9-0f3ac311b189 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:06:47.273871 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e9b8a69-83f0-405e-8df9-0f3ac311b189 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:06:47.275262 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e9b8a69-83f0-405e-8df9-0f3ac311b189 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:06:47.276690 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e9b8a69-83f0-405e-8df9-0f3ac311b189 None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:06:47.278211 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 10/18] 10.222.0.18 () {56 vars in 1108 bytes} [Tue Sep 13 03:06:47 2022] GET /identity/v3/domains/default => generated 199 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 03:06:47.279332 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application [None req-0379bd85-f56f-43f9-ac6c-7d6732625e64 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 03:06:47.279332 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 03:06:47.279332 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 03:06:47.279332 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 03:06:47.279332 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 03:06:47.279332 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 03:06:47.279332 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 03:06:47.279332 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 03:06:47.279332 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 03:06:47.279332 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 03:06:47.279332 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 03:06:47.279332 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 03:06:47.279332 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 13 03:06:47.279332 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 13 03:06:47.279332 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 13 03:06:47.279332 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 13 03:06:47.280313 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 03:06:47.280313 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 03:06:47.280313 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 03:06:47.280313 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 03:06:47.280313 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 03:06:47.280313 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 03:06:47.280313 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 03:06:47.280313 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application with Lock( Sep 13 03:06:47.280313 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 03:06:47.280313 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._enter() Sep 13 03:06:47.280313 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 03:06:47.280313 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 03:06:47.280313 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 03:06:47.280313 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.creator() Sep 13 03:06:47.280313 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 03:06:47.280313 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application created_value = creator( Sep 13 03:06:47.280313 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 13 03:06:47.281133 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 13 03:06:47.281133 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 03:06:47.281133 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Sep 13 03:06:47.281133 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 9/19] 10.222.0.18 () {56 vars in 1108 bytes} [Tue Sep 13 03:06:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 13 03:06:47.284488 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e9b8a69-83f0-405e-8df9-0f3ac311b189 None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:06:47.285916 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0379bd85-f56f-43f9-ac6c-7d6732625e64 None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:06:47.340787 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:06:47.363472 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:06:47.384247 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e9b8a69-83f0-405e-8df9-0f3ac311b189 None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:06:47.387166 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0379bd85-f56f-43f9-ac6c-7d6732625e64 None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:06:47.417325 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00275d82-6326-40e8-abd3-339d94e13b81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:06:47.417922 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00275d82-6326-40e8-abd3-339d94e13b81 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:06:47.418198 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00275d82-6326-40e8-abd3-339d94e13b81 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:06:47.418468 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00275d82-6326-40e8-abd3-339d94e13b81 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:06:47.419226 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00275d82-6326-40e8-abd3-339d94e13b81 None admin] RBAC: Adding query filter params (role.id=425d4b481c7846e0bc07e7e805d5b15f, scope.domain.id=default, user.id=49f280ab086b4610855c4ffa378517d9) {{(pid=81400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 03:06:47.419526 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00275d82-6326-40e8-abd3-339d94e13b81 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:06:47.420279 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5066eec7-8312-4a28-bf1a-d2ab4ea19cb3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:06:47.420880 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5066eec7-8312-4a28-bf1a-d2ab4ea19cb3 None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:06:47.420977 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00275d82-6326-40e8-abd3-339d94e13b81 None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:06:47.421159 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5066eec7-8312-4a28-bf1a-d2ab4ea19cb3 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:06:47.421429 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5066eec7-8312-4a28-bf1a-d2ab4ea19cb3 None admin] PATH_INFO: `/v3/domains` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:06:47.422060 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5066eec7-8312-4a28-bf1a-d2ab4ea19cb3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 03:06:47.422361 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5066eec7-8312-4a28-bf1a-d2ab4ea19cb3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:06:47.423764 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5066eec7-8312-4a28-bf1a-d2ab4ea19cb3 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:06:47.428015 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 11/20] 10.222.0.18 () {56 vars in 1322 bytes} [Tue Sep 13 03:06:47 2022] GET /identity/v3/role_assignments?user.id=49f280ab086b4610855c4ffa378517d9&scope.domain.id=default&role.id=425d4b481c7846e0bc07e7e805d5b15f => generated 233 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 03:06:47.443104 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 10/21] 10.222.0.18 () {56 vars in 1109 bytes} [Tue Sep 13 03:06:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 03:06:49.992630 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42c149bb-f405-4cef-a7db-ea33dd887e4a None None] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:06:49.993654 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42c149bb-f405-4cef-a7db-ea33dd887e4a None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:06:49.994154 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42c149bb-f405-4cef-a7db-ea33dd887e4a None None] PATH_INFO: `/` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:06:49.995726 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 12/22] 10.222.0.18 () {54 vars in 927 bytes} [Tue Sep 13 03:06:49 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 03:06:50.001158 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2aeef65-c24a-403d-88eb-98f20cfb52ae None None] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:06:50.001993 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2aeef65-c24a-403d-88eb-98f20cfb52ae None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:06:50.002516 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2aeef65-c24a-403d-88eb-98f20cfb52ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:06:50.031595 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.auth.core [None req-e2aeef65-c24a-403d-88eb-98f20cfb52ae None None] MFA Rules not processed for user `49f280ab086b4610855c4ffa378517d9`. Rule list: `[]` (Enabled: `True`). {{(pid=81399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 03:06:50.033184 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-e2aeef65-c24a-403d-88eb-98f20cfb52ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:06:50.131556 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 11/23] 10.222.0.18 () {58 vars in 1024 bytes} [Tue Sep 13 03:06:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 13 03:06:50.296376 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e774a0d-d75f-42b6-a9b4-b6a778fe42b7 None None] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:06:50.297004 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e774a0d-d75f-42b6-a9b4-b6a778fe42b7 None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:06:50.297500 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e774a0d-d75f-42b6-a9b4-b6a778fe42b7 None None] PATH_INFO: `/` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:06:50.299109 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 13/24] 10.222.0.18 () {54 vars in 927 bytes} [Tue Sep 13 03:06:50 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 03:06:50.304668 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f943314f-82b4-4ee0-88f2-fc393d44243d None None] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:06:50.305247 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f943314f-82b4-4ee0-88f2-fc393d44243d None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:06:50.305714 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f943314f-82b4-4ee0-88f2-fc393d44243d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:06:50.328815 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.auth.core [None req-f943314f-82b4-4ee0-88f2-fc393d44243d None None] MFA Rules not processed for user `49f280ab086b4610855c4ffa378517d9`. Rule list: `[]` (Enabled: `True`). {{(pid=81399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 03:06:50.330877 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-f943314f-82b4-4ee0-88f2-fc393d44243d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:06:50.395266 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:06:50.427136 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 12/25] 10.222.0.18 () {58 vars in 1024 bytes} [Tue Sep 13 03:06:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 13 03:06:50.511407 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e52615-1b6e-4731-8fd1-559a92242270 None None] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:06:50.511827 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e52615-1b6e-4731-8fd1-559a92242270 None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:06:50.511930 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e52615-1b6e-4731-8fd1-559a92242270 None None] PATH_INFO: `/` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:06:50.513295 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 14/26] 10.222.0.18 () {54 vars in 927 bytes} [Tue Sep 13 03:06:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 03:06:50.519320 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d42ec9f-068d-47aa-ade6-3a5035f13aa6 None None] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:06:50.519610 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d42ec9f-068d-47aa-ade6-3a5035f13aa6 None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:06:50.519886 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d42ec9f-068d-47aa-ade6-3a5035f13aa6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:06:50.547715 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.auth.core [None req-4d42ec9f-068d-47aa-ade6-3a5035f13aa6 None None] MFA Rules not processed for user `49f280ab086b4610855c4ffa378517d9`. Rule list: `[]` (Enabled: `True`). {{(pid=81399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 03:06:50.548850 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-4d42ec9f-068d-47aa-ade6-3a5035f13aa6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:06:50.572681 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30e52615-1b6e-4731-8fd1-559a92242270 None None] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:06:50.630030 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 13/27] 10.222.0.18 () {58 vars in 1024 bytes} [Tue Sep 13 03:06:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 13 03:06:50.631737 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc10899-2539-4141-828b-63cf814ea51e None None] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:06:50.632160 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc10899-2539-4141-828b-63cf814ea51e None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:06:50.632666 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc10899-2539-4141-828b-63cf814ea51e None None] PATH_INFO: `/` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:06:50.634146 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 14/28] 10.222.0.18 () {54 vars in 927 bytes} [Tue Sep 13 03:06:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 03:06:50.640039 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d0fe773-f757-4625-8717-7d9b09a7fdb6 None None] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:06:50.640488 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d0fe773-f757-4625-8717-7d9b09a7fdb6 None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:06:50.640897 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d0fe773-f757-4625-8717-7d9b09a7fdb6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:06:50.660458 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.auth.core [None req-1d0fe773-f757-4625-8717-7d9b09a7fdb6 None None] MFA Rules not processed for user `49f280ab086b4610855c4ffa378517d9`. Rule list: `[]` (Enabled: `True`). {{(pid=81399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 03:06:50.661950 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-1d0fe773-f757-4625-8717-7d9b09a7fdb6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:06:50.747369 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 15/29] 10.222.0.18 () {58 vars in 1024 bytes} [Tue Sep 13 03:06:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 13 03:06:50.799443 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30e52615-1b6e-4731-8fd1-559a92242270 None None] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:06:50.849663 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8266b6ae-7593-4d25-8f71-b41c874e07f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:06:50.850662 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8266b6ae-7593-4d25-8f71-b41c874e07f5 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:06:50.851134 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8266b6ae-7593-4d25-8f71-b41c874e07f5 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:06:50.851675 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8266b6ae-7593-4d25-8f71-b41c874e07f5 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:06:50.864183 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8266b6ae-7593-4d25-8f71-b41c874e07f5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 03:06:50.869707 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application [None req-8266b6ae-7593-4d25-8f71-b41c874e07f5 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 13 03:06:50.869707 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 03:06:50.869707 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 03:06:50.869707 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 03:06:50.869707 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 03:06:50.869707 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 03:06:50.869707 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 03:06:50.869707 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 03:06:50.869707 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 03:06:50.869707 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 03:06:50.869707 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 03:06:50.869707 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 03:06:50.869707 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 03:06:50.869707 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 03:06:50.869707 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 03:06:50.869707 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 03:06:50.870546 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 03:06:50.870546 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 03:06:50.870546 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 03:06:50.870546 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 03:06:50.870546 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 03:06:50.870546 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 03:06:50.870546 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 03:06:50.870546 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 03:06:50.870546 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 03:06:50.870546 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 03:06:50.870546 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 03:06:50.870546 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application with Lock( Sep 13 03:06:50.870546 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 03:06:50.870546 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._enter() Sep 13 03:06:50.870546 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 03:06:50.870546 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 03:06:50.871449 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 03:06:50.871449 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.creator() Sep 13 03:06:50.871449 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 03:06:50.871449 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application created_value = creator( Sep 13 03:06:50.871449 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 03:06:50.871449 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 03:06:50.871449 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 03:06:50.871449 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 03:06:50.871449 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 03:06:50.871449 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 03:06:50.871449 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 13 03:06:50.871449 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Sep 13 03:06:50.873676 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 15/30] 10.222.0.18 () {56 vars in 1096 bytes} [Tue Sep 13 03:06:50 2022] GET /identity/v3/roles/admin => generated 83 bytes in 301 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 13 03:06:50.879221 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d0fe773-f757-4625-8717-7d9b09a7fdb6 None None] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:06:50.935399 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:06:50.974147 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8266b6ae-7593-4d25-8f71-b41c874e07f5 None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:06:50.978057 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:06:51.042180 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d0fe773-f757-4625-8717-7d9b09a7fdb6 None None] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:06:51.109860 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58142a93-51cc-43ab-b138-418f0a578f77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:06:51.110906 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58142a93-51cc-43ab-b138-418f0a578f77 None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:06:51.111488 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58142a93-51cc-43ab-b138-418f0a578f77 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:06:51.111961 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58142a93-51cc-43ab-b138-418f0a578f77 None admin] PATH_INFO: `/v3/roles` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:06:51.112845 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58142a93-51cc-43ab-b138-418f0a578f77 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 03:06:51.113418 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58142a93-51cc-43ab-b138-418f0a578f77 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:06:51.115179 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58142a93-51cc-43ab-b138-418f0a578f77 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:06:51.129972 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 16/31] 10.222.0.18 () {56 vars in 1099 bytes} [Tue Sep 13 03:06:50 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 251 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 03:06:51.135794 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58142a93-51cc-43ab-b138-418f0a578f77 None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:06:51.141398 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8266b6ae-7593-4d25-8f71-b41c874e07f5 None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:06:51.183588 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5540623e-f404-440e-8fa6-c2c7d140810c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:06:51.184742 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5540623e-f404-440e-8fa6-c2c7d140810c None admin] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:06:51.185190 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5540623e-f404-440e-8fa6-c2c7d140810c None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:06:51.185632 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5540623e-f404-440e-8fa6-c2c7d140810c None admin] PATH_INFO: `/v3/roles` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:06:51.186700 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5540623e-f404-440e-8fa6-c2c7d140810c None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:06:51.188938 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5540623e-f404-440e-8fa6-c2c7d140810c None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:06:51.252925 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:06:51.257583 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58142a93-51cc-43ab-b138-418f0a578f77 None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:06:51.295084 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b34f0f32-0f8d-44a7-90ce-25a135e86324 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:06:51.296685 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b34f0f32-0f8d-44a7-90ce-25a135e86324 None admin] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:06:51.297194 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b34f0f32-0f8d-44a7-90ce-25a135e86324 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:06:51.297722 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b34f0f32-0f8d-44a7-90ce-25a135e86324 None admin] PATH_INFO: `/v3/roles` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:06:51.298668 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b34f0f32-0f8d-44a7-90ce-25a135e86324 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:06:51.300370 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b34f0f32-0f8d-44a7-90ce-25a135e86324 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:06:51.335131 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 16/32] 10.222.0.18 () {60 vars in 1131 bytes} [Tue Sep 13 03:06:50 2022] POST /identity/v3/roles => generated 219 bytes in 361 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 13 03:06:51.336236 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5540623e-f404-440e-8fa6-c2c7d140810c None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:06:51.415409 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:06:51.449146 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 17/33] 10.222.0.18 () {60 vars in 1131 bytes} [Tue Sep 13 03:06:51 2022] POST /identity/v3/roles => generated 225 bytes in 319 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 03:06:51.450702 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b34f0f32-0f8d-44a7-90ce-25a135e86324 None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:06:51.481621 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5540623e-f404-440e-8fa6-c2c7d140810c None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:06:51.538806 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f397aad9-6b18-46a7-8b79-a54ab11e361e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:06:51.539508 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f397aad9-6b18-46a7-8b79-a54ab11e361e None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:06:51.539794 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f397aad9-6b18-46a7-8b79-a54ab11e361e None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:06:51.540076 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f397aad9-6b18-46a7-8b79-a54ab11e361e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:06:51.555168 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f397aad9-6b18-46a7-8b79-a54ab11e361e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:06:51.557023 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f397aad9-6b18-46a7-8b79-a54ab11e361e None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:06:51.579841 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application [None req-f397aad9-6b18-46a7-8b79-a54ab11e361e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 03:06:51.579841 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 03:06:51.579841 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 03:06:51.579841 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 03:06:51.579841 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 03:06:51.579841 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 03:06:51.579841 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 03:06:51.579841 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 03:06:51.579841 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 03:06:51.579841 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 03:06:51.579841 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 03:06:51.579841 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 03:06:51.579841 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 13 03:06:51.579841 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 13 03:06:51.579841 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 13 03:06:51.579841 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 13 03:06:51.580838 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 03:06:51.580838 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 03:06:51.580838 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 03:06:51.580838 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 03:06:51.580838 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 03:06:51.580838 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 03:06:51.580838 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 03:06:51.580838 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application with Lock( Sep 13 03:06:51.580838 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 03:06:51.580838 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._enter() Sep 13 03:06:51.580838 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 03:06:51.580838 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 03:06:51.580838 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 03:06:51.580838 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.creator() Sep 13 03:06:51.580838 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 03:06:51.580838 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application created_value = creator( Sep 13 03:06:51.580838 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 13 03:06:51.582097 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 13 03:06:51.582097 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 03:06:51.582097 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Sep 13 03:06:51.582097 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 17/34] 10.222.0.18 () {56 vars in 1108 bytes} [Tue Sep 13 03:06:51 2022] GET /identity/v3/domains/Default => generated 87 bytes in 246 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 03:06:51.583515 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d827e875-109a-4ce9-9806-5a7ec721e235 None None] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:06:51.583515 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d827e875-109a-4ce9-9806-5a7ec721e235 None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:06:51.583515 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d827e875-109a-4ce9-9806-5a7ec721e235 None None] PATH_INFO: `/` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:06:51.584855 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 18/35] 10.222.0.18 () {54 vars in 927 bytes} [Tue Sep 13 03:06:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 03:06:51.585919 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b34f0f32-0f8d-44a7-90ce-25a135e86324 None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:06:51.590162 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d827e875-109a-4ce9-9806-5a7ec721e235 None None] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:06:51.632730 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f55441d7-4e41-49ff-903d-3a511ae3aaed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:06:51.633401 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f55441d7-4e41-49ff-903d-3a511ae3aaed None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:06:51.633680 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f55441d7-4e41-49ff-903d-3a511ae3aaed None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:06:51.634828 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f55441d7-4e41-49ff-903d-3a511ae3aaed None admin] PATH_INFO: `/v3/users/49f280ab086b4610855c4ffa378517d9` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:06:51.641130 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f55441d7-4e41-49ff-903d-3a511ae3aaed None admin] RBAC: Authorizing `identity:get_user(user_id=49f280ab086b4610855c4ffa378517d9)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:06:51.643349 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f55441d7-4e41-49ff-903d-3a511ae3aaed None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:06:51.645757 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 18/36] 10.222.0.18 () {56 vars in 1177 bytes} [Tue Sep 13 03:06:51 2022] GET /identity/v3/users/49f280ab086b4610855c4ffa378517d9 => generated 247 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 03:06:51.647048 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ffcae8-0607-4b06-9e95-72a19cc12a4d None None] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:06:51.647324 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ffcae8-0607-4b06-9e95-72a19cc12a4d None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:06:51.647590 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ffcae8-0607-4b06-9e95-72a19cc12a4d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:06:51.669141 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.auth.core [None req-78ffcae8-0607-4b06-9e95-72a19cc12a4d None None] MFA Rules not processed for user `49f280ab086b4610855c4ffa378517d9`. Rule list: `[]` (Enabled: `True`). {{(pid=81399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 03:06:51.670727 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-78ffcae8-0607-4b06-9e95-72a19cc12a4d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:06:51.697149 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:06:51.747016 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d827e875-109a-4ce9-9806-5a7ec721e235 None None] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:06:51.760756 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 19/37] 10.222.0.18 () {58 vars in 1024 bytes} [Tue Sep 13 03:06:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 13 03:06:51.761629 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78ffcae8-0607-4b06-9e95-72a19cc12a4d None None] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:06:51.773552 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:06:51.800339 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af45b6b6-dadd-44de-bb54-900e193747a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:06:51.801277 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af45b6b6-dadd-44de-bb54-900e193747a7 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:06:51.801567 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af45b6b6-dadd-44de-bb54-900e193747a7 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:06:51.801844 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af45b6b6-dadd-44de-bb54-900e193747a7 None admin] PATH_INFO: `/v3/domains` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:06:51.803121 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af45b6b6-dadd-44de-bb54-900e193747a7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 03:06:51.803246 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af45b6b6-dadd-44de-bb54-900e193747a7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:06:51.805345 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af45b6b6-dadd-44de-bb54-900e193747a7 None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:06:51.829040 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 19/38] 10.222.0.18 () {56 vars in 1109 bytes} [Tue Sep 13 03:06:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 239 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 03:06:51.835868 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af45b6b6-dadd-44de-bb54-900e193747a7 None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:06:51.893271 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78ffcae8-0607-4b06-9e95-72a19cc12a4d None None] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:06:51.936734 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d0a87c9-696a-48d5-ac21-f9ce798c6133 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:06:51.937645 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d0a87c9-696a-48d5-ac21-f9ce798c6133 None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:06:51.938123 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d0a87c9-696a-48d5-ac21-f9ce798c6133 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:06:51.938561 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d0a87c9-696a-48d5-ac21-f9ce798c6133 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:06:51.939891 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d0a87c9-696a-48d5-ac21-f9ce798c6133 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:06:51.941513 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d0a87c9-696a-48d5-ac21-f9ce798c6133 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:06:51.947997 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 20/39] 10.222.0.18 () {56 vars in 1108 bytes} [Tue Sep 13 03:06:51 2022] GET /identity/v3/domains/default => generated 199 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 03:06:51.949594 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d0a87c9-696a-48d5-ac21-f9ce798c6133 None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:06:51.976038 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af45b6b6-dadd-44de-bb54-900e193747a7 None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:06:51.978898 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:06:52.014027 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a8dd516-d8f1-40d9-b1e2-6404c7dd5156 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:06:52.014596 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a8dd516-d8f1-40d9-b1e2-6404c7dd5156 None admin] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:06:52.015114 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a8dd516-d8f1-40d9-b1e2-6404c7dd5156 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:06:52.015279 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a8dd516-d8f1-40d9-b1e2-6404c7dd5156 None admin] PATH_INFO: `/v3/projects` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:06:52.015920 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a8dd516-d8f1-40d9-b1e2-6404c7dd5156 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:06:52.017365 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a8dd516-d8f1-40d9-b1e2-6404c7dd5156 None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:06:52.048450 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 20/40] 10.222.0.18 () {60 vars in 1141 bytes} [Tue Sep 13 03:06:51 2022] POST /identity/v3/projects => generated 301 bytes in 213 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 13 03:06:52.054844 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a56330be-d934-42b6-9435-fba5b1eb59f9 None None] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:06:52.054844 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a56330be-d934-42b6-9435-fba5b1eb59f9 None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:06:52.054844 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a56330be-d934-42b6-9435-fba5b1eb59f9 None None] PATH_INFO: `/` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:06:52.054844 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 21/41] 10.222.0.18 () {54 vars in 927 bytes} [Tue Sep 13 03:06:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 03:06:52.058147 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-044ec83b-3593-4a67-8535-06cd093b321f None None] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:06:52.058432 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-044ec83b-3593-4a67-8535-06cd093b321f None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:06:52.058740 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-044ec83b-3593-4a67-8535-06cd093b321f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:06:52.072740 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d0a87c9-696a-48d5-ac21-f9ce798c6133 None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:06:52.079335 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.auth.core [None req-044ec83b-3593-4a67-8535-06cd093b321f None None] MFA Rules not processed for user `49f280ab086b4610855c4ffa378517d9`. Rule list: `[]` (Enabled: `True`). {{(pid=81400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 03:06:52.081288 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-044ec83b-3593-4a67-8535-06cd093b321f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:06:52.112064 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdcb45ef-355a-4666-b04e-b9ce545759e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:06:52.112773 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdcb45ef-355a-4666-b04e-b9ce545759e6 None admin] REQUEST_METHOD: `PUT` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:06:52.113051 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdcb45ef-355a-4666-b04e-b9ce545759e6 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:06:52.113330 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdcb45ef-355a-4666-b04e-b9ce545759e6 None admin] PATH_INFO: `/v3/domains/default/users/49f280ab086b4610855c4ffa378517d9/roles/425d4b481c7846e0bc07e7e805d5b15f` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:06:52.113421 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:06:52.115512 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdcb45ef-355a-4666-b04e-b9ce545759e6 None admin] RBAC: Authorizing `identity:create_grant(domain_id=default, user_id=49f280ab086b4610855c4ffa378517d9, role_id=425d4b481c7846e0bc07e7e805d5b15f)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:06:52.117013 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdcb45ef-355a-4666-b04e-b9ce545759e6 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:06:52.138219 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 21/42] 10.222.0.18 () {58 vars in 1361 bytes} [Tue Sep 13 03:06:51 2022] PUT /identity/v3/domains/default/users/49f280ab086b4610855c4ffa378517d9/roles/425d4b481c7846e0bc07e7e805d5b15f => generated 0 bytes in 189 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 03:06:52.139883 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdcb45ef-355a-4666-b04e-b9ce545759e6 None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:06:52.171458 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 22/43] 10.222.0.18 () {58 vars in 1024 bytes} [Tue Sep 13 03:06:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 13 03:06:52.252989 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:06:52.272479 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdcb45ef-355a-4666-b04e-b9ce545759e6 None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:06:52.312635 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10e56e53-0961-4129-92e4-6a15b2de53bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:06:52.313585 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10e56e53-0961-4129-92e4-6a15b2de53bb None admin] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:06:52.314036 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10e56e53-0961-4129-92e4-6a15b2de53bb None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:06:52.314637 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10e56e53-0961-4129-92e4-6a15b2de53bb None admin] PATH_INFO: `/v3/roles` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:06:52.315505 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10e56e53-0961-4129-92e4-6a15b2de53bb None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:06:52.317160 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10e56e53-0961-4129-92e4-6a15b2de53bb None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:06:52.334403 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 22/44] 10.222.0.18 () {60 vars in 1131 bytes} [Tue Sep 13 03:06:52 2022] POST /identity/v3/roles => generated 223 bytes in 195 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 03:06:52.447885 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-044ec83b-3593-4a67-8535-06cd093b321f None None] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:06:52.554905 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-044ec83b-3593-4a67-8535-06cd093b321f None None] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:06:52.591307 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6925050c-9739-41c1-866e-1a955d13e25d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:06:52.591931 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6925050c-9739-41c1-866e-1a955d13e25d None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:06:52.592206 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6925050c-9739-41c1-866e-1a955d13e25d None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:06:52.592475 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6925050c-9739-41c1-866e-1a955d13e25d None admin] PATH_INFO: `/v3/domains/default` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:06:52.593559 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6925050c-9739-41c1-866e-1a955d13e25d None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:06:52.595023 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6925050c-9739-41c1-866e-1a955d13e25d None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:06:52.597127 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 23/45] 10.222.0.18 () {56 vars in 1108 bytes} [Tue Sep 13 03:06:52 2022] GET /identity/v3/domains/default => generated 199 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 03:06:52.602793 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10e56e53-0961-4129-92e4-6a15b2de53bb None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:06:52.699952 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10e56e53-0961-4129-92e4-6a15b2de53bb None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:06:52.732754 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35679d09-6820-47eb-b352-58b34d459a0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:06:52.733373 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35679d09-6820-47eb-b352-58b34d459a0c None admin] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:06:52.733655 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35679d09-6820-47eb-b352-58b34d459a0c None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:06:52.733928 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35679d09-6820-47eb-b352-58b34d459a0c None admin] PATH_INFO: `/v3/projects` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:06:52.734572 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35679d09-6820-47eb-b352-58b34d459a0c None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:06:52.736052 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35679d09-6820-47eb-b352-58b34d459a0c None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:06:52.753161 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:06:52.766059 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 23/46] 10.222.0.18 () {60 vars in 1141 bytes} [Tue Sep 13 03:06:52 2022] POST /identity/v3/projects => generated 312 bytes in 164 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 03:06:54.336432 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b4d2eb2-9cb3-4bf1-b34a-df03f37c0adb None None] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:06:54.336936 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b4d2eb2-9cb3-4bf1-b34a-df03f37c0adb None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:06:54.336936 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b4d2eb2-9cb3-4bf1-b34a-df03f37c0adb None None] PATH_INFO: `/` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:06:54.338234 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 24/47] 10.222.0.18 () {54 vars in 927 bytes} [Tue Sep 13 03:06:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 03:06:54.343412 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d6c5c20-56da-4ed0-a0a8-620500b0356a None None] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:06:54.343770 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d6c5c20-56da-4ed0-a0a8-620500b0356a None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:06:54.343968 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d6c5c20-56da-4ed0-a0a8-620500b0356a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:06:54.362451 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.auth.core [None req-7d6c5c20-56da-4ed0-a0a8-620500b0356a None None] MFA Rules not processed for user `49f280ab086b4610855c4ffa378517d9`. Rule list: `[]` (Enabled: `True`). {{(pid=81399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 03:06:54.364231 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-7d6c5c20-56da-4ed0-a0a8-620500b0356a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:06:54.428364 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 24/48] 10.222.0.18 () {58 vars in 1024 bytes} [Tue Sep 13 03:06:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 13 03:06:54.696433 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b4d2eb2-9cb3-4bf1-b34a-df03f37c0adb None None] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:06:54.724409 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:06:54.905380 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b4d2eb2-9cb3-4bf1-b34a-df03f37c0adb None None] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:06:54.915581 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-811bb5e4-ec9a-4ded-8725-880f8aa862cc None None] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:06:54.915953 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-811bb5e4-ec9a-4ded-8725-880f8aa862cc None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:06:54.916298 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-811bb5e4-ec9a-4ded-8725-880f8aa862cc None None] PATH_INFO: `/` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:06:54.917786 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 25/49] 10.222.0.18 () {54 vars in 927 bytes} [Tue Sep 13 03:06:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 13 03:06:54.922793 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f26056e-8c9e-41d2-ac9c-42d12d2bc18b None None] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:06:54.923083 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f26056e-8c9e-41d2-ac9c-42d12d2bc18b None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:06:54.923423 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f26056e-8c9e-41d2-ac9c-42d12d2bc18b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:06:54.940155 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bafb708e-b9ad-479c-ba8d-7469e5999839 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:06:54.940781 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bafb708e-b9ad-479c-ba8d-7469e5999839 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:06:54.941059 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bafb708e-b9ad-479c-ba8d-7469e5999839 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:06:54.941328 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bafb708e-b9ad-479c-ba8d-7469e5999839 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:06:54.942166 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.auth.core [None req-2f26056e-8c9e-41d2-ac9c-42d12d2bc18b None None] MFA Rules not processed for user `49f280ab086b4610855c4ffa378517d9`. Rule list: `[]` (Enabled: `True`). {{(pid=81399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 03:06:54.943376 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-2f26056e-8c9e-41d2-ac9c-42d12d2bc18b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:06:54.951412 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bafb708e-b9ad-479c-ba8d-7469e5999839 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 03:06:54.956805 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application [None req-bafb708e-b9ad-479c-ba8d-7469e5999839 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 13 03:06:54.956805 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 03:06:54.956805 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 03:06:54.956805 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 03:06:54.956805 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 03:06:54.956805 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 03:06:54.956805 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 03:06:54.956805 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 03:06:54.956805 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 03:06:54.956805 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 03:06:54.956805 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 03:06:54.956805 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 03:06:54.956805 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 03:06:54.956805 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 03:06:54.956805 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 03:06:54.956805 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 03:06:54.957666 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 03:06:54.957666 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 03:06:54.957666 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 03:06:54.957666 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 03:06:54.957666 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 03:06:54.957666 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 03:06:54.957666 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 03:06:54.957666 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 03:06:54.957666 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 03:06:54.957666 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 03:06:54.957666 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 03:06:54.957666 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application with Lock( Sep 13 03:06:54.957666 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 03:06:54.957666 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._enter() Sep 13 03:06:54.957666 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 03:06:54.957666 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 03:06:54.958433 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 03:06:54.958433 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.creator() Sep 13 03:06:54.958433 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 03:06:54.958433 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application created_value = creator( Sep 13 03:06:54.958433 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 03:06:54.958433 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 03:06:54.958433 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 03:06:54.958433 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 03:06:54.958433 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 03:06:54.958433 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 03:06:54.958433 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 13 03:06:54.958433 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Sep 13 03:06:54.958433 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 25/50] 10.222.0.18 () {56 vars in 1096 bytes} [Tue Sep 13 03:06:54 2022] GET /identity/v3/roles/admin => generated 83 bytes in 262 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 03:06:54.964147 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bafb708e-b9ad-479c-ba8d-7469e5999839 None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:06:55.053597 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:06:55.074240 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bafb708e-b9ad-479c-ba8d-7469e5999839 None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:06:55.109820 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac71601c-ffd0-4ea9-9f7b-1cb68b7f584d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:06:55.110429 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac71601c-ffd0-4ea9-9f7b-1cb68b7f584d None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:06:55.110744 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac71601c-ffd0-4ea9-9f7b-1cb68b7f584d None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:06:55.111022 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac71601c-ffd0-4ea9-9f7b-1cb68b7f584d None admin] PATH_INFO: `/v3/roles` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:06:55.111712 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac71601c-ffd0-4ea9-9f7b-1cb68b7f584d None admin] RBAC: Adding query filter params (name=admin) {{(pid=81400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 03:06:55.112012 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac71601c-ffd0-4ea9-9f7b-1cb68b7f584d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:06:55.113403 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac71601c-ffd0-4ea9-9f7b-1cb68b7f584d None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:06:55.114550 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 26/51] 10.222.0.18 () {58 vars in 1024 bytes} [Tue Sep 13 03:06:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 13 03:06:55.130731 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 26/52] 10.222.0.18 () {56 vars in 1099 bytes} [Tue Sep 13 03:06:54 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 03:06:55.132003 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f26056e-8c9e-41d2-ac9c-42d12d2bc18b None None] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:06:55.185223 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:06:55.234228 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f26056e-8c9e-41d2-ac9c-42d12d2bc18b None None] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:06:55.265914 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2ce5300-bfb3-4fb4-b9af-3a741c91263d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:06:55.266539 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2ce5300-bfb3-4fb4-b9af-3a741c91263d None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:06:55.266845 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2ce5300-bfb3-4fb4-b9af-3a741c91263d None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:06:55.267115 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2ce5300-bfb3-4fb4-b9af-3a741c91263d None admin] PATH_INFO: `/v3/users/49f280ab086b4610855c4ffa378517d9` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:06:55.268832 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2ce5300-bfb3-4fb4-b9af-3a741c91263d None admin] RBAC: Authorizing `identity:get_user(user_id=49f280ab086b4610855c4ffa378517d9)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:06:55.270260 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2ce5300-bfb3-4fb4-b9af-3a741c91263d None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:06:55.271851 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 27/53] 10.222.0.18 () {56 vars in 1177 bytes} [Tue Sep 13 03:06:55 2022] GET /identity/v3/users/49f280ab086b4610855c4ffa378517d9 => generated 247 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 03:06:55.277303 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac71601c-ffd0-4ea9-9f7b-1cb68b7f584d None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:06:55.373066 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac71601c-ffd0-4ea9-9f7b-1cb68b7f584d None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:06:55.378011 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2ce5300-bfb3-4fb4-b9af-3a741c91263d None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:06:55.406314 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ac6113d-e9b4-44ee-8f90-2e02611eadea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:06:55.406987 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac6113d-e9b4-44ee-8f90-2e02611eadea None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:06:55.407271 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac6113d-e9b4-44ee-8f90-2e02611eadea None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:06:55.407544 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac6113d-e9b4-44ee-8f90-2e02611eadea None admin] PATH_INFO: `/v3/domains/default` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:06:55.408693 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ac6113d-e9b4-44ee-8f90-2e02611eadea None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:06:55.410136 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ac6113d-e9b4-44ee-8f90-2e02611eadea None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:06:55.411742 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 27/54] 10.222.0.18 () {56 vars in 1108 bytes} [Tue Sep 13 03:06:55 2022] GET /identity/v3/domains/default => generated 199 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 03:06:55.417860 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ac6113d-e9b4-44ee-8f90-2e02611eadea None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:06:55.436773 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:06:55.477722 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2ce5300-bfb3-4fb4-b9af-3a741c91263d None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:06:55.500090 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:06:55.511867 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d678e22-1076-400b-b8f4-b1cb5f3aefbc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:06:55.512497 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d678e22-1076-400b-b8f4-b1cb5f3aefbc None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:06:55.512778 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d678e22-1076-400b-b8f4-b1cb5f3aefbc None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:06:55.513047 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d678e22-1076-400b-b8f4-b1cb5f3aefbc None admin] PATH_INFO: `/v3/domains/default` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:06:55.514143 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d678e22-1076-400b-b8f4-b1cb5f3aefbc None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:06:55.515605 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d678e22-1076-400b-b8f4-b1cb5f3aefbc None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:06:55.517166 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 28/55] 10.222.0.18 () {56 vars in 1108 bytes} [Tue Sep 13 03:06:55 2022] GET /identity/v3/domains/default => generated 199 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 03:06:55.519015 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ac6113d-e9b4-44ee-8f90-2e02611eadea None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:06:55.522593 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d678e22-1076-400b-b8f4-b1cb5f3aefbc None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:06:55.552484 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba3bb0ab-5bf6-4f14-81af-84fce08e51dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:06:55.553092 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba3bb0ab-5bf6-4f14-81af-84fce08e51dd None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:06:55.553375 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba3bb0ab-5bf6-4f14-81af-84fce08e51dd None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:06:55.553653 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba3bb0ab-5bf6-4f14-81af-84fce08e51dd None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:06:55.554375 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba3bb0ab-5bf6-4f14-81af-84fce08e51dd None admin] RBAC: Adding query filter params (role.id=425d4b481c7846e0bc07e7e805d5b15f, scope.domain.id=default, user.id=49f280ab086b4610855c4ffa378517d9) {{(pid=81400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 03:06:55.554719 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba3bb0ab-5bf6-4f14-81af-84fce08e51dd None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:06:55.556168 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba3bb0ab-5bf6-4f14-81af-84fce08e51dd None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:06:55.562186 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 28/56] 10.222.0.18 () {56 vars in 1322 bytes} [Tue Sep 13 03:06:55 2022] GET /identity/v3/role_assignments?user.id=49f280ab086b4610855c4ffa378517d9&scope.domain.id=default&role.id=425d4b481c7846e0bc07e7e805d5b15f => generated 530 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 03:06:55.625278 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d678e22-1076-400b-b8f4-b1cb5f3aefbc None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:06:55.661077 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09c4662b-094b-4361-ad9d-8fa30fa16d09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:06:55.661718 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c4662b-094b-4361-ad9d-8fa30fa16d09 None admin] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:06:55.662003 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c4662b-094b-4361-ad9d-8fa30fa16d09 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:06:55.662279 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c4662b-094b-4361-ad9d-8fa30fa16d09 None admin] PATH_INFO: `/v3/projects` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:06:55.662980 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09c4662b-094b-4361-ad9d-8fa30fa16d09 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:06:55.664455 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09c4662b-094b-4361-ad9d-8fa30fa16d09 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:06:55.693350 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 29/57] 10.222.0.18 () {60 vars in 1140 bytes} [Tue Sep 13 03:06:55 2022] POST /identity/v3/projects => generated 298 bytes in 171 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 13 03:06:57.820146 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ab41ac-7861-4ed9-be35-1c0a8ff5dcb6 None None] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:06:57.820662 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ab41ac-7861-4ed9-be35-1c0a8ff5dcb6 None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:06:57.820744 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ab41ac-7861-4ed9-be35-1c0a8ff5dcb6 None None] PATH_INFO: `/` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:06:57.822046 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 29/58] 10.222.0.18 () {54 vars in 927 bytes} [Tue Sep 13 03:06:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 03:06:57.827235 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-141dd115-6a42-4eef-bf83-81772dc3979d None None] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:06:57.827612 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-141dd115-6a42-4eef-bf83-81772dc3979d None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:06:57.827780 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-141dd115-6a42-4eef-bf83-81772dc3979d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:06:57.846254 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.auth.core [None req-141dd115-6a42-4eef-bf83-81772dc3979d None None] MFA Rules not processed for user `49f280ab086b4610855c4ffa378517d9`. Rule list: `[]` (Enabled: `True`). {{(pid=81399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 03:06:57.847473 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-141dd115-6a42-4eef-bf83-81772dc3979d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:06:57.884863 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:06:57.911491 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 30/59] 10.222.0.18 () {58 vars in 1024 bytes} [Tue Sep 13 03:06:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 13 03:06:58.176312 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03ab41ac-7861-4ed9-be35-1c0a8ff5dcb6 None None] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:06:58.256413 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:06:58.275939 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03ab41ac-7861-4ed9-be35-1c0a8ff5dcb6 None None] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:06:58.307958 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc1fbb08-7a98-40e6-8ec1-3a0b17f86775 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:06:58.308566 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc1fbb08-7a98-40e6-8ec1-3a0b17f86775 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:06:58.308841 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc1fbb08-7a98-40e6-8ec1-3a0b17f86775 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:06:58.309110 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc1fbb08-7a98-40e6-8ec1-3a0b17f86775 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:06:58.310193 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc1fbb08-7a98-40e6-8ec1-3a0b17f86775 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:06:58.311656 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc1fbb08-7a98-40e6-8ec1-3a0b17f86775 None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:06:58.313185 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 30/60] 10.222.0.18 () {56 vars in 1108 bytes} [Tue Sep 13 03:06:58 2022] GET /identity/v3/domains/default => generated 199 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 03:06:58.319072 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-141dd115-6a42-4eef-bf83-81772dc3979d None None] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:06:58.410809 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-141dd115-6a42-4eef-bf83-81772dc3979d None None] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:06:58.443308 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4435a6a0-f183-476c-bb18-b83c35aff41a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:06:58.443851 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4435a6a0-f183-476c-bb18-b83c35aff41a None admin] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:06:58.444127 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4435a6a0-f183-476c-bb18-b83c35aff41a None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:06:58.444395 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4435a6a0-f183-476c-bb18-b83c35aff41a None admin] PATH_INFO: `/v3/users` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:06:58.445041 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4435a6a0-f183-476c-bb18-b83c35aff41a None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:06:58.446477 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4435a6a0-f183-476c-bb18-b83c35aff41a None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:06:58.472031 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 31/61] 10.222.0.18 () {60 vars in 1132 bytes} [Tue Sep 13 03:06:58 2022] POST /identity/v3/users => generated 275 bytes in 153 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 03:07:01.940251 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d4b7ec7-50cf-48a3-831c-50406b9d6e1f None None] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:07:01.941256 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d4b7ec7-50cf-48a3-831c-50406b9d6e1f None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:07:01.941738 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d4b7ec7-50cf-48a3-831c-50406b9d6e1f None None] PATH_INFO: `/` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:07:01.943667 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 31/62] 10.222.0.18 () {54 vars in 927 bytes} [Tue Sep 13 03:07:01 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 03:07:01.948923 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-605f3237-8114-4188-9895-a03d84c2737f None None] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:07:01.949346 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-605f3237-8114-4188-9895-a03d84c2737f None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:07:01.949481 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-605f3237-8114-4188-9895-a03d84c2737f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:07:01.961128 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a78f35a-fa00-4bbf-962e-f1ed614b8707 None None] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:07:01.961405 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a78f35a-fa00-4bbf-962e-f1ed614b8707 None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:07:01.961682 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a78f35a-fa00-4bbf-962e-f1ed614b8707 None None] PATH_INFO: `/` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:07:01.963050 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 32/63] 10.222.0.18 () {54 vars in 927 bytes} [Tue Sep 13 03:07:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 13 03:07:01.971245 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71da7ebf-720d-46d5-8cd9-2a2bb260f01b None None] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:07:01.971758 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71da7ebf-720d-46d5-8cd9-2a2bb260f01b None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:07:01.972219 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71da7ebf-720d-46d5-8cd9-2a2bb260f01b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:07:01.976192 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.auth.core [None req-605f3237-8114-4188-9895-a03d84c2737f None None] MFA Rules not processed for user `49f280ab086b4610855c4ffa378517d9`. Rule list: `[]` (Enabled: `True`). {{(pid=81399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 03:07:01.977389 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-605f3237-8114-4188-9895-a03d84c2737f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:07:01.998562 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.auth.core [None req-71da7ebf-720d-46d5-8cd9-2a2bb260f01b None None] MFA Rules not processed for user `49f280ab086b4610855c4ffa378517d9`. Rule list: `[]` (Enabled: `True`). {{(pid=81400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 03:07:01.999805 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-71da7ebf-720d-46d5-8cd9-2a2bb260f01b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:07:02.024033 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:07:02.088534 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 32/64] 10.222.0.18 () {58 vars in 1024 bytes} [Tue Sep 13 03:07:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 13 03:07:02.091000 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f6402bc-b5e0-461f-9bc3-0b80cf54a38c None None] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:07:02.091531 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f6402bc-b5e0-461f-9bc3-0b80cf54a38c None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:07:02.091951 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f6402bc-b5e0-461f-9bc3-0b80cf54a38c None None] PATH_INFO: `/` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:07:02.093913 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 33/65] 10.222.0.18 () {54 vars in 927 bytes} [Tue Sep 13 03:07:02 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 03:07:02.100194 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d84120c-0b4b-4c49-bc5d-4caefc0a2608 None None] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:07:02.100489 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d84120c-0b4b-4c49-bc5d-4caefc0a2608 None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:07:02.100773 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d84120c-0b4b-4c49-bc5d-4caefc0a2608 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:07:02.114328 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 33/66] 10.222.0.18 () {58 vars in 1024 bytes} [Tue Sep 13 03:07:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 13 03:07:02.125515 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.auth.core [None req-7d84120c-0b4b-4c49-bc5d-4caefc0a2608 None None] MFA Rules not processed for user `49f280ab086b4610855c4ffa378517d9`. Rule list: `[]` (Enabled: `True`). {{(pid=81399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 03:07:02.126766 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-7d84120c-0b4b-4c49-bc5d-4caefc0a2608 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:07:02.218543 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d8616b-e645-4dd7-9018-c538501b299a None None] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:07:02.219181 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d8616b-e645-4dd7-9018-c538501b299a None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:07:02.219643 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d8616b-e645-4dd7-9018-c538501b299a None None] PATH_INFO: `/` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:07:02.221789 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 34/67] 10.222.0.18 () {54 vars in 927 bytes} [Tue Sep 13 03:07:02 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 13 03:07:02.225143 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 34/68] 10.222.0.18 () {58 vars in 1024 bytes} [Tue Sep 13 03:07:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 13 03:07:02.227404 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ec70f4-775c-4eb5-9b9c-171ab6a842fe None None] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:07:02.227890 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ec70f4-775c-4eb5-9b9c-171ab6a842fe None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:07:02.228373 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ec70f4-775c-4eb5-9b9c-171ab6a842fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:07:02.246328 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.auth.core [None req-74ec70f4-775c-4eb5-9b9c-171ab6a842fe None None] MFA Rules not processed for user `49f280ab086b4610855c4ffa378517d9`. Rule list: `[]` (Enabled: `True`). {{(pid=81400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 03:07:02.247820 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-74ec70f4-775c-4eb5-9b9c-171ab6a842fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:07:02.300793 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ea9e40-cd0f-44dc-a22a-e852114e4c44 None None] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:07:02.301400 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ea9e40-cd0f-44dc-a22a-e852114e4c44 None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:07:02.301868 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ea9e40-cd0f-44dc-a22a-e852114e4c44 None None] PATH_INFO: `/` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:07:02.303453 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 35/69] 10.222.0.18 () {54 vars in 927 bytes} [Tue Sep 13 03:07:02 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 13 03:07:02.311794 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4609afa2-dc36-46c5-bc89-b86a594d8da1 None None] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:07:02.312299 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4609afa2-dc36-46c5-bc89-b86a594d8da1 None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:07:02.312767 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4609afa2-dc36-46c5-bc89-b86a594d8da1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:07:02.337233 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 35/70] 10.222.0.18 () {58 vars in 1024 bytes} [Tue Sep 13 03:07:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 13 03:07:02.347601 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.auth.core [None req-4609afa2-dc36-46c5-bc89-b86a594d8da1 None None] MFA Rules not processed for user `49f280ab086b4610855c4ffa378517d9`. Rule list: `[]` (Enabled: `True`). {{(pid=81399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 03:07:02.348839 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-4609afa2-dc36-46c5-bc89-b86a594d8da1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:07:02.419826 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74ec70f4-775c-4eb5-9b9c-171ab6a842fe None None] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:07:02.433771 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:07:02.456215 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 36/71] 10.222.0.18 () {58 vars in 1024 bytes} [Tue Sep 13 03:07:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 13 03:07:02.496419 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4609afa2-dc36-46c5-bc89-b86a594d8da1 None None] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:07:02.523523 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:07:02.604983 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74ec70f4-775c-4eb5-9b9c-171ab6a842fe None None] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:07:02.644422 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4609afa2-dc36-46c5-bc89-b86a594d8da1 None None] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:07:02.646089 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b665632c-9723-4302-8d88-f09fe15be899 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:07:02.646772 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b665632c-9723-4302-8d88-f09fe15be899 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:07:02.647076 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b665632c-9723-4302-8d88-f09fe15be899 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:07:02.647431 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b665632c-9723-4302-8d88-f09fe15be899 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:07:02.658946 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b665632c-9723-4302-8d88-f09fe15be899 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 03:07:02.665281 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application [None req-b665632c-9723-4302-8d88-f09fe15be899 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 13 03:07:02.665281 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 03:07:02.665281 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 03:07:02.665281 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 03:07:02.665281 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 03:07:02.665281 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 03:07:02.665281 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 03:07:02.665281 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 03:07:02.665281 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 03:07:02.665281 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 03:07:02.665281 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 03:07:02.665281 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 03:07:02.665281 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 03:07:02.665281 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 03:07:02.665281 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 03:07:02.665281 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 03:07:02.666130 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 03:07:02.666130 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 03:07:02.666130 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 03:07:02.666130 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 03:07:02.666130 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 03:07:02.666130 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 03:07:02.666130 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 03:07:02.666130 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 03:07:02.666130 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 03:07:02.666130 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 03:07:02.666130 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 03:07:02.666130 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application with Lock( Sep 13 03:07:02.666130 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 03:07:02.666130 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._enter() Sep 13 03:07:02.666130 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 03:07:02.666130 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 03:07:02.667368 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 03:07:02.667368 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.creator() Sep 13 03:07:02.667368 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 03:07:02.667368 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application created_value = creator( Sep 13 03:07:02.667368 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 03:07:02.667368 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 03:07:02.667368 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 03:07:02.667368 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 03:07:02.667368 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 03:07:02.667368 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 03:07:02.667368 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 13 03:07:02.667368 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Sep 13 03:07:02.667368 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 36/72] 10.222.0.18 () {56 vars in 1114 bytes} [Tue Sep 13 03:07:02 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 248 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 13 03:07:02.668260 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b665632c-9723-4302-8d88-f09fe15be899 None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:07:02.677842 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b7f98de-5cda-493a-a427-16183f791992 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:07:02.678482 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b7f98de-5cda-493a-a427-16183f791992 None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:07:02.678797 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b7f98de-5cda-493a-a427-16183f791992 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:07:02.679070 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b7f98de-5cda-493a-a427-16183f791992 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:07:02.691133 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b7f98de-5cda-493a-a427-16183f791992 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 03:07:02.697686 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application [None req-6b7f98de-5cda-493a-a427-16183f791992 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 13 03:07:02.697686 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 03:07:02.697686 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 03:07:02.697686 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 03:07:02.697686 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 03:07:02.697686 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 03:07:02.697686 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 03:07:02.697686 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 03:07:02.697686 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 03:07:02.697686 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 03:07:02.697686 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 03:07:02.697686 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 03:07:02.697686 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 03:07:02.697686 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 03:07:02.697686 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 03:07:02.697686 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 03:07:02.698727 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 03:07:02.698727 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 03:07:02.698727 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 03:07:02.698727 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 03:07:02.698727 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 03:07:02.698727 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 03:07:02.698727 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 03:07:02.698727 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 03:07:02.698727 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 03:07:02.698727 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 03:07:02.698727 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 03:07:02.698727 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application with Lock( Sep 13 03:07:02.698727 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 03:07:02.698727 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._enter() Sep 13 03:07:02.698727 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 03:07:02.698727 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 03:07:02.699715 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 03:07:02.699715 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.creator() Sep 13 03:07:02.699715 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 03:07:02.699715 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application created_value = creator( Sep 13 03:07:02.699715 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 03:07:02.699715 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 03:07:02.699715 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 03:07:02.699715 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 03:07:02.699715 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 03:07:02.699715 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 03:07:02.699715 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 13 03:07:02.699715 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Sep 13 03:07:02.700306 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 37/73] 10.222.0.18 () {56 vars in 1096 bytes} [Tue Sep 13 03:07:02 2022] GET /identity/v3/roles/admin => generated 83 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 13 03:07:02.703571 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b7f98de-5cda-493a-a427-16183f791992 None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:07:02.753204 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:07:02.790900 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:07:02.791695 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b665632c-9723-4302-8d88-f09fe15be899 None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:07:02.810168 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b7f98de-5cda-493a-a427-16183f791992 None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:07:02.826586 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-027666b0-1ba7-4d79-8522-3efe8df71242 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:07:02.827238 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-027666b0-1ba7-4d79-8522-3efe8df71242 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:07:02.827520 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-027666b0-1ba7-4d79-8522-3efe8df71242 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:07:02.827793 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-027666b0-1ba7-4d79-8522-3efe8df71242 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:07:02.837876 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-027666b0-1ba7-4d79-8522-3efe8df71242 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 03:07:02.842806 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application [None req-027666b0-1ba7-4d79-8522-3efe8df71242 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 13 03:07:02.842806 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 03:07:02.842806 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 03:07:02.842806 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 03:07:02.842806 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 03:07:02.842806 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 03:07:02.842806 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 03:07:02.842806 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 03:07:02.842806 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 03:07:02.842806 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 03:07:02.842806 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 03:07:02.842806 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 03:07:02.842806 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 03:07:02.842806 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 03:07:02.842806 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 03:07:02.842806 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 03:07:02.843664 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 03:07:02.843664 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 03:07:02.843664 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 03:07:02.843664 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 03:07:02.843664 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 03:07:02.843664 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 03:07:02.843664 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 03:07:02.843664 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 03:07:02.843664 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 03:07:02.843664 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 03:07:02.843664 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 03:07:02.843664 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application with Lock( Sep 13 03:07:02.843664 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 03:07:02.843664 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._enter() Sep 13 03:07:02.843664 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 03:07:02.843664 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 03:07:02.844435 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 03:07:02.844435 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.creator() Sep 13 03:07:02.844435 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 03:07:02.844435 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application created_value = creator( Sep 13 03:07:02.844435 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 03:07:02.844435 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 03:07:02.844435 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 03:07:02.844435 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 03:07:02.844435 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 03:07:02.844435 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 03:07:02.844435 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 13 03:07:02.844435 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Sep 13 03:07:02.845032 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-194d0a3c-1964-4479-9965-d5385a3b3b5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:07:02.845032 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-194d0a3c-1964-4479-9965-d5385a3b3b5a None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:07:02.845032 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-194d0a3c-1964-4479-9965-d5385a3b3b5a None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:07:02.845216 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 37/74] 10.222.0.18 () {56 vars in 1099 bytes} [Tue Sep 13 03:07:02 2022] GET /identity/v3/roles/member => generated 84 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 03:07:02.845280 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-194d0a3c-1964-4479-9965-d5385a3b3b5a None admin] PATH_INFO: `/v3/roles/member` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:07:02.845353 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-027666b0-1ba7-4d79-8522-3efe8df71242 None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:07:02.858914 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-194d0a3c-1964-4479-9965-d5385a3b3b5a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 03:07:02.864393 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application [None req-194d0a3c-1964-4479-9965-d5385a3b3b5a None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 13 03:07:02.864393 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 03:07:02.864393 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 03:07:02.864393 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 03:07:02.864393 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 03:07:02.864393 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 03:07:02.864393 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 03:07:02.864393 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 03:07:02.864393 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 03:07:02.864393 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 03:07:02.864393 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 03:07:02.864393 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 03:07:02.864393 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 03:07:02.864393 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 03:07:02.864393 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 03:07:02.864393 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 03:07:02.865282 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 03:07:02.865282 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 03:07:02.865282 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 03:07:02.865282 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 03:07:02.865282 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 03:07:02.865282 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 03:07:02.865282 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 03:07:02.865282 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 03:07:02.865282 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 03:07:02.865282 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 03:07:02.865282 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 03:07:02.865282 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application with Lock( Sep 13 03:07:02.865282 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 03:07:02.865282 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._enter() Sep 13 03:07:02.865282 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 03:07:02.865282 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 03:07:02.866033 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 03:07:02.866033 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.creator() Sep 13 03:07:02.866033 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 03:07:02.866033 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application created_value = creator( Sep 13 03:07:02.866033 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 03:07:02.866033 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 03:07:02.866033 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 03:07:02.866033 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 03:07:02.866033 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 03:07:02.866033 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 03:07:02.866033 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 13 03:07:02.866033 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Sep 13 03:07:02.866033 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 38/75] 10.222.0.18 () {56 vars in 1099 bytes} [Tue Sep 13 03:07:02 2022] GET /identity/v3/roles/member => generated 84 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 03:07:02.866692 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-194d0a3c-1964-4479-9965-d5385a3b3b5a None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:07:02.946839 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-027666b0-1ba7-4d79-8522-3efe8df71242 None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:07:02.963219 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-194d0a3c-1964-4479-9965-d5385a3b3b5a None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:07:02.972769 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:07:02.980794 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00e36a63-1d56-4fd2-bdc3-d88f57388e94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:07:02.981396 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e36a63-1d56-4fd2-bdc3-d88f57388e94 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:07:02.981684 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e36a63-1d56-4fd2-bdc3-d88f57388e94 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:07:02.981957 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e36a63-1d56-4fd2-bdc3-d88f57388e94 None admin] PATH_INFO: `/v3/roles` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:07:02.982632 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00e36a63-1d56-4fd2-bdc3-d88f57388e94 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 03:07:02.982944 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00e36a63-1d56-4fd2-bdc3-d88f57388e94 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:07:02.984335 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00e36a63-1d56-4fd2-bdc3-d88f57388e94 None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:07:02.992710 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 38/76] 10.222.0.18 () {56 vars in 1111 bytes} [Tue Sep 13 03:07:02 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 03:07:02.995426 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00e36a63-1d56-4fd2-bdc3-d88f57388e94 None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:07:02.996309 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-153f01d2-267f-462a-8908-63410e594bb4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:07:02.996929 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-153f01d2-267f-462a-8908-63410e594bb4 None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:07:02.997215 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-153f01d2-267f-462a-8908-63410e594bb4 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:07:02.997533 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-153f01d2-267f-462a-8908-63410e594bb4 None admin] PATH_INFO: `/v3/roles` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:07:02.998208 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-153f01d2-267f-462a-8908-63410e594bb4 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 03:07:02.998514 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-153f01d2-267f-462a-8908-63410e594bb4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:07:02.999960 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-153f01d2-267f-462a-8908-63410e594bb4 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:07:03.009312 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 39/77] 10.222.0.18 () {56 vars in 1099 bytes} [Tue Sep 13 03:07:02 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 03:07:03.010011 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-153f01d2-267f-462a-8908-63410e594bb4 None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:07:03.012677 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:07:03.099825 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00e36a63-1d56-4fd2-bdc3-d88f57388e94 None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:07:03.109808 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-153f01d2-267f-462a-8908-63410e594bb4 None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:07:03.133336 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06e35528-2a71-4075-92f7-58c70d527146 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:07:03.133959 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e35528-2a71-4075-92f7-58c70d527146 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:07:03.134234 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e35528-2a71-4075-92f7-58c70d527146 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:07:03.134502 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e35528-2a71-4075-92f7-58c70d527146 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:07:03.135628 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06e35528-2a71-4075-92f7-58c70d527146 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:07:03.137065 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06e35528-2a71-4075-92f7-58c70d527146 None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:07:03.138643 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 39/78] 10.222.0.18 () {56 vars in 1108 bytes} [Tue Sep 13 03:07:02 2022] GET /identity/v3/domains/default => generated 199 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 03:07:03.139382 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06e35528-2a71-4075-92f7-58c70d527146 None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:07:03.147458 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-771d523a-5aa8-478b-8547-90d86693fe63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:07:03.148103 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-771d523a-5aa8-478b-8547-90d86693fe63 None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:07:03.148390 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-771d523a-5aa8-478b-8547-90d86693fe63 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:07:03.148663 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-771d523a-5aa8-478b-8547-90d86693fe63 None admin] PATH_INFO: `/v3/roles` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:07:03.149327 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-771d523a-5aa8-478b-8547-90d86693fe63 None admin] RBAC: Adding query filter params (name=member) {{(pid=81399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 03:07:03.149891 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-771d523a-5aa8-478b-8547-90d86693fe63 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:07:03.151241 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-771d523a-5aa8-478b-8547-90d86693fe63 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:07:03.159842 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 40/79] 10.222.0.18 () {56 vars in 1101 bytes} [Tue Sep 13 03:07:03 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 03:07:03.160552 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-771d523a-5aa8-478b-8547-90d86693fe63 None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:07:03.196796 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:07:03.220196 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:07:03.239391 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06e35528-2a71-4075-92f7-58c70d527146 None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:07:03.258336 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-771d523a-5aa8-478b-8547-90d86693fe63 None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:07:03.271458 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5183b843-3461-40cb-8bac-2f36729d5170 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:07:03.272002 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5183b843-3461-40cb-8bac-2f36729d5170 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:07:03.272284 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5183b843-3461-40cb-8bac-2f36729d5170 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:07:03.272558 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5183b843-3461-40cb-8bac-2f36729d5170 None admin] PATH_INFO: `/v3/roles` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:07:03.273223 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5183b843-3461-40cb-8bac-2f36729d5170 None admin] RBAC: Adding query filter params (name=member) {{(pid=81400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 03:07:03.273531 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5183b843-3461-40cb-8bac-2f36729d5170 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:07:03.274930 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5183b843-3461-40cb-8bac-2f36729d5170 None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:07:03.283360 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 40/80] 10.222.0.18 () {56 vars in 1101 bytes} [Tue Sep 13 03:07:03 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 03:07:03.284154 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5183b843-3461-40cb-8bac-2f36729d5170 None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:07:03.296393 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0ec333b-3065-46ca-8394-c10aac1fae4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:07:03.297026 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0ec333b-3065-46ca-8394-c10aac1fae4c None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:07:03.297303 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0ec333b-3065-46ca-8394-c10aac1fae4c None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:07:03.297575 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0ec333b-3065-46ca-8394-c10aac1fae4c None admin] PATH_INFO: `/v3/users/06632bd878b1474f8d5432f99e899155` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:07:03.312819 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0ec333b-3065-46ca-8394-c10aac1fae4c None admin] RBAC: Authorizing `identity:get_user(user_id=06632bd878b1474f8d5432f99e899155)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:07:03.314291 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0ec333b-3065-46ca-8394-c10aac1fae4c None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:07:03.315935 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 41/81] 10.222.0.18 () {56 vars in 1177 bytes} [Tue Sep 13 03:07:03 2022] GET /identity/v3/users/06632bd878b1474f8d5432f99e899155 => generated 275 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 03:07:03.316668 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0ec333b-3065-46ca-8394-c10aac1fae4c None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:07:03.384710 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5183b843-3461-40cb-8bac-2f36729d5170 None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:07:03.403458 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:07:03.411624 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0ec333b-3065-46ca-8394-c10aac1fae4c None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:07:03.418034 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86ed2aa7-8cb5-42c6-839c-d67f35eb0e86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:07:03.418702 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86ed2aa7-8cb5-42c6-839c-d67f35eb0e86 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:07:03.418982 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86ed2aa7-8cb5-42c6-839c-d67f35eb0e86 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:07:03.419256 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86ed2aa7-8cb5-42c6-839c-d67f35eb0e86 None admin] PATH_INFO: `/v3/users/49f280ab086b4610855c4ffa378517d9` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:07:03.420378 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86ed2aa7-8cb5-42c6-839c-d67f35eb0e86 None admin] RBAC: Authorizing `identity:get_user(user_id=49f280ab086b4610855c4ffa378517d9)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:07:03.421846 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86ed2aa7-8cb5-42c6-839c-d67f35eb0e86 None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:07:03.424232 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 41/82] 10.222.0.18 () {56 vars in 1177 bytes} [Tue Sep 13 03:07:03 2022] GET /identity/v3/users/49f280ab086b4610855c4ffa378517d9 => generated 247 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 03:07:03.424840 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86ed2aa7-8cb5-42c6-839c-d67f35eb0e86 None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:07:03.438014 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:07:03.449626 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-590710da-2aae-4e24-9920-b2f3178ae61a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:07:03.450228 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-590710da-2aae-4e24-9920-b2f3178ae61a None admin] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:07:03.450511 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-590710da-2aae-4e24-9920-b2f3178ae61a None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:07:03.450811 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-590710da-2aae-4e24-9920-b2f3178ae61a None admin] PATH_INFO: `/v3/users` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:07:03.451448 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-590710da-2aae-4e24-9920-b2f3178ae61a None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:07:03.452882 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-590710da-2aae-4e24-9920-b2f3178ae61a None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:07:03.475152 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 42/83] 10.222.0.18 () {60 vars in 1132 bytes} [Tue Sep 13 03:07:03 2022] POST /identity/v3/users => generated 289 bytes in 159 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 03:07:03.475956 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-590710da-2aae-4e24-9920-b2f3178ae61a None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:07:03.537958 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86ed2aa7-8cb5-42c6-839c-d67f35eb0e86 None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:07:03.575297 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d48bde5-7bbf-45c1-853c-b3572087fc58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:07:03.575954 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d48bde5-7bbf-45c1-853c-b3572087fc58 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:07:03.576240 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d48bde5-7bbf-45c1-853c-b3572087fc58 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:07:03.576519 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d48bde5-7bbf-45c1-853c-b3572087fc58 None admin] PATH_INFO: `/v3/users/06632bd878b1474f8d5432f99e899155` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:07:03.578342 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d48bde5-7bbf-45c1-853c-b3572087fc58 None admin] RBAC: Authorizing `identity:get_user(user_id=06632bd878b1474f8d5432f99e899155)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:07:03.580810 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d48bde5-7bbf-45c1-853c-b3572087fc58 None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:07:03.583294 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 42/84] 10.222.0.18 () {56 vars in 1177 bytes} [Tue Sep 13 03:07:03 2022] GET /identity/v3/users/06632bd878b1474f8d5432f99e899155 => generated 275 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 03:07:03.583898 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d48bde5-7bbf-45c1-853c-b3572087fc58 None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:07:03.616541 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-590710da-2aae-4e24-9920-b2f3178ae61a None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:07:03.649584 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:07:03.659326 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3071935c-f8a6-4daa-b47f-f69e597f5e38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:07:03.660038 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3071935c-f8a6-4daa-b47f-f69e597f5e38 None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:07:03.660320 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3071935c-f8a6-4daa-b47f-f69e597f5e38 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:07:03.660590 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3071935c-f8a6-4daa-b47f-f69e597f5e38 None admin] PATH_INFO: `/v3/users/06632bd878b1474f8d5432f99e899155` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:07:03.663352 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3071935c-f8a6-4daa-b47f-f69e597f5e38 None admin] RBAC: Authorizing `identity:get_user(user_id=06632bd878b1474f8d5432f99e899155)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:07:03.664505 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3071935c-f8a6-4daa-b47f-f69e597f5e38 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:07:03.666700 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 43/85] 10.222.0.18 () {56 vars in 1177 bytes} [Tue Sep 13 03:07:03 2022] GET /identity/v3/users/06632bd878b1474f8d5432f99e899155 => generated 275 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 03:07:03.666700 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3071935c-f8a6-4daa-b47f-f69e597f5e38 None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:07:03.702800 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d48bde5-7bbf-45c1-853c-b3572087fc58 None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:07:03.719691 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:07:03.746875 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f5bf962-9e87-4fa3-9c55-61dbf45d68f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:07:03.747209 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f5bf962-9e87-4fa3-9c55-61dbf45d68f4 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:07:03.747474 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f5bf962-9e87-4fa3-9c55-61dbf45d68f4 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:07:03.747724 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f5bf962-9e87-4fa3-9c55-61dbf45d68f4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:07:03.797196 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3071935c-f8a6-4daa-b47f-f69e597f5e38 None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:07:03.830217 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f5bf962-9e87-4fa3-9c55-61dbf45d68f4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:07:03.831851 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f5bf962-9e87-4fa3-9c55-61dbf45d68f4 None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:07:03.846363 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06d817c6-421c-415e-ad93-24613199a2de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:07:03.847322 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d817c6-421c-415e-ad93-24613199a2de None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:07:03.847627 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d817c6-421c-415e-ad93-24613199a2de None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:07:03.847896 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d817c6-421c-415e-ad93-24613199a2de None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:07:03.909171 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:07:03.925509 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06d817c6-421c-415e-ad93-24613199a2de None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:07:03.927301 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06d817c6-421c-415e-ad93-24613199a2de None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:07:03.965592 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 43/86] 10.222.0.18 () {58 vars in 1282 bytes} [Tue Sep 13 03:07:03 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 382 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 13 03:07:03.966639 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f5bf962-9e87-4fa3-9c55-61dbf45d68f4 None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:07:03.988754 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:07:04.070934 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 44/87] 10.222.0.18 () {58 vars in 1282 bytes} [Tue Sep 13 03:07:03 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 405 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 13 03:07:04.075522 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06d817c6-421c-415e-ad93-24613199a2de None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:07:04.088147 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f5bf962-9e87-4fa3-9c55-61dbf45d68f4 None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:07:04.127616 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa7abb32-9b37-4379-81af-c4206ce5efe2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:07:04.128607 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa7abb32-9b37-4379-81af-c4206ce5efe2 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:07:04.129378 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa7abb32-9b37-4379-81af-c4206ce5efe2 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:07:04.130166 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa7abb32-9b37-4379-81af-c4206ce5efe2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:07:04.164249 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:07:04.191843 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06d817c6-421c-415e-ad93-24613199a2de None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:07:04.218466 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa7abb32-9b37-4379-81af-c4206ce5efe2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:07:04.220049 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa7abb32-9b37-4379-81af-c4206ce5efe2 None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:07:04.238151 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02e8dad4-f498-47bf-ac14-2c23c589a668 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:07:04.239420 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02e8dad4-f498-47bf-ac14-2c23c589a668 None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:07:04.239881 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02e8dad4-f498-47bf-ac14-2c23c589a668 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:07:04.240353 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02e8dad4-f498-47bf-ac14-2c23c589a668 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:07:04.262482 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:07:04.323316 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02e8dad4-f498-47bf-ac14-2c23c589a668 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:07:04.324990 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02e8dad4-f498-47bf-ac14-2c23c589a668 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:07:04.335829 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 44/88] 10.222.0.18 () {58 vars in 1282 bytes} [Tue Sep 13 03:07:03 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 370 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 13 03:07:04.337042 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa7abb32-9b37-4379-81af-c4206ce5efe2 None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:07:04.396436 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:07:04.435228 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 45/89] 10.222.0.18 () {58 vars in 1282 bytes} [Tue Sep 13 03:07:04 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 356 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 13 03:07:04.436264 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02e8dad4-f498-47bf-ac14-2c23c589a668 None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:07:04.444764 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa7abb32-9b37-4379-81af-c4206ce5efe2 None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:07:04.480396 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5eca28b-cd92-424e-a22b-af14f14da610 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:07:04.481220 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5eca28b-cd92-424e-a22b-af14f14da610 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:07:04.481606 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5eca28b-cd92-424e-a22b-af14f14da610 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:07:04.481984 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5eca28b-cd92-424e-a22b-af14f14da610 None admin] PATH_INFO: `/v3/projects/83aeaa07fb5d40d3b285aeb01d1252b1` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:07:04.483216 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5eca28b-cd92-424e-a22b-af14f14da610 None admin] RBAC: Authorizing `identity:get_project(project_id=83aeaa07fb5d40d3b285aeb01d1252b1)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:07:04.484763 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5eca28b-cd92-424e-a22b-af14f14da610 None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:07:04.485804 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:07:04.487454 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 45/90] 10.222.0.18 () {56 vars in 1186 bytes} [Tue Sep 13 03:07:04 2022] GET /identity/v3/projects/83aeaa07fb5d40d3b285aeb01d1252b1 => generated 298 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 03:07:04.488421 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5eca28b-cd92-424e-a22b-af14f14da610 None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:07:04.544895 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02e8dad4-f498-47bf-ac14-2c23c589a668 None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:07:04.583147 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59bad048-6595-4620-80bc-7b7314f0c782 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:07:04.583996 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59bad048-6595-4620-80bc-7b7314f0c782 None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:07:04.584478 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59bad048-6595-4620-80bc-7b7314f0c782 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:07:04.584917 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59bad048-6595-4620-80bc-7b7314f0c782 None admin] PATH_INFO: `/v3/projects/83aeaa07fb5d40d3b285aeb01d1252b1` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:07:04.586203 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59bad048-6595-4620-80bc-7b7314f0c782 None admin] RBAC: Authorizing `identity:get_project(project_id=83aeaa07fb5d40d3b285aeb01d1252b1)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:07:04.587842 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59bad048-6595-4620-80bc-7b7314f0c782 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:07:04.589954 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 46/91] 10.222.0.18 () {56 vars in 1186 bytes} [Tue Sep 13 03:07:04 2022] GET /identity/v3/projects/83aeaa07fb5d40d3b285aeb01d1252b1 => generated 298 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 03:07:04.590925 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59bad048-6595-4620-80bc-7b7314f0c782 None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:07:04.611436 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5eca28b-cd92-424e-a22b-af14f14da610 None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:07:04.637809 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:07:04.651396 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6db158af-91df-46a6-84ab-cd41001c82e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:07:04.652140 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6db158af-91df-46a6-84ab-cd41001c82e6 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:07:04.652473 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6db158af-91df-46a6-84ab-cd41001c82e6 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:07:04.652825 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6db158af-91df-46a6-84ab-cd41001c82e6 None admin] PATH_INFO: `/v3/projects/83aeaa07fb5d40d3b285aeb01d1252b1` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:07:04.654094 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6db158af-91df-46a6-84ab-cd41001c82e6 None admin] RBAC: Authorizing `identity:get_project(project_id=83aeaa07fb5d40d3b285aeb01d1252b1)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:07:04.655794 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6db158af-91df-46a6-84ab-cd41001c82e6 None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:07:04.658685 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 46/92] 10.222.0.18 () {56 vars in 1186 bytes} [Tue Sep 13 03:07:04 2022] GET /identity/v3/projects/83aeaa07fb5d40d3b285aeb01d1252b1 => generated 298 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 03:07:04.659630 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6db158af-91df-46a6-84ab-cd41001c82e6 None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:07:04.699159 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59bad048-6595-4620-80bc-7b7314f0c782 None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:07:04.714104 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:07:04.735250 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-922b1c93-13f6-437b-a044-d9add2f4a8ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:07:04.736550 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-922b1c93-13f6-437b-a044-d9add2f4a8ab None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:07:04.736932 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-922b1c93-13f6-437b-a044-d9add2f4a8ab None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:07:04.737297 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-922b1c93-13f6-437b-a044-d9add2f4a8ab None admin] PATH_INFO: `/v3/projects/d730110113be4b61aed1777ca25db340` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:07:04.738522 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-922b1c93-13f6-437b-a044-d9add2f4a8ab None admin] RBAC: Authorizing `identity:get_project(project_id=d730110113be4b61aed1777ca25db340)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:07:04.740088 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-922b1c93-13f6-437b-a044-d9add2f4a8ab None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:07:04.742519 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 47/93] 10.222.0.18 () {56 vars in 1186 bytes} [Tue Sep 13 03:07:04 2022] GET /identity/v3/projects/d730110113be4b61aed1777ca25db340 => generated 312 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 03:07:04.742656 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-922b1c93-13f6-437b-a044-d9add2f4a8ab None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:07:04.775515 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6db158af-91df-46a6-84ab-cd41001c82e6 None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:07:04.814432 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdb3f2d2-48a9-4af2-9922-a5d220429a30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:07:04.815436 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdb3f2d2-48a9-4af2-9922-a5d220429a30 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:07:04.815911 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdb3f2d2-48a9-4af2-9922-a5d220429a30 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:07:04.816353 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdb3f2d2-48a9-4af2-9922-a5d220429a30 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:07:04.817227 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdb3f2d2-48a9-4af2-9922-a5d220429a30 None admin] RBAC: Adding query filter params (role.id=a9aa35ebda4d4aff80e62f750e9104ab, scope.project.id=83aeaa07fb5d40d3b285aeb01d1252b1, user.id=06632bd878b1474f8d5432f99e899155) {{(pid=81400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 03:07:04.817680 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdb3f2d2-48a9-4af2-9922-a5d220429a30 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:07:04.819575 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdb3f2d2-48a9-4af2-9922-a5d220429a30 None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:07:04.825630 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 47/94] 10.222.0.18 () {56 vars in 1374 bytes} [Tue Sep 13 03:07:04 2022] GET /identity/v3/role_assignments?user.id=06632bd878b1474f8d5432f99e899155&scope.project.id=83aeaa07fb5d40d3b285aeb01d1252b1&role.id=a9aa35ebda4d4aff80e62f750e9104ab => generated 259 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 03:07:04.826809 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdb3f2d2-48a9-4af2-9922-a5d220429a30 None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:07:04.852366 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-922b1c93-13f6-437b-a044-d9add2f4a8ab None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:07:04.889070 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30ab07f7-ae1c-46ad-98d3-18af7099f665 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:07:04.889846 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ab07f7-ae1c-46ad-98d3-18af7099f665 None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:07:04.890223 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ab07f7-ae1c-46ad-98d3-18af7099f665 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:07:04.890637 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ab07f7-ae1c-46ad-98d3-18af7099f665 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:07:04.892006 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30ab07f7-ae1c-46ad-98d3-18af7099f665 None admin] RBAC: Adding query filter params (role.id=425d4b481c7846e0bc07e7e805d5b15f, scope.project.id=83aeaa07fb5d40d3b285aeb01d1252b1, user.id=49f280ab086b4610855c4ffa378517d9) {{(pid=81399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 03:07:04.892006 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30ab07f7-ae1c-46ad-98d3-18af7099f665 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:07:04.893493 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30ab07f7-ae1c-46ad-98d3-18af7099f665 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:07:04.896390 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:07:04.902510 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 48/95] 10.222.0.18 () {56 vars in 1374 bytes} [Tue Sep 13 03:07:04 2022] GET /identity/v3/role_assignments?user.id=49f280ab086b4610855c4ffa378517d9&scope.project.id=83aeaa07fb5d40d3b285aeb01d1252b1&role.id=425d4b481c7846e0bc07e7e805d5b15f => generated 259 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 03:07:04.906784 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30ab07f7-ae1c-46ad-98d3-18af7099f665 None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:07:04.997817 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:07:05.002854 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdb3f2d2-48a9-4af2-9922-a5d220429a30 None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:07:05.048550 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce614a8d-43bf-42a1-970b-f008811c40a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:07:05.049529 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce614a8d-43bf-42a1-970b-f008811c40a7 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:07:05.049956 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce614a8d-43bf-42a1-970b-f008811c40a7 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:07:05.050373 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce614a8d-43bf-42a1-970b-f008811c40a7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:07:05.051663 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce614a8d-43bf-42a1-970b-f008811c40a7 None admin] RBAC: Adding query filter params (role.id=1eb7a5b5203945c29e6d22d653f47847, scope.project.id=83aeaa07fb5d40d3b285aeb01d1252b1, user.id=06632bd878b1474f8d5432f99e899155) {{(pid=81400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 03:07:05.052118 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce614a8d-43bf-42a1-970b-f008811c40a7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:07:05.053892 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce614a8d-43bf-42a1-970b-f008811c40a7 None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:07:05.061067 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 48/96] 10.222.0.18 () {56 vars in 1374 bytes} [Tue Sep 13 03:07:04 2022] GET /identity/v3/role_assignments?user.id=06632bd878b1474f8d5432f99e899155&scope.project.id=83aeaa07fb5d40d3b285aeb01d1252b1&role.id=1eb7a5b5203945c29e6d22d653f47847 => generated 259 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 03:07:05.066652 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30ab07f7-ae1c-46ad-98d3-18af7099f665 None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:07:05.111022 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-970bfd1b-d316-4860-be6f-9ebac0f35e9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:07:05.111643 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-970bfd1b-d316-4860-be6f-9ebac0f35e9f None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:07:05.111927 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-970bfd1b-d316-4860-be6f-9ebac0f35e9f None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:07:05.112201 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-970bfd1b-d316-4860-be6f-9ebac0f35e9f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:07:05.112942 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-970bfd1b-d316-4860-be6f-9ebac0f35e9f None admin] RBAC: Adding query filter params (role.id=1eb7a5b5203945c29e6d22d653f47847, scope.project.id=d730110113be4b61aed1777ca25db340, user.id=06632bd878b1474f8d5432f99e899155) {{(pid=81399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 03:07:05.113250 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-970bfd1b-d316-4860-be6f-9ebac0f35e9f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:07:05.114751 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-970bfd1b-d316-4860-be6f-9ebac0f35e9f None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:07:05.121343 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 49/97] 10.222.0.18 () {56 vars in 1374 bytes} [Tue Sep 13 03:07:04 2022] GET /identity/v3/role_assignments?user.id=06632bd878b1474f8d5432f99e899155&scope.project.id=d730110113be4b61aed1777ca25db340&role.id=1eb7a5b5203945c29e6d22d653f47847 => generated 259 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 03:07:06.460073 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-519deb21-503b-4312-99f3-49b4ef89a8b2 None None] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:07:06.460612 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-519deb21-503b-4312-99f3-49b4ef89a8b2 None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:07:06.460612 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-519deb21-503b-4312-99f3-49b4ef89a8b2 None None] PATH_INFO: `/` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:07:06.462141 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 49/98] 10.222.0.18 () {54 vars in 927 bytes} [Tue Sep 13 03:07:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 03:07:06.467301 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfa9c774-4999-461e-a837-ae0a54250e00 None None] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:07:06.468111 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfa9c774-4999-461e-a837-ae0a54250e00 None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:07:06.468577 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfa9c774-4999-461e-a837-ae0a54250e00 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:07:06.490541 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.auth.core [None req-bfa9c774-4999-461e-a837-ae0a54250e00 None None] MFA Rules not processed for user `49f280ab086b4610855c4ffa378517d9`. Rule list: `[]` (Enabled: `True`). {{(pid=81399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 03:07:06.492141 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-bfa9c774-4999-461e-a837-ae0a54250e00 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:07:06.585396 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 50/99] 10.222.0.18 () {58 vars in 1024 bytes} [Tue Sep 13 03:07:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 13 03:07:06.659429 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a139ef-2b23-4bf2-a222-610e8914c375 None None] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:07:06.660111 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a139ef-2b23-4bf2-a222-610e8914c375 None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:07:06.660578 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a139ef-2b23-4bf2-a222-610e8914c375 None None] PATH_INFO: `/` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:07:06.662231 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 50/100] 10.222.0.18 () {54 vars in 927 bytes} [Tue Sep 13 03:07:06 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 13 03:07:06.680270 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-194f5566-26c1-4a5e-8558-4c1c333f46fb None None] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:07:06.680828 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-194f5566-26c1-4a5e-8558-4c1c333f46fb None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:07:06.681542 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-194f5566-26c1-4a5e-8558-4c1c333f46fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:07:06.700363 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.auth.core [None req-194f5566-26c1-4a5e-8558-4c1c333f46fb None None] MFA Rules not processed for user `49f280ab086b4610855c4ffa378517d9`. Rule list: `[]` (Enabled: `True`). {{(pid=81399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 03:07:06.701866 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-194f5566-26c1-4a5e-8558-4c1c333f46fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:07:06.749755 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:07:06.789683 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 51/101] 10.222.0.18 () {58 vars in 1024 bytes} [Tue Sep 13 03:07:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 13 03:07:07.000774 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32a139ef-2b23-4bf2-a222-610e8914c375 None None] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:07:07.131338 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32a139ef-2b23-4bf2-a222-610e8914c375 None None] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:07:07.156770 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:07:07.162076 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-194f5566-26c1-4a5e-8558-4c1c333f46fb None None] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:07:07.201559 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81660624-991b-4a0a-8f8f-0808e5065466 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:07:07.202246 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81660624-991b-4a0a-8f8f-0808e5065466 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:07:07.202527 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81660624-991b-4a0a-8f8f-0808e5065466 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:07:07.202825 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81660624-991b-4a0a-8f8f-0808e5065466 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:07:07.204669 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81660624-991b-4a0a-8f8f-0808e5065466 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:07:07.206168 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81660624-991b-4a0a-8f8f-0808e5065466 None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:07:07.211306 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 51/102] 10.222.0.18 () {56 vars in 1108 bytes} [Tue Sep 13 03:07:06 2022] GET /identity/v3/domains/default => generated 199 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 03:07:07.219247 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81660624-991b-4a0a-8f8f-0808e5065466 None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:07:07.362527 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-194f5566-26c1-4a5e-8558-4c1c333f46fb None None] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:07:07.390331 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81660624-991b-4a0a-8f8f-0808e5065466 None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:07:07.448703 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53faeb69-8a5c-46e1-96c5-46c0acc68efa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:07:07.448896 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a62ccfb9-e355-4b3b-84cc-1313b72397e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:07:07.449350 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a62ccfb9-e355-4b3b-84cc-1313b72397e7 None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:07:07.449638 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a62ccfb9-e355-4b3b-84cc-1313b72397e7 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:07:07.449912 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a62ccfb9-e355-4b3b-84cc-1313b72397e7 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:07:07.450417 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53faeb69-8a5c-46e1-96c5-46c0acc68efa None admin] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:07:07.450918 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53faeb69-8a5c-46e1-96c5-46c0acc68efa None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:07:07.451377 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53faeb69-8a5c-46e1-96c5-46c0acc68efa None admin] PATH_INFO: `/v3/projects` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:07:07.452451 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53faeb69-8a5c-46e1-96c5-46c0acc68efa None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:07:07.454089 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53faeb69-8a5c-46e1-96c5-46c0acc68efa None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:07:07.476541 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a62ccfb9-e355-4b3b-84cc-1313b72397e7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 03:07:07.483058 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application [None req-a62ccfb9-e355-4b3b-84cc-1313b72397e7 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 13 03:07:07.483058 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 03:07:07.483058 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 03:07:07.483058 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 03:07:07.483058 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 03:07:07.483058 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 03:07:07.483058 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 03:07:07.483058 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 03:07:07.483058 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 03:07:07.483058 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 03:07:07.483058 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 03:07:07.483058 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 03:07:07.483058 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 03:07:07.483058 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 03:07:07.483058 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 03:07:07.483058 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 03:07:07.484294 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 03:07:07.484294 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 03:07:07.484294 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 03:07:07.484294 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 03:07:07.484294 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 03:07:07.484294 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 03:07:07.484294 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 03:07:07.484294 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 03:07:07.484294 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 03:07:07.484294 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 03:07:07.484294 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 03:07:07.484294 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application with Lock( Sep 13 03:07:07.484294 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 03:07:07.484294 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._enter() Sep 13 03:07:07.484294 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 03:07:07.484294 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 03:07:07.485061 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 03:07:07.485061 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.creator() Sep 13 03:07:07.485061 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 03:07:07.485061 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application created_value = creator( Sep 13 03:07:07.485061 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 03:07:07.485061 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 03:07:07.485061 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 03:07:07.485061 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 03:07:07.485061 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 03:07:07.485061 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 03:07:07.485061 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 13 03:07:07.485061 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Sep 13 03:07:07.496287 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 52/104] 10.222.0.18 () {60 vars in 1141 bytes} [Tue Sep 13 03:07:07 2022] POST /identity/v3/projects => generated 302 bytes in 277 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 03:07:07.496917 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 52/104] 10.222.0.18 () {56 vars in 1099 bytes} [Tue Sep 13 03:07:07 2022] GET /identity/v3/roles/reader => generated 84 bytes in 333 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 13 03:07:07.503956 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53faeb69-8a5c-46e1-96c5-46c0acc68efa None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:07:07.562002 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:07:07.648014 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53faeb69-8a5c-46e1-96c5-46c0acc68efa None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:07:07.701643 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4bb5ed6-7165-4c65-ad2c-fdbe50d197a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:07:07.702704 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4bb5ed6-7165-4c65-ad2c-fdbe50d197a1 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:07:07.703188 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4bb5ed6-7165-4c65-ad2c-fdbe50d197a1 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:07:07.703724 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4bb5ed6-7165-4c65-ad2c-fdbe50d197a1 None admin] PATH_INFO: `/v3/roles` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:07:07.705336 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4bb5ed6-7165-4c65-ad2c-fdbe50d197a1 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 03:07:07.706020 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4bb5ed6-7165-4c65-ad2c-fdbe50d197a1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:07:07.707678 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4bb5ed6-7165-4c65-ad2c-fdbe50d197a1 None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:07:07.719344 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 53/105] 10.222.0.18 () {56 vars in 1101 bytes} [Tue Sep 13 03:07:07 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 03:07:07.725701 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a62ccfb9-e355-4b3b-84cc-1313b72397e7 None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:07:07.782461 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:07:07.863925 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb98d717-63ea-4401-8a7e-7de28904f22e None None] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:07:07.864216 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb98d717-63ea-4401-8a7e-7de28904f22e None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:07:07.864531 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb98d717-63ea-4401-8a7e-7de28904f22e None None] PATH_INFO: `/` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:07:07.865973 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 54/106] 10.222.0.18 () {54 vars in 927 bytes} [Tue Sep 13 03:07:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 03:07:07.873094 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85a20a3c-88c6-41cb-9682-7af0fde9d462 None None] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:07:07.873382 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85a20a3c-88c6-41cb-9682-7af0fde9d462 None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:07:07.873660 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85a20a3c-88c6-41cb-9682-7af0fde9d462 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:07:07.880084 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a62ccfb9-e355-4b3b-84cc-1313b72397e7 None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:07:07.903559 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.auth.core [None req-85a20a3c-88c6-41cb-9682-7af0fde9d462 None None] MFA Rules not processed for user `49f280ab086b4610855c4ffa378517d9`. Rule list: `[]` (Enabled: `True`). {{(pid=81400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 03:07:07.904991 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-85a20a3c-88c6-41cb-9682-7af0fde9d462 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:07:07.921443 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f44257c8-dd4e-4c22-aa1b-7b9cf9e52a3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:07:07.922443 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f44257c8-dd4e-4c22-aa1b-7b9cf9e52a3e None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:07:07.923012 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f44257c8-dd4e-4c22-aa1b-7b9cf9e52a3e None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:07:07.923532 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f44257c8-dd4e-4c22-aa1b-7b9cf9e52a3e None admin] PATH_INFO: `/v3/users/cfb765821b764d438d4fcad2b5d6007e` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:07:07.940889 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f44257c8-dd4e-4c22-aa1b-7b9cf9e52a3e None admin] RBAC: Authorizing `identity:get_user(user_id=cfb765821b764d438d4fcad2b5d6007e)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:07:07.943023 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f44257c8-dd4e-4c22-aa1b-7b9cf9e52a3e None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:07:07.945348 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 53/107] 10.222.0.18 () {56 vars in 1177 bytes} [Tue Sep 13 03:07:07 2022] GET /identity/v3/users/cfb765821b764d438d4fcad2b5d6007e => generated 289 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 03:07:07.951118 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f44257c8-dd4e-4c22-aa1b-7b9cf9e52a3e None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:07:08.013860 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 55/108] 10.222.0.18 () {58 vars in 1024 bytes} [Tue Sep 13 03:07:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 13 03:07:08.043988 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c4d8bc2-e390-4267-89f4-31ebbe1ba0a0 None None] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:07:08.044297 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c4d8bc2-e390-4267-89f4-31ebbe1ba0a0 None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:07:08.044946 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c4d8bc2-e390-4267-89f4-31ebbe1ba0a0 None None] PATH_INFO: `/` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:07:08.046048 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 56/109] 10.222.0.18 () {54 vars in 927 bytes} [Tue Sep 13 03:07:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 03:07:08.051800 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c96a6917-cf9a-42d6-8712-cbda26e562bb None None] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:07:08.052087 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c96a6917-cf9a-42d6-8712-cbda26e562bb None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:07:08.052365 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c96a6917-cf9a-42d6-8712-cbda26e562bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:07:08.061057 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:07:08.076271 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f44257c8-dd4e-4c22-aa1b-7b9cf9e52a3e None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:07:08.089080 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.auth.core [None req-c96a6917-cf9a-42d6-8712-cbda26e562bb None None] MFA Rules not processed for user `49f280ab086b4610855c4ffa378517d9`. Rule list: `[]` (Enabled: `True`). {{(pid=81400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 03:07:08.090275 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-c96a6917-cf9a-42d6-8712-cbda26e562bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:07:08.114744 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b14a10e-a81a-4b84-a45d-c04bce10aae5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:07:08.115805 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b14a10e-a81a-4b84-a45d-c04bce10aae5 None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:07:08.116303 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b14a10e-a81a-4b84-a45d-c04bce10aae5 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:07:08.116817 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b14a10e-a81a-4b84-a45d-c04bce10aae5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:07:08.136863 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:07:08.197216 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b14a10e-a81a-4b84-a45d-c04bce10aae5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:07:08.199749 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b14a10e-a81a-4b84-a45d-c04bce10aae5 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:07:08.204292 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 57/110] 10.222.0.18 () {58 vars in 1024 bytes} [Tue Sep 13 03:07:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 13 03:07:08.290984 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:07:08.305433 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 54/111] 10.222.0.18 () {58 vars in 1282 bytes} [Tue Sep 13 03:07:07 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 354 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 13 03:07:08.311749 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c96a6917-cf9a-42d6-8712-cbda26e562bb None None] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:07:08.445838 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c96a6917-cf9a-42d6-8712-cbda26e562bb None None] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:07:08.452339 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a70dca03-4d92-43c6-ade1-71abf17f205a None None] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:07:08.452560 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a70dca03-4d92-43c6-ade1-71abf17f205a None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:07:08.454867 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a70dca03-4d92-43c6-ade1-71abf17f205a None None] PATH_INFO: `/` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:07:08.454867 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 55/112] 10.222.0.18 () {54 vars in 927 bytes} [Tue Sep 13 03:07:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 13 03:07:08.460353 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-358c5bfa-ecce-4d69-8b54-c1ff0d6bc420 None None] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:07:08.460873 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-358c5bfa-ecce-4d69-8b54-c1ff0d6bc420 None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:07:08.461345 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-358c5bfa-ecce-4d69-8b54-c1ff0d6bc420 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:07:08.484893 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.auth.core [None req-358c5bfa-ecce-4d69-8b54-c1ff0d6bc420 None None] MFA Rules not processed for user `49f280ab086b4610855c4ffa378517d9`. Rule list: `[]` (Enabled: `True`). {{(pid=81399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 03:07:08.487695 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-358c5bfa-ecce-4d69-8b54-c1ff0d6bc420 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:07:08.491669 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5b55129-98f3-4a4e-8a06-1ad504291bbd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:07:08.492199 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b55129-98f3-4a4e-8a06-1ad504291bbd None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:07:08.492487 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b55129-98f3-4a4e-8a06-1ad504291bbd None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:07:08.492762 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b55129-98f3-4a4e-8a06-1ad504291bbd None admin] PATH_INFO: `/v3/projects/83aeaa07fb5d40d3b285aeb01d1252b1` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:07:08.493908 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5b55129-98f3-4a4e-8a06-1ad504291bbd None admin] RBAC: Authorizing `identity:get_project(project_id=83aeaa07fb5d40d3b285aeb01d1252b1)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:07:08.495415 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5b55129-98f3-4a4e-8a06-1ad504291bbd None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:07:08.497064 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 58/113] 10.222.0.18 () {56 vars in 1186 bytes} [Tue Sep 13 03:07:08 2022] GET /identity/v3/projects/83aeaa07fb5d40d3b285aeb01d1252b1 => generated 298 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 03:07:08.497836 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5b55129-98f3-4a4e-8a06-1ad504291bbd None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:07:08.544452 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:07:08.572566 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 56/114] 10.222.0.18 () {58 vars in 1024 bytes} [Tue Sep 13 03:07:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 13 03:07:08.578430 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-358c5bfa-ecce-4d69-8b54-c1ff0d6bc420 None None] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:07:08.620424 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5b55129-98f3-4a4e-8a06-1ad504291bbd None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:07:08.656452 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-698ceead-e1bd-4040-82e0-03c03fe71599 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:07:08.657092 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-698ceead-e1bd-4040-82e0-03c03fe71599 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:07:08.657373 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-698ceead-e1bd-4040-82e0-03c03fe71599 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:07:08.658175 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-698ceead-e1bd-4040-82e0-03c03fe71599 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:07:08.669249 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-698ceead-e1bd-4040-82e0-03c03fe71599 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 03:07:08.675058 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application [None req-698ceead-e1bd-4040-82e0-03c03fe71599 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 13 03:07:08.675058 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 03:07:08.675058 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 03:07:08.675058 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 03:07:08.675058 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 03:07:08.675058 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 03:07:08.675058 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 03:07:08.675058 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 03:07:08.675058 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 03:07:08.675058 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 03:07:08.675058 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 03:07:08.675058 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 03:07:08.675058 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 03:07:08.675058 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 03:07:08.675058 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 03:07:08.675058 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 03:07:08.675941 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 03:07:08.675941 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 03:07:08.675941 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 03:07:08.675941 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 03:07:08.675941 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 03:07:08.675941 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 03:07:08.675941 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 03:07:08.675941 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 03:07:08.675941 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 03:07:08.675941 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 03:07:08.675941 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 03:07:08.675941 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application with Lock( Sep 13 03:07:08.675941 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 03:07:08.675941 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._enter() Sep 13 03:07:08.675941 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 03:07:08.675941 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 03:07:08.676723 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 03:07:08.676723 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.creator() Sep 13 03:07:08.676723 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 03:07:08.676723 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application created_value = creator( Sep 13 03:07:08.676723 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 03:07:08.676723 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 03:07:08.676723 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 03:07:08.676723 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 03:07:08.676723 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 03:07:08.676723 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 03:07:08.676723 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 13 03:07:08.676723 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Sep 13 03:07:08.676723 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 59/115] 10.222.0.18 () {56 vars in 1114 bytes} [Tue Sep 13 03:07:08 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 03:07:08.678088 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff2e711a-b23c-4ed4-8c5d-4c5ac1d0c474 None None] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:07:08.678088 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff2e711a-b23c-4ed4-8c5d-4c5ac1d0c474 None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:07:08.678344 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff2e711a-b23c-4ed4-8c5d-4c5ac1d0c474 None None] PATH_INFO: `/` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:07:08.679877 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 60/116] 10.222.0.18 () {54 vars in 927 bytes} [Tue Sep 13 03:07:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 03:07:08.680444 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff2e711a-b23c-4ed4-8c5d-4c5ac1d0c474 None None] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:07:08.701356 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-358c5bfa-ecce-4d69-8b54-c1ff0d6bc420 None None] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:07:08.722551 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:07:08.741985 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a075b8b-f3a4-4380-8096-af6323fd6239 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:07:08.742946 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a075b8b-f3a4-4380-8096-af6323fd6239 None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:07:08.743474 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a075b8b-f3a4-4380-8096-af6323fd6239 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:07:08.743938 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a075b8b-f3a4-4380-8096-af6323fd6239 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:07:08.745011 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a075b8b-f3a4-4380-8096-af6323fd6239 None admin] RBAC: Adding query filter params (role.id=502e6c11e747432988d7f723cee0af64, scope.project.id=83aeaa07fb5d40d3b285aeb01d1252b1, user.id=cfb765821b764d438d4fcad2b5d6007e) {{(pid=81399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 03:07:08.745625 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a075b8b-f3a4-4380-8096-af6323fd6239 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:07:08.747486 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a075b8b-f3a4-4380-8096-af6323fd6239 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:07:08.753595 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 57/117] 10.222.0.18 () {56 vars in 1374 bytes} [Tue Sep 13 03:07:08 2022] GET /identity/v3/role_assignments?user.id=cfb765821b764d438d4fcad2b5d6007e&scope.project.id=83aeaa07fb5d40d3b285aeb01d1252b1&role.id=502e6c11e747432988d7f723cee0af64 => generated 259 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 03:07:08.754822 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a075b8b-f3a4-4380-8096-af6323fd6239 None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:07:08.784223 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:07:08.797823 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff2e711a-b23c-4ed4-8c5d-4c5ac1d0c474 None None] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:07:08.837007 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fd5336e-96ef-4166-a910-24e06db1a500 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:07:08.837655 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd5336e-96ef-4166-a910-24e06db1a500 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:07:08.837942 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd5336e-96ef-4166-a910-24e06db1a500 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:07:08.838222 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd5336e-96ef-4166-a910-24e06db1a500 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:07:08.859390 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fd5336e-96ef-4166-a910-24e06db1a500 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 03:07:08.864758 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application [None req-5fd5336e-96ef-4166-a910-24e06db1a500 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 13 03:07:08.864758 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 03:07:08.864758 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 03:07:08.864758 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 03:07:08.864758 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 03:07:08.864758 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 03:07:08.864758 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 03:07:08.864758 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 03:07:08.864758 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 03:07:08.864758 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 03:07:08.864758 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 03:07:08.864758 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 03:07:08.864758 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 03:07:08.864758 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 03:07:08.864758 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 03:07:08.864758 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 03:07:08.865617 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 03:07:08.865617 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 03:07:08.865617 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 03:07:08.865617 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 03:07:08.865617 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 03:07:08.865617 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 03:07:08.865617 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 03:07:08.865617 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 03:07:08.865617 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 03:07:08.865617 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 03:07:08.865617 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 03:07:08.865617 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application with Lock( Sep 13 03:07:08.865617 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 03:07:08.865617 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._enter() Sep 13 03:07:08.865617 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 03:07:08.865617 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 03:07:08.866457 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 03:07:08.866457 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.creator() Sep 13 03:07:08.866457 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 03:07:08.866457 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application created_value = creator( Sep 13 03:07:08.866457 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 03:07:08.866457 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 03:07:08.866457 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 03:07:08.866457 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 03:07:08.866457 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 03:07:08.866457 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 03:07:08.866457 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 13 03:07:08.866457 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Sep 13 03:07:08.866457 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 61/118] 10.222.0.18 () {56 vars in 1096 bytes} [Tue Sep 13 03:07:08 2022] GET /identity/v3/roles/admin => generated 83 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 03:07:08.875493 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a255fe1-80a9-436d-bf62-5eab21acef43 None None] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:07:08.875916 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a255fe1-80a9-436d-bf62-5eab21acef43 None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:07:08.876080 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a255fe1-80a9-436d-bf62-5eab21acef43 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:07:08.878280 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a075b8b-f3a4-4380-8096-af6323fd6239 None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:07:08.911052 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.auth.core [None req-8a255fe1-80a9-436d-bf62-5eab21acef43 None None] MFA Rules not processed for user `49f280ab086b4610855c4ffa378517d9`. Rule list: `[]` (Enabled: `True`). {{(pid=81400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 03:07:08.912409 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-8a255fe1-80a9-436d-bf62-5eab21acef43 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:07:08.924019 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef837c34-1f73-4979-abad-58953828cdee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:07:08.924993 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef837c34-1f73-4979-abad-58953828cdee None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:07:08.925535 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef837c34-1f73-4979-abad-58953828cdee None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:07:08.925998 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef837c34-1f73-4979-abad-58953828cdee None admin] PATH_INFO: `/v3/roles` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:07:08.927581 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef837c34-1f73-4979-abad-58953828cdee None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 03:07:08.928862 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef837c34-1f73-4979-abad-58953828cdee None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:07:08.930308 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef837c34-1f73-4979-abad-58953828cdee None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:07:08.944209 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 58/119] 10.222.0.18 () {56 vars in 1111 bytes} [Tue Sep 13 03:07:08 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 03:07:08.945048 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef837c34-1f73-4979-abad-58953828cdee None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:07:09.000303 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:07:09.021336 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 62/120] 10.222.0.18 () {58 vars in 1024 bytes} [Tue Sep 13 03:07:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 13 03:07:09.022322 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a255fe1-80a9-436d-bf62-5eab21acef43 None None] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:07:09.075028 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef837c34-1f73-4979-abad-58953828cdee None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:07:09.112593 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecb0f99f-1ed2-4b82-a142-d0f0b38059fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:07:09.113363 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecb0f99f-1ed2-4b82-a142-d0f0b38059fb None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:07:09.114232 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecb0f99f-1ed2-4b82-a142-d0f0b38059fb None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:07:09.114582 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecb0f99f-1ed2-4b82-a142-d0f0b38059fb None admin] PATH_INFO: `/v3/roles` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:07:09.115611 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecb0f99f-1ed2-4b82-a142-d0f0b38059fb None admin] RBAC: Adding query filter params (name=admin) {{(pid=81399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 03:07:09.115989 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecb0f99f-1ed2-4b82-a142-d0f0b38059fb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:07:09.117425 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecb0f99f-1ed2-4b82-a142-d0f0b38059fb None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:07:09.117855 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:07:09.127257 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 59/121] 10.222.0.18 () {56 vars in 1099 bytes} [Tue Sep 13 03:07:08 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 03:07:09.128141 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecb0f99f-1ed2-4b82-a142-d0f0b38059fb None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:07:09.136648 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a255fe1-80a9-436d-bf62-5eab21acef43 None None] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:07:09.175400 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-917be6b0-2638-4284-8693-19541f18a0f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:07:09.176144 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-917be6b0-2638-4284-8693-19541f18a0f4 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:07:09.176489 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-917be6b0-2638-4284-8693-19541f18a0f4 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:07:09.176827 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-917be6b0-2638-4284-8693-19541f18a0f4 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:07:09.191643 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-917be6b0-2638-4284-8693-19541f18a0f4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 03:07:09.199311 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application [None req-917be6b0-2638-4284-8693-19541f18a0f4 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 13 03:07:09.199311 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 03:07:09.199311 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 03:07:09.199311 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 03:07:09.199311 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 03:07:09.199311 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 03:07:09.199311 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 03:07:09.199311 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 03:07:09.199311 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 03:07:09.199311 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 03:07:09.199311 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 03:07:09.199311 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 03:07:09.199311 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 03:07:09.199311 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 03:07:09.199311 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 03:07:09.199311 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 03:07:09.200320 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 03:07:09.200320 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 03:07:09.200320 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 03:07:09.200320 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 03:07:09.200320 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 03:07:09.200320 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 03:07:09.200320 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 03:07:09.200320 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 03:07:09.200320 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 03:07:09.200320 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 03:07:09.200320 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 03:07:09.200320 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application with Lock( Sep 13 03:07:09.200320 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 03:07:09.200320 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._enter() Sep 13 03:07:09.200320 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 03:07:09.200320 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 03:07:09.201273 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 03:07:09.201273 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.creator() Sep 13 03:07:09.201273 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 03:07:09.201273 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application created_value = creator( Sep 13 03:07:09.201273 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 03:07:09.201273 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 03:07:09.201273 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 03:07:09.201273 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 03:07:09.201273 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 03:07:09.201273 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 03:07:09.201273 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 13 03:07:09.201273 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Sep 13 03:07:09.202729 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 63/122] 10.222.0.18 () {56 vars in 1099 bytes} [Tue Sep 13 03:07:09 2022] GET /identity/v3/roles/member => generated 84 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 03:07:09.204177 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-917be6b0-2638-4284-8693-19541f18a0f4 None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:07:09.283569 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecb0f99f-1ed2-4b82-a142-d0f0b38059fb None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:07:09.296534 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:07:09.354193 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57554ffd-c7a4-436e-9da1-782894035a96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:07:09.354892 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57554ffd-c7a4-436e-9da1-782894035a96 None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:07:09.355175 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57554ffd-c7a4-436e-9da1-782894035a96 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:07:09.355449 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57554ffd-c7a4-436e-9da1-782894035a96 None admin] PATH_INFO: `/v3/users/06632bd878b1474f8d5432f99e899155` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:07:09.356790 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57554ffd-c7a4-436e-9da1-782894035a96 None admin] RBAC: Authorizing `identity:get_user(user_id=06632bd878b1474f8d5432f99e899155)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:07:09.358229 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57554ffd-c7a4-436e-9da1-782894035a96 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:07:09.361639 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 60/123] 10.222.0.18 () {56 vars in 1177 bytes} [Tue Sep 13 03:07:09 2022] GET /identity/v3/users/06632bd878b1474f8d5432f99e899155 => generated 275 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 03:07:09.362402 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57554ffd-c7a4-436e-9da1-782894035a96 None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:07:09.391758 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-917be6b0-2638-4284-8693-19541f18a0f4 None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:07:09.431473 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be69bd54-f3ba-4218-8b40-b56e6d9897c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:07:09.432573 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be69bd54-f3ba-4218-8b40-b56e6d9897c3 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:07:09.433025 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be69bd54-f3ba-4218-8b40-b56e6d9897c3 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:07:09.433477 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be69bd54-f3ba-4218-8b40-b56e6d9897c3 None admin] PATH_INFO: `/v3/users/49f280ab086b4610855c4ffa378517d9` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:07:09.434940 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be69bd54-f3ba-4218-8b40-b56e6d9897c3 None admin] RBAC: Authorizing `identity:get_user(user_id=49f280ab086b4610855c4ffa378517d9)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:07:09.436702 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be69bd54-f3ba-4218-8b40-b56e6d9897c3 None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:07:09.438724 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 64/124] 10.222.0.18 () {56 vars in 1177 bytes} [Tue Sep 13 03:07:09 2022] GET /identity/v3/users/49f280ab086b4610855c4ffa378517d9 => generated 247 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 03:07:09.440221 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be69bd54-f3ba-4218-8b40-b56e6d9897c3 None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:07:09.453673 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:07:09.494572 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57554ffd-c7a4-436e-9da1-782894035a96 None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:07:09.532189 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79b6aad5-7f23-454b-a42c-d968416c3860 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:07:09.532838 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b6aad5-7f23-454b-a42c-d968416c3860 None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:07:09.533123 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b6aad5-7f23-454b-a42c-d968416c3860 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:07:09.533398 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b6aad5-7f23-454b-a42c-d968416c3860 None admin] PATH_INFO: `/v3/roles` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:07:09.534114 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79b6aad5-7f23-454b-a42c-d968416c3860 None admin] RBAC: Adding query filter params (name=member) {{(pid=81399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 03:07:09.534366 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79b6aad5-7f23-454b-a42c-d968416c3860 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:07:09.536317 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79b6aad5-7f23-454b-a42c-d968416c3860 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:07:09.544795 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 61/125] 10.222.0.18 () {56 vars in 1101 bytes} [Tue Sep 13 03:07:09 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 03:07:09.547066 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79b6aad5-7f23-454b-a42c-d968416c3860 None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:07:09.563090 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be69bd54-f3ba-4218-8b40-b56e6d9897c3 None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:07:09.588621 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:07:09.604264 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e96bed7d-ba57-4371-8554-c2fd91710843 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:07:09.605320 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e96bed7d-ba57-4371-8554-c2fd91710843 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:07:09.605770 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e96bed7d-ba57-4371-8554-c2fd91710843 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:07:09.606209 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e96bed7d-ba57-4371-8554-c2fd91710843 None admin] PATH_INFO: `/v3/projects/83aeaa07fb5d40d3b285aeb01d1252b1` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:07:09.607623 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e96bed7d-ba57-4371-8554-c2fd91710843 None admin] RBAC: Authorizing `identity:get_project(project_id=83aeaa07fb5d40d3b285aeb01d1252b1)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:07:09.609395 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e96bed7d-ba57-4371-8554-c2fd91710843 None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:07:09.612265 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 65/126] 10.222.0.18 () {56 vars in 1186 bytes} [Tue Sep 13 03:07:09 2022] GET /identity/v3/projects/83aeaa07fb5d40d3b285aeb01d1252b1 => generated 298 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 03:07:09.613337 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e96bed7d-ba57-4371-8554-c2fd91710843 None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:07:09.658103 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79b6aad5-7f23-454b-a42c-d968416c3860 None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:07:09.696496 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ffb93ce-3fab-4b63-904b-8fa31c252c43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:07:09.697532 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ffb93ce-3fab-4b63-904b-8fa31c252c43 None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:07:09.697835 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ffb93ce-3fab-4b63-904b-8fa31c252c43 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:07:09.698112 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ffb93ce-3fab-4b63-904b-8fa31c252c43 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:07:09.700298 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:07:09.709686 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ffb93ce-3fab-4b63-904b-8fa31c252c43 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 03:07:09.716334 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application [None req-0ffb93ce-3fab-4b63-904b-8fa31c252c43 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 13 03:07:09.716334 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 03:07:09.716334 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 03:07:09.716334 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 03:07:09.716334 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 03:07:09.716334 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 03:07:09.716334 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 03:07:09.716334 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 03:07:09.716334 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 03:07:09.716334 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 03:07:09.716334 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 03:07:09.716334 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 03:07:09.716334 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 03:07:09.716334 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 03:07:09.716334 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 03:07:09.716334 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 03:07:09.717211 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 03:07:09.717211 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 03:07:09.717211 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 03:07:09.717211 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 03:07:09.717211 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 03:07:09.717211 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 03:07:09.717211 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 03:07:09.717211 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 03:07:09.717211 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 03:07:09.717211 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 03:07:09.717211 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 03:07:09.717211 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application with Lock( Sep 13 03:07:09.717211 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 03:07:09.717211 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._enter() Sep 13 03:07:09.717211 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 03:07:09.717211 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 03:07:09.717984 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 03:07:09.717984 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.creator() Sep 13 03:07:09.717984 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 03:07:09.717984 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application created_value = creator( Sep 13 03:07:09.717984 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 03:07:09.717984 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 03:07:09.717984 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 03:07:09.717984 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 03:07:09.717984 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 03:07:09.717984 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 03:07:09.717984 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 13 03:07:09.717984 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Sep 13 03:07:09.717984 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 62/127] 10.222.0.18 () {56 vars in 1099 bytes} [Tue Sep 13 03:07:09 2022] GET /identity/v3/roles/member => generated 84 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 03:07:09.718654 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ffb93ce-3fab-4b63-904b-8fa31c252c43 None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:07:09.730142 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e96bed7d-ba57-4371-8554-c2fd91710843 None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:07:09.768506 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d94ad173-59d3-4958-ace5-7a752226f32a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:07:09.769694 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d94ad173-59d3-4958-ace5-7a752226f32a None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:07:09.770179 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d94ad173-59d3-4958-ace5-7a752226f32a None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:07:09.770728 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d94ad173-59d3-4958-ace5-7a752226f32a None admin] PATH_INFO: `/v3/projects/83aeaa07fb5d40d3b285aeb01d1252b1` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:07:09.772128 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d94ad173-59d3-4958-ace5-7a752226f32a None admin] RBAC: Authorizing `identity:get_project(project_id=83aeaa07fb5d40d3b285aeb01d1252b1)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:07:09.773856 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d94ad173-59d3-4958-ace5-7a752226f32a None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:07:09.776107 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 66/128] 10.222.0.18 () {56 vars in 1186 bytes} [Tue Sep 13 03:07:09 2022] GET /identity/v3/projects/83aeaa07fb5d40d3b285aeb01d1252b1 => generated 298 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 03:07:09.777192 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d94ad173-59d3-4958-ace5-7a752226f32a None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:07:09.834910 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:07:09.841887 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ffb93ce-3fab-4b63-904b-8fa31c252c43 None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:07:09.880058 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-935834ee-eeaf-45b4-84ca-2d3acc409a50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:07:09.880696 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-935834ee-eeaf-45b4-84ca-2d3acc409a50 None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:07:09.880980 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-935834ee-eeaf-45b4-84ca-2d3acc409a50 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:07:09.881257 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-935834ee-eeaf-45b4-84ca-2d3acc409a50 None admin] PATH_INFO: `/v3/users/06632bd878b1474f8d5432f99e899155` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:07:09.882391 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-935834ee-eeaf-45b4-84ca-2d3acc409a50 None admin] RBAC: Authorizing `identity:get_user(user_id=06632bd878b1474f8d5432f99e899155)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:07:09.884142 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-935834ee-eeaf-45b4-84ca-2d3acc409a50 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:07:09.886375 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 63/129] 10.222.0.18 () {56 vars in 1177 bytes} [Tue Sep 13 03:07:09 2022] GET /identity/v3/users/06632bd878b1474f8d5432f99e899155 => generated 275 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 03:07:09.887212 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-935834ee-eeaf-45b4-84ca-2d3acc409a50 None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:07:09.895976 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d94ad173-59d3-4958-ace5-7a752226f32a None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:07:09.934016 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ba1b0e7-c6ac-4f1f-bdec-bb83d48f0e1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:07:09.935172 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba1b0e7-c6ac-4f1f-bdec-bb83d48f0e1c None admin] REQUEST_METHOD: `PUT` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:07:09.935646 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba1b0e7-c6ac-4f1f-bdec-bb83d48f0e1c None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:07:09.936140 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba1b0e7-c6ac-4f1f-bdec-bb83d48f0e1c None admin] PATH_INFO: `/v3/projects/83aeaa07fb5d40d3b285aeb01d1252b1/users/06632bd878b1474f8d5432f99e899155/roles/a9aa35ebda4d4aff80e62f750e9104ab` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:07:09.938846 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ba1b0e7-c6ac-4f1f-bdec-bb83d48f0e1c None admin] RBAC: Authorizing `identity:create_grant(project_id=83aeaa07fb5d40d3b285aeb01d1252b1, user_id=06632bd878b1474f8d5432f99e899155, role_id=a9aa35ebda4d4aff80e62f750e9104ab)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:07:09.940548 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ba1b0e7-c6ac-4f1f-bdec-bb83d48f0e1c None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:07:09.960434 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 67/130] 10.222.0.18 () {58 vars in 1439 bytes} [Tue Sep 13 03:07:09 2022] PUT /identity/v3/projects/83aeaa07fb5d40d3b285aeb01d1252b1/users/06632bd878b1474f8d5432f99e899155/roles/a9aa35ebda4d4aff80e62f750e9104ab => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 03:07:09.961865 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ba1b0e7-c6ac-4f1f-bdec-bb83d48f0e1c None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:07:09.968315 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:07:10.016909 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-935834ee-eeaf-45b4-84ca-2d3acc409a50 None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:07:10.102064 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2db6f5e3-2b31-416e-84c7-3c8b3ee7b9d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:07:10.102631 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2db6f5e3-2b31-416e-84c7-3c8b3ee7b9d7 None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:07:10.102914 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2db6f5e3-2b31-416e-84c7-3c8b3ee7b9d7 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:07:10.103190 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2db6f5e3-2b31-416e-84c7-3c8b3ee7b9d7 None admin] PATH_INFO: `/v3/roles` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:07:10.103873 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2db6f5e3-2b31-416e-84c7-3c8b3ee7b9d7 None admin] RBAC: Adding query filter params (name=member) {{(pid=81399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 03:07:10.104409 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2db6f5e3-2b31-416e-84c7-3c8b3ee7b9d7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:07:10.106723 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2db6f5e3-2b31-416e-84c7-3c8b3ee7b9d7 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:07:10.123615 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 64/131] 10.222.0.18 () {56 vars in 1101 bytes} [Tue Sep 13 03:07:09 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 03:07:10.124486 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2db6f5e3-2b31-416e-84c7-3c8b3ee7b9d7 None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:07:10.163096 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:07:10.192202 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ba1b0e7-c6ac-4f1f-bdec-bb83d48f0e1c None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:07:10.237965 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc4bdd3c-9870-4104-ab51-8059e22e7282 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:07:10.238992 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc4bdd3c-9870-4104-ab51-8059e22e7282 None admin] REQUEST_METHOD: `PUT` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:07:10.239491 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc4bdd3c-9870-4104-ab51-8059e22e7282 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:07:10.239950 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc4bdd3c-9870-4104-ab51-8059e22e7282 None admin] PATH_INFO: `/v3/projects/83aeaa07fb5d40d3b285aeb01d1252b1/users/49f280ab086b4610855c4ffa378517d9/roles/425d4b481c7846e0bc07e7e805d5b15f` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:07:10.242822 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc4bdd3c-9870-4104-ab51-8059e22e7282 None admin] RBAC: Authorizing `identity:create_grant(project_id=83aeaa07fb5d40d3b285aeb01d1252b1, user_id=49f280ab086b4610855c4ffa378517d9, role_id=425d4b481c7846e0bc07e7e805d5b15f)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:07:10.244172 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2db6f5e3-2b31-416e-84c7-3c8b3ee7b9d7 None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:07:10.244742 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc4bdd3c-9870-4104-ab51-8059e22e7282 None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:07:10.262845 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 68/132] 10.222.0.18 () {58 vars in 1439 bytes} [Tue Sep 13 03:07:09 2022] PUT /identity/v3/projects/83aeaa07fb5d40d3b285aeb01d1252b1/users/49f280ab086b4610855c4ffa378517d9/roles/425d4b481c7846e0bc07e7e805d5b15f => generated 0 bytes in 301 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 03:07:10.264202 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc4bdd3c-9870-4104-ab51-8059e22e7282 None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:07:10.287429 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b85dd4f-16f6-45ee-825e-c9171b9716eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:07:10.288101 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b85dd4f-16f6-45ee-825e-c9171b9716eb None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:07:10.288639 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b85dd4f-16f6-45ee-825e-c9171b9716eb None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:07:10.288766 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b85dd4f-16f6-45ee-825e-c9171b9716eb None admin] PATH_INFO: `/v3/projects/83aeaa07fb5d40d3b285aeb01d1252b1` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:07:10.292858 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b85dd4f-16f6-45ee-825e-c9171b9716eb None admin] RBAC: Authorizing `identity:get_project(project_id=83aeaa07fb5d40d3b285aeb01d1252b1)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:07:10.294349 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b85dd4f-16f6-45ee-825e-c9171b9716eb None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:07:10.296263 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 65/133] 10.222.0.18 () {56 vars in 1186 bytes} [Tue Sep 13 03:07:10 2022] GET /identity/v3/projects/83aeaa07fb5d40d3b285aeb01d1252b1 => generated 298 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 03:07:10.302280 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b85dd4f-16f6-45ee-825e-c9171b9716eb None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:07:10.342270 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:07:10.388309 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc4bdd3c-9870-4104-ab51-8059e22e7282 None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:07:10.426706 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42166239-425c-4da5-9e23-61c82ce1ec10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:07:10.427703 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42166239-425c-4da5-9e23-61c82ce1ec10 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:07:10.428180 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42166239-425c-4da5-9e23-61c82ce1ec10 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:07:10.428678 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42166239-425c-4da5-9e23-61c82ce1ec10 None admin] PATH_INFO: `/v3/users/06632bd878b1474f8d5432f99e899155` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:07:10.430205 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42166239-425c-4da5-9e23-61c82ce1ec10 None admin] RBAC: Authorizing `identity:get_user(user_id=06632bd878b1474f8d5432f99e899155)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:07:10.432255 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42166239-425c-4da5-9e23-61c82ce1ec10 None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:07:10.434147 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 69/134] 10.222.0.18 () {56 vars in 1177 bytes} [Tue Sep 13 03:07:10 2022] GET /identity/v3/users/06632bd878b1474f8d5432f99e899155 => generated 275 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 03:07:10.439825 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42166239-425c-4da5-9e23-61c82ce1ec10 None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:07:10.457113 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:07:10.465549 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b85dd4f-16f6-45ee-825e-c9171b9716eb None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:07:10.528281 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0a7b9dc-4da9-4230-a432-8cb9d9723d38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:07:10.529043 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0a7b9dc-4da9-4230-a432-8cb9d9723d38 None admin] REQUEST_METHOD: `PUT` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:07:10.529236 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0a7b9dc-4da9-4230-a432-8cb9d9723d38 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:07:10.529512 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0a7b9dc-4da9-4230-a432-8cb9d9723d38 None admin] PATH_INFO: `/v3/projects/83aeaa07fb5d40d3b285aeb01d1252b1/users/06632bd878b1474f8d5432f99e899155/roles/1eb7a5b5203945c29e6d22d653f47847` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:07:10.532376 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0a7b9dc-4da9-4230-a432-8cb9d9723d38 None admin] RBAC: Authorizing `identity:create_grant(project_id=83aeaa07fb5d40d3b285aeb01d1252b1, user_id=06632bd878b1474f8d5432f99e899155, role_id=1eb7a5b5203945c29e6d22d653f47847)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:07:10.534461 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0a7b9dc-4da9-4230-a432-8cb9d9723d38 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:07:10.563596 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 66/135] 10.222.0.18 () {58 vars in 1439 bytes} [Tue Sep 13 03:07:10 2022] PUT /identity/v3/projects/83aeaa07fb5d40d3b285aeb01d1252b1/users/06632bd878b1474f8d5432f99e899155/roles/1eb7a5b5203945c29e6d22d653f47847 => generated 0 bytes in 262 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 13 03:07:10.565019 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-089ad566-42a2-4061-b5af-52c493f1a896 None None] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:07:10.565311 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-089ad566-42a2-4061-b5af-52c493f1a896 None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:07:10.565589 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-089ad566-42a2-4061-b5af-52c493f1a896 None None] PATH_INFO: `/` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:07:10.567379 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 67/136] 10.222.0.18 () {54 vars in 927 bytes} [Tue Sep 13 03:07:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 03:07:10.576764 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce8f1b01-5c50-4110-aa69-ed4b75f35022 None None] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:07:10.576764 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce8f1b01-5c50-4110-aa69-ed4b75f35022 None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:07:10.576764 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce8f1b01-5c50-4110-aa69-ed4b75f35022 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:07:10.611754 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.auth.core [None req-ce8f1b01-5c50-4110-aa69-ed4b75f35022 None None] MFA Rules not processed for user `49f280ab086b4610855c4ffa378517d9`. Rule list: `[]` (Enabled: `True`). {{(pid=81399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 03:07:10.613075 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-ce8f1b01-5c50-4110-aa69-ed4b75f35022 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:07:10.624729 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42166239-425c-4da5-9e23-61c82ce1ec10 None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:07:10.663267 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:07:10.667144 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e62a09d2-3af5-4f64-a0f4-452dedbec162 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:07:10.668199 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e62a09d2-3af5-4f64-a0f4-452dedbec162 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:07:10.668708 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e62a09d2-3af5-4f64-a0f4-452dedbec162 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:07:10.669188 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e62a09d2-3af5-4f64-a0f4-452dedbec162 None admin] PATH_INFO: `/v3/projects/d730110113be4b61aed1777ca25db340` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:07:10.670847 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e62a09d2-3af5-4f64-a0f4-452dedbec162 None admin] RBAC: Authorizing `identity:get_project(project_id=d730110113be4b61aed1777ca25db340)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:07:10.672718 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e62a09d2-3af5-4f64-a0f4-452dedbec162 None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:07:10.676578 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 70/137] 10.222.0.18 () {56 vars in 1186 bytes} [Tue Sep 13 03:07:10 2022] GET /identity/v3/projects/d730110113be4b61aed1777ca25db340 => generated 312 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 03:07:10.681356 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e62a09d2-3af5-4f64-a0f4-452dedbec162 None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:07:10.729663 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 68/138] 10.222.0.18 () {58 vars in 1024 bytes} [Tue Sep 13 03:07:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 13 03:07:10.812695 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e62a09d2-3af5-4f64-a0f4-452dedbec162 None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:07:10.871441 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-954274a6-e708-4dfd-9f78-71a69f70ad9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:07:10.872512 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-954274a6-e708-4dfd-9f78-71a69f70ad9c None admin] REQUEST_METHOD: `PUT` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:07:10.873072 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-954274a6-e708-4dfd-9f78-71a69f70ad9c None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:07:10.873538 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-954274a6-e708-4dfd-9f78-71a69f70ad9c None admin] PATH_INFO: `/v3/projects/d730110113be4b61aed1777ca25db340/users/06632bd878b1474f8d5432f99e899155/roles/1eb7a5b5203945c29e6d22d653f47847` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:07:10.875902 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-954274a6-e708-4dfd-9f78-71a69f70ad9c None admin] RBAC: Authorizing `identity:create_grant(project_id=d730110113be4b61aed1777ca25db340, user_id=06632bd878b1474f8d5432f99e899155, role_id=1eb7a5b5203945c29e6d22d653f47847)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:07:10.878109 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-954274a6-e708-4dfd-9f78-71a69f70ad9c None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:07:10.906004 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 71/139] 10.222.0.18 () {58 vars in 1439 bytes} [Tue Sep 13 03:07:10 2022] PUT /identity/v3/projects/d730110113be4b61aed1777ca25db340/users/06632bd878b1474f8d5432f99e899155/roles/1eb7a5b5203945c29e6d22d653f47847 => generated 0 bytes in 225 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 03:07:11.275222 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce8f1b01-5c50-4110-aa69-ed4b75f35022 None None] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:07:11.341313 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a31f535-fdf1-4bc7-b060-a02cd7bf88df None None] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:07:11.341961 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a31f535-fdf1-4bc7-b060-a02cd7bf88df None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:07:11.342442 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a31f535-fdf1-4bc7-b060-a02cd7bf88df None None] PATH_INFO: `/` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:07:11.344284 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 72/140] 10.222.0.18 () {54 vars in 927 bytes} [Tue Sep 13 03:07:11 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 03:07:11.350537 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b745fe4-2eff-465e-8a85-f5f650c0e307 None None] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:07:11.351110 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b745fe4-2eff-465e-8a85-f5f650c0e307 None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:07:11.351573 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b745fe4-2eff-465e-8a85-f5f650c0e307 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:07:11.373586 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.auth.core [None req-0b745fe4-2eff-465e-8a85-f5f650c0e307 None None] MFA Rules not processed for user `49f280ab086b4610855c4ffa378517d9`. Rule list: `[]` (Enabled: `True`). {{(pid=81400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 03:07:11.375108 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-0b745fe4-2eff-465e-8a85-f5f650c0e307 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:07:11.382639 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:07:11.416742 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce8f1b01-5c50-4110-aa69-ed4b75f35022 None None] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:07:11.447071 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:07:11.457840 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 73/141] 10.222.0.18 () {58 vars in 1024 bytes} [Tue Sep 13 03:07:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 13 03:07:11.501750 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa694dd2-f9dd-47b7-a880-e03351bfce2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:07:11.502404 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa694dd2-f9dd-47b7-a880-e03351bfce2c None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:07:11.502716 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa694dd2-f9dd-47b7-a880-e03351bfce2c None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:07:11.503000 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa694dd2-f9dd-47b7-a880-e03351bfce2c None admin] PATH_INFO: `/v3/domains/default` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:07:11.507273 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa694dd2-f9dd-47b7-a880-e03351bfce2c None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:07:11.508753 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa694dd2-f9dd-47b7-a880-e03351bfce2c None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:07:11.510567 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 69/142] 10.222.0.18 () {56 vars in 1108 bytes} [Tue Sep 13 03:07:11 2022] GET /identity/v3/domains/default => generated 199 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 03:07:11.516022 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b745fe4-2eff-465e-8a85-f5f650c0e307 None None] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:07:11.656400 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b745fe4-2eff-465e-8a85-f5f650c0e307 None None] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:07:11.704812 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5f2aa67-49bb-4c7e-9c0f-8193b1baa3cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:07:11.705769 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f2aa67-49bb-4c7e-9c0f-8193b1baa3cf None admin] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:07:11.706312 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f2aa67-49bb-4c7e-9c0f-8193b1baa3cf None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:07:11.706788 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f2aa67-49bb-4c7e-9c0f-8193b1baa3cf None admin] PATH_INFO: `/v3/users` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:07:11.707618 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5f2aa67-49bb-4c7e-9c0f-8193b1baa3cf None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:07:11.709284 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5f2aa67-49bb-4c7e-9c0f-8193b1baa3cf None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:07:11.754379 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 74/143] 10.222.0.18 () {60 vars in 1132 bytes} [Tue Sep 13 03:07:11 2022] POST /identity/v3/users => generated 283 bytes in 239 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 03:07:11.832163 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa694dd2-f9dd-47b7-a880-e03351bfce2c None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:07:11.973673 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa694dd2-f9dd-47b7-a880-e03351bfce2c None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:07:12.015779 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92e7fc21-9330-4452-ba17-07243509d91a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:07:12.016425 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92e7fc21-9330-4452-ba17-07243509d91a None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:07:12.016710 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92e7fc21-9330-4452-ba17-07243509d91a None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:07:12.017528 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92e7fc21-9330-4452-ba17-07243509d91a None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:07:12.028003 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92e7fc21-9330-4452-ba17-07243509d91a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 03:07:12.033518 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application [None req-92e7fc21-9330-4452-ba17-07243509d91a None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 13 03:07:12.033518 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 03:07:12.033518 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 03:07:12.033518 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 03:07:12.033518 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 03:07:12.033518 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 03:07:12.033518 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 03:07:12.033518 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 03:07:12.033518 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 03:07:12.033518 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 03:07:12.033518 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 03:07:12.033518 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 03:07:12.033518 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 03:07:12.033518 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 03:07:12.033518 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 03:07:12.033518 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 03:07:12.034369 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 03:07:12.034369 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 03:07:12.034369 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 03:07:12.034369 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 03:07:12.034369 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 03:07:12.034369 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 03:07:12.034369 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 03:07:12.034369 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 03:07:12.034369 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 03:07:12.034369 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 03:07:12.034369 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 03:07:12.034369 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application with Lock( Sep 13 03:07:12.034369 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 03:07:12.034369 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._enter() Sep 13 03:07:12.034369 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 03:07:12.034369 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 03:07:12.035165 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 03:07:12.035165 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.creator() Sep 13 03:07:12.035165 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 03:07:12.035165 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application created_value = creator( Sep 13 03:07:12.035165 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 03:07:12.035165 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 03:07:12.035165 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 03:07:12.035165 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 03:07:12.035165 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 03:07:12.035165 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 03:07:12.035165 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 13 03:07:12.035165 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Sep 13 03:07:12.035165 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 70/144] 10.222.0.18 () {56 vars in 1099 bytes} [Tue Sep 13 03:07:11 2022] GET /identity/v3/roles/reader => generated 84 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 03:07:12.040609 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5f2aa67-49bb-4c7e-9c0f-8193b1baa3cf None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:07:12.102286 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:07:12.174868 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5f2aa67-49bb-4c7e-9c0f-8193b1baa3cf None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:07:12.221742 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0984f6a-6a67-4a35-b582-102c6cc21736 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:07:12.222359 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0984f6a-6a67-4a35-b582-102c6cc21736 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:07:12.222677 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0984f6a-6a67-4a35-b582-102c6cc21736 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:07:12.222956 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0984f6a-6a67-4a35-b582-102c6cc21736 None admin] PATH_INFO: `/v3/roles` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:07:12.223626 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0984f6a-6a67-4a35-b582-102c6cc21736 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 03:07:12.223980 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0984f6a-6a67-4a35-b582-102c6cc21736 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:07:12.225327 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0984f6a-6a67-4a35-b582-102c6cc21736 None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:07:12.237704 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 75/145] 10.222.0.18 () {56 vars in 1101 bytes} [Tue Sep 13 03:07:12 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 03:07:12.243377 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92e7fc21-9330-4452-ba17-07243509d91a None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:07:12.259484 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:07:12.357848 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92e7fc21-9330-4452-ba17-07243509d91a None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:07:12.398993 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03b4f26e-b93c-44db-8fd3-44efa77cf5f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:07:12.399637 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b4f26e-b93c-44db-8fd3-44efa77cf5f8 None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:07:12.399916 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b4f26e-b93c-44db-8fd3-44efa77cf5f8 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:07:12.400191 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b4f26e-b93c-44db-8fd3-44efa77cf5f8 None admin] PATH_INFO: `/v3/users/cfb765821b764d438d4fcad2b5d6007e` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:07:12.401340 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03b4f26e-b93c-44db-8fd3-44efa77cf5f8 None admin] RBAC: Authorizing `identity:get_user(user_id=cfb765821b764d438d4fcad2b5d6007e)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:07:12.402889 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03b4f26e-b93c-44db-8fd3-44efa77cf5f8 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:07:12.404492 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 71/146] 10.222.0.18 () {56 vars in 1177 bytes} [Tue Sep 13 03:07:12 2022] GET /identity/v3/users/cfb765821b764d438d4fcad2b5d6007e => generated 289 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 03:07:12.410291 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0984f6a-6a67-4a35-b582-102c6cc21736 None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:07:12.534758 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0984f6a-6a67-4a35-b582-102c6cc21736 None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:07:12.544907 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:07:12.573355 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80b04729-f4e2-4da1-9654-e7479ec94182 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:07:12.573992 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b04729-f4e2-4da1-9654-e7479ec94182 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:07:12.574268 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b04729-f4e2-4da1-9654-e7479ec94182 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:07:12.574541 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b04729-f4e2-4da1-9654-e7479ec94182 None admin] PATH_INFO: `/v3/projects/83aeaa07fb5d40d3b285aeb01d1252b1` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:07:12.575680 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80b04729-f4e2-4da1-9654-e7479ec94182 None admin] RBAC: Authorizing `identity:get_project(project_id=83aeaa07fb5d40d3b285aeb01d1252b1)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:07:12.577138 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80b04729-f4e2-4da1-9654-e7479ec94182 None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:07:12.578800 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 76/147] 10.222.0.18 () {56 vars in 1186 bytes} [Tue Sep 13 03:07:12 2022] GET /identity/v3/projects/83aeaa07fb5d40d3b285aeb01d1252b1 => generated 298 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 03:07:12.584758 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03b4f26e-b93c-44db-8fd3-44efa77cf5f8 None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:07:12.676955 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:07:12.700950 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03b4f26e-b93c-44db-8fd3-44efa77cf5f8 None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:07:12.739579 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-858d003f-b71f-49fd-bd64-231bcc41b1bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:07:12.740308 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-858d003f-b71f-49fd-bd64-231bcc41b1bb None admin] REQUEST_METHOD: `PUT` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:07:12.740591 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-858d003f-b71f-49fd-bd64-231bcc41b1bb None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:07:12.740863 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-858d003f-b71f-49fd-bd64-231bcc41b1bb None admin] PATH_INFO: `/v3/projects/83aeaa07fb5d40d3b285aeb01d1252b1/users/cfb765821b764d438d4fcad2b5d6007e/roles/502e6c11e747432988d7f723cee0af64` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:07:12.742827 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-858d003f-b71f-49fd-bd64-231bcc41b1bb None admin] RBAC: Authorizing `identity:create_grant(project_id=83aeaa07fb5d40d3b285aeb01d1252b1, user_id=cfb765821b764d438d4fcad2b5d6007e, role_id=502e6c11e747432988d7f723cee0af64)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:07:12.744393 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-858d003f-b71f-49fd-bd64-231bcc41b1bb None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:07:12.773541 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 72/148] 10.222.0.18 () {58 vars in 1439 bytes} [Tue Sep 13 03:07:12 2022] PUT /identity/v3/projects/83aeaa07fb5d40d3b285aeb01d1252b1/users/cfb765821b764d438d4fcad2b5d6007e/roles/502e6c11e747432988d7f723cee0af64 => generated 0 bytes in 189 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 03:07:13.344611 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e3ad4c1-0fea-47c2-a6ea-daaa163e2b6e None None] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:07:13.345675 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e3ad4c1-0fea-47c2-a6ea-daaa163e2b6e None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:07:13.345764 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e3ad4c1-0fea-47c2-a6ea-daaa163e2b6e None None] PATH_INFO: `/` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:07:13.347112 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 77/149] 10.222.0.18 () {54 vars in 927 bytes} [Tue Sep 13 03:07:13 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 03:07:13.352689 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c1d7e9-eb70-4962-b128-ce20d578d18b None None] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:07:13.353117 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c1d7e9-eb70-4962-b128-ce20d578d18b None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:07:13.353312 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c1d7e9-eb70-4962-b128-ce20d578d18b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:07:13.374471 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.auth.core [None req-14c1d7e9-eb70-4962-b128-ce20d578d18b None None] MFA Rules not processed for user `49f280ab086b4610855c4ffa378517d9`. Rule list: `[]` (Enabled: `True`). {{(pid=81399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 03:07:13.375779 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-14c1d7e9-eb70-4962-b128-ce20d578d18b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:07:13.452759 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 73/150] 10.222.0.18 () {58 vars in 1024 bytes} [Tue Sep 13 03:07:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 13 03:07:13.717934 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-365eafcf-850d-40dc-91d2-44ce551a250d None None] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:07:13.718323 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-365eafcf-850d-40dc-91d2-44ce551a250d None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:07:13.718513 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-365eafcf-850d-40dc-91d2-44ce551a250d None None] PATH_INFO: `/` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:07:13.720075 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 78/151] 10.222.0.18 () {54 vars in 927 bytes} [Tue Sep 13 03:07:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 03:07:13.725752 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0abf3926-c890-42e6-ac4c-312f5db5e572 None None] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:07:13.726046 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0abf3926-c890-42e6-ac4c-312f5db5e572 None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:07:13.726326 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0abf3926-c890-42e6-ac4c-312f5db5e572 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:07:13.749005 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.auth.core [None req-0abf3926-c890-42e6-ac4c-312f5db5e572 None None] MFA Rules not processed for user `49f280ab086b4610855c4ffa378517d9`. Rule list: `[]` (Enabled: `True`). {{(pid=81399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 03:07:13.750194 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-0abf3926-c890-42e6-ac4c-312f5db5e572 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:07:13.824470 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:07:13.830468 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 74/152] 10.222.0.18 () {58 vars in 1024 bytes} [Tue Sep 13 03:07:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 13 03:07:13.943181 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-365eafcf-850d-40dc-91d2-44ce551a250d None None] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:07:14.075383 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-365eafcf-850d-40dc-91d2-44ce551a250d None None] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:07:14.120339 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-782fa06d-d962-484e-b93d-7220f4c734a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:07:14.120978 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-782fa06d-d962-484e-b93d-7220f4c734a1 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:07:14.121258 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-782fa06d-d962-484e-b93d-7220f4c734a1 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:07:14.121529 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-782fa06d-d962-484e-b93d-7220f4c734a1 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:07:14.132792 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-782fa06d-d962-484e-b93d-7220f4c734a1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 03:07:14.138355 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application [None req-782fa06d-d962-484e-b93d-7220f4c734a1 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 13 03:07:14.138355 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 03:07:14.138355 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 03:07:14.138355 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 03:07:14.138355 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 03:07:14.138355 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 03:07:14.138355 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 03:07:14.138355 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 03:07:14.138355 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 03:07:14.138355 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 03:07:14.138355 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 03:07:14.138355 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 03:07:14.138355 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 03:07:14.138355 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 03:07:14.138355 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 03:07:14.138355 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 03:07:14.139337 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 03:07:14.139337 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 03:07:14.139337 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 03:07:14.139337 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 03:07:14.139337 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 03:07:14.139337 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 03:07:14.139337 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 03:07:14.139337 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 03:07:14.139337 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 03:07:14.139337 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 03:07:14.139337 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 03:07:14.139337 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application with Lock( Sep 13 03:07:14.139337 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 03:07:14.139337 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._enter() Sep 13 03:07:14.139337 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 03:07:14.139337 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 03:07:14.140756 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 03:07:14.140756 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.creator() Sep 13 03:07:14.140756 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 03:07:14.140756 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application created_value = creator( Sep 13 03:07:14.140756 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 03:07:14.140756 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 03:07:14.140756 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 03:07:14.140756 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 03:07:14.140756 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 03:07:14.140756 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 03:07:14.140756 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 13 03:07:14.140756 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Sep 13 03:07:14.140756 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 79/153] 10.222.0.18 () {56 vars in 1114 bytes} [Tue Sep 13 03:07:13 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 03:07:14.145727 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0abf3926-c890-42e6-ac4c-312f5db5e572 None None] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:07:14.257052 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0abf3926-c890-42e6-ac4c-312f5db5e572 None None] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:07:14.295235 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d981133-ca36-4018-b356-f58891bea90d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:07:14.295867 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d981133-ca36-4018-b356-f58891bea90d None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:07:14.296147 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d981133-ca36-4018-b356-f58891bea90d None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:07:14.296416 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d981133-ca36-4018-b356-f58891bea90d None admin] PATH_INFO: `/v3/roles` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:07:14.297096 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d981133-ca36-4018-b356-f58891bea90d None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 03:07:14.297399 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d981133-ca36-4018-b356-f58891bea90d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:07:14.299431 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d981133-ca36-4018-b356-f58891bea90d None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:07:14.308699 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 75/154] 10.222.0.18 () {56 vars in 1111 bytes} [Tue Sep 13 03:07:14 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 03:07:14.315502 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-782fa06d-d962-484e-b93d-7220f4c734a1 None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:07:14.351559 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:07:14.419114 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-782fa06d-d962-484e-b93d-7220f4c734a1 None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:07:14.431619 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d981133-ca36-4018-b356-f58891bea90d None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:07:14.491066 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b41e181d-7329-4ca6-9d97-0b30d7475d19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:07:14.491773 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b41e181d-7329-4ca6-9d97-0b30d7475d19 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:07:14.492052 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b41e181d-7329-4ca6-9d97-0b30d7475d19 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:07:14.492329 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b41e181d-7329-4ca6-9d97-0b30d7475d19 None admin] PATH_INFO: `/v3/users/06632bd878b1474f8d5432f99e899155` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:07:14.496274 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b41e181d-7329-4ca6-9d97-0b30d7475d19 None admin] RBAC: Authorizing `identity:get_user(user_id=06632bd878b1474f8d5432f99e899155)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:07:14.497784 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b41e181d-7329-4ca6-9d97-0b30d7475d19 None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:07:14.503649 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 80/155] 10.222.0.18 () {56 vars in 1177 bytes} [Tue Sep 13 03:07:14 2022] GET /identity/v3/users/06632bd878b1474f8d5432f99e899155 => generated 275 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 03:07:14.511297 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e8120cf-abb3-42a4-aaf7-07a201e6cc82 None None] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:07:14.511599 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e8120cf-abb3-42a4-aaf7-07a201e6cc82 None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:07:14.511945 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e8120cf-abb3-42a4-aaf7-07a201e6cc82 None None] PATH_INFO: `/` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:07:14.513679 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 81/156] 10.222.0.18 () {54 vars in 927 bytes} [Tue Sep 13 03:07:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 03:07:14.519290 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f5470c-394a-4031-b409-3c7fbce22ddb None None] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:07:14.519575 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f5470c-394a-4031-b409-3c7fbce22ddb None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:07:14.519947 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f5470c-394a-4031-b409-3c7fbce22ddb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:07:14.536239 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:07:14.555113 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.auth.core [None req-40f5470c-394a-4031-b409-3c7fbce22ddb None None] MFA Rules not processed for user `49f280ab086b4610855c4ffa378517d9`. Rule list: `[]` (Enabled: `True`). {{(pid=81400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 03:07:14.556666 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-40f5470c-394a-4031-b409-3c7fbce22ddb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:07:14.578295 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d981133-ca36-4018-b356-f58891bea90d None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:07:14.614330 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26fcd24d-f78a-433a-acd5-75505145752c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:07:14.614950 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26fcd24d-f78a-433a-acd5-75505145752c None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:07:14.615228 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26fcd24d-f78a-433a-acd5-75505145752c None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:07:14.615499 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26fcd24d-f78a-433a-acd5-75505145752c None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:07:14.626629 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26fcd24d-f78a-433a-acd5-75505145752c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 03:07:14.632358 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application [None req-26fcd24d-f78a-433a-acd5-75505145752c None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 13 03:07:14.632358 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 03:07:14.632358 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 03:07:14.632358 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 03:07:14.632358 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 03:07:14.632358 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 03:07:14.632358 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 03:07:14.632358 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 03:07:14.632358 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 03:07:14.632358 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 03:07:14.632358 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 03:07:14.632358 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 03:07:14.632358 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 03:07:14.632358 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 03:07:14.632358 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 03:07:14.632358 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 03:07:14.633395 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 03:07:14.633395 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 03:07:14.633395 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 03:07:14.633395 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 03:07:14.633395 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 03:07:14.633395 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 03:07:14.633395 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 03:07:14.633395 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 03:07:14.633395 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 03:07:14.633395 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 03:07:14.633395 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 03:07:14.633395 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application with Lock( Sep 13 03:07:14.633395 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 03:07:14.633395 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._enter() Sep 13 03:07:14.633395 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 03:07:14.633395 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 03:07:14.634234 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 03:07:14.634234 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.creator() Sep 13 03:07:14.634234 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 03:07:14.634234 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application created_value = creator( Sep 13 03:07:14.634234 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 03:07:14.634234 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 03:07:14.634234 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 03:07:14.634234 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 03:07:14.634234 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 03:07:14.634234 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 03:07:14.634234 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 13 03:07:14.634234 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Sep 13 03:07:14.634234 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 76/157] 10.222.0.18 () {56 vars in 1096 bytes} [Tue Sep 13 03:07:14 2022] GET /identity/v3/roles/admin => generated 83 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 13 03:07:14.635381 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26fcd24d-f78a-433a-acd5-75505145752c None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:07:14.702448 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 82/158] 10.222.0.18 () {58 vars in 1024 bytes} [Tue Sep 13 03:07:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 13 03:07:14.703473 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40f5470c-394a-4031-b409-3c7fbce22ddb None None] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:07:14.741817 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26fcd24d-f78a-433a-acd5-75505145752c None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:07:14.775079 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:07:14.780655 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9f0a49d-345f-4a1e-8beb-ed2744a6f727 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:07:14.781518 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f0a49d-345f-4a1e-8beb-ed2744a6f727 None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:07:14.781798 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f0a49d-345f-4a1e-8beb-ed2744a6f727 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:07:14.782073 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f0a49d-345f-4a1e-8beb-ed2744a6f727 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:07:14.791036 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:07:14.926360 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9f0a49d-345f-4a1e-8beb-ed2744a6f727 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:07:14.928322 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40f5470c-394a-4031-b409-3c7fbce22ddb None None] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:07:14.931571 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9f0a49d-345f-4a1e-8beb-ed2744a6f727 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:07:15.023342 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0d0de78-8d46-40ce-9c53-5273b15d45f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:07:15.023979 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d0de78-8d46-40ce-9c53-5273b15d45f6 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:07:15.024257 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d0de78-8d46-40ce-9c53-5273b15d45f6 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:07:15.024538 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d0de78-8d46-40ce-9c53-5273b15d45f6 None admin] PATH_INFO: `/v3/roles` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:07:15.025213 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0d0de78-8d46-40ce-9c53-5273b15d45f6 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 03:07:15.025519 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0d0de78-8d46-40ce-9c53-5273b15d45f6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:07:15.027002 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0d0de78-8d46-40ce-9c53-5273b15d45f6 None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:07:15.044003 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 77/159] 10.222.0.18 () {58 vars in 1282 bytes} [Tue Sep 13 03:07:14 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 410 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 13 03:07:15.049117 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 83/160] 10.222.0.18 () {56 vars in 1099 bytes} [Tue Sep 13 03:07:14 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 346 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 03:07:15.051033 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9f0a49d-345f-4a1e-8beb-ed2744a6f727 None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:07:15.055196 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0d0de78-8d46-40ce-9c53-5273b15d45f6 None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:07:15.082646 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:07:15.165376 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9f0a49d-345f-4a1e-8beb-ed2744a6f727 None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:07:15.201900 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8aead3d-ba95-4b9d-b884-04e3fe28621f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:07:15.202967 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8aead3d-ba95-4b9d-b884-04e3fe28621f None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:07:15.203406 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8aead3d-ba95-4b9d-b884-04e3fe28621f None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:07:15.203834 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8aead3d-ba95-4b9d-b884-04e3fe28621f None admin] PATH_INFO: `/v3/projects/83aeaa07fb5d40d3b285aeb01d1252b1` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:07:15.205148 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8aead3d-ba95-4b9d-b884-04e3fe28621f None admin] RBAC: Authorizing `identity:get_project(project_id=83aeaa07fb5d40d3b285aeb01d1252b1)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:07:15.206866 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8aead3d-ba95-4b9d-b884-04e3fe28621f None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:07:15.209437 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 78/161] 10.222.0.18 () {56 vars in 1186 bytes} [Tue Sep 13 03:07:15 2022] GET /identity/v3/projects/83aeaa07fb5d40d3b285aeb01d1252b1 => generated 298 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 03:07:15.218864 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8aead3d-ba95-4b9d-b884-04e3fe28621f None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:07:15.296930 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:07:15.307869 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0d0de78-8d46-40ce-9c53-5273b15d45f6 None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:07:15.334961 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:07:15.341942 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8aead3d-ba95-4b9d-b884-04e3fe28621f None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:07:15.382380 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b224a613-38ca-48e0-a482-93e032081f5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:07:15.384417 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b224a613-38ca-48e0-a482-93e032081f5b None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:07:15.384696 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b224a613-38ca-48e0-a482-93e032081f5b None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:07:15.384967 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b224a613-38ca-48e0-a482-93e032081f5b None admin] PATH_INFO: `/v3/users/49f280ab086b4610855c4ffa378517d9` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:07:15.388414 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b224a613-38ca-48e0-a482-93e032081f5b None admin] RBAC: Authorizing `identity:get_user(user_id=49f280ab086b4610855c4ffa378517d9)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:07:15.389903 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b224a613-38ca-48e0-a482-93e032081f5b None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:07:15.391569 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 84/162] 10.222.0.18 () {56 vars in 1177 bytes} [Tue Sep 13 03:07:15 2022] GET /identity/v3/users/49f280ab086b4610855c4ffa378517d9 => generated 247 bytes in 337 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 03:07:15.397725 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b224a613-38ca-48e0-a482-93e032081f5b None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:07:15.402175 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72421224-8922-4778-81ad-498003abffbc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:07:15.403205 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72421224-8922-4778-81ad-498003abffbc None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:07:15.403205 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72421224-8922-4778-81ad-498003abffbc None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:07:15.403439 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72421224-8922-4778-81ad-498003abffbc None admin] PATH_INFO: `/v3/roles/member` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:07:15.415311 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72421224-8922-4778-81ad-498003abffbc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 03:07:15.420660 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application [None req-72421224-8922-4778-81ad-498003abffbc None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 13 03:07:15.420660 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 03:07:15.420660 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 03:07:15.420660 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 03:07:15.420660 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 03:07:15.420660 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 03:07:15.420660 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 03:07:15.420660 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 03:07:15.420660 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 03:07:15.420660 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 03:07:15.420660 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 03:07:15.420660 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 03:07:15.420660 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 03:07:15.420660 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 03:07:15.420660 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 03:07:15.420660 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 03:07:15.421564 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 03:07:15.421564 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 03:07:15.421564 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 03:07:15.421564 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 03:07:15.421564 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 03:07:15.421564 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 03:07:15.421564 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 03:07:15.421564 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 03:07:15.421564 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 03:07:15.421564 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 03:07:15.421564 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 03:07:15.421564 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application with Lock( Sep 13 03:07:15.421564 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 03:07:15.421564 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._enter() Sep 13 03:07:15.421564 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 03:07:15.421564 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 03:07:15.422354 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 03:07:15.422354 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.creator() Sep 13 03:07:15.422354 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 03:07:15.422354 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application created_value = creator( Sep 13 03:07:15.422354 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 03:07:15.422354 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 03:07:15.422354 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 03:07:15.422354 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 03:07:15.422354 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 03:07:15.422354 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 03:07:15.422354 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 13 03:07:15.422354 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Sep 13 03:07:15.422354 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 79/163] 10.222.0.18 () {56 vars in 1099 bytes} [Tue Sep 13 03:07:15 2022] GET /identity/v3/roles/member => generated 84 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 03:07:15.423119 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72421224-8922-4778-81ad-498003abffbc None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:07:15.530503 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72421224-8922-4778-81ad-498003abffbc None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:07:15.567777 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5ce3e1b-7d16-4781-895b-a8da850c5d7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:07:15.567777 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ce3e1b-7d16-4781-895b-a8da850c5d7f None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:07:15.567777 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ce3e1b-7d16-4781-895b-a8da850c5d7f None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:07:15.567777 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ce3e1b-7d16-4781-895b-a8da850c5d7f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:07:15.588147 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:07:15.684887 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5ce3e1b-7d16-4781-895b-a8da850c5d7f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:07:15.686501 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5ce3e1b-7d16-4781-895b-a8da850c5d7f None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:07:15.707542 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b224a613-38ca-48e0-a482-93e032081f5b None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:07:15.790116 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-717ebaf2-6f2f-4d12-995a-d033aaaf6621 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:07:15.791469 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-717ebaf2-6f2f-4d12-995a-d033aaaf6621 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:07:15.791755 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-717ebaf2-6f2f-4d12-995a-d033aaaf6621 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:07:15.792720 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-717ebaf2-6f2f-4d12-995a-d033aaaf6621 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:07:15.799288 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-717ebaf2-6f2f-4d12-995a-d033aaaf6621 None admin] RBAC: Adding query filter params (role.id=a9aa35ebda4d4aff80e62f750e9104ab, scope.project.id=83aeaa07fb5d40d3b285aeb01d1252b1, user.id=06632bd878b1474f8d5432f99e899155) {{(pid=81400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 03:07:15.800673 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-717ebaf2-6f2f-4d12-995a-d033aaaf6621 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:07:15.802910 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-717ebaf2-6f2f-4d12-995a-d033aaaf6621 None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:07:15.820811 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 85/164] 10.222.0.18 () {56 vars in 1374 bytes} [Tue Sep 13 03:07:15 2022] GET /identity/v3/role_assignments?user.id=06632bd878b1474f8d5432f99e899155&scope.project.id=83aeaa07fb5d40d3b285aeb01d1252b1&role.id=a9aa35ebda4d4aff80e62f750e9104ab => generated 608 bytes in 429 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 03:07:15.825265 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c187c2e7-f301-4152-99ea-8e4e4a5bcc77 None None] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:07:15.839181 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c187c2e7-f301-4152-99ea-8e4e4a5bcc77 None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:07:15.839524 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c187c2e7-f301-4152-99ea-8e4e4a5bcc77 None None] PATH_INFO: `/` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:07:15.840981 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 86/165] 10.222.0.18 () {54 vars in 927 bytes} [Tue Sep 13 03:07:15 2022] GET /identity => generated 270 bytes in 20 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 03:07:15.841738 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c187c2e7-f301-4152-99ea-8e4e4a5bcc77 None None] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:07:15.920070 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 80/166] 10.222.0.18 () {58 vars in 1282 bytes} [Tue Sep 13 03:07:15 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 497 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 13 03:07:15.921352 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf15f872-d89b-46c0-b6e7-f4803f5eb6fd None None] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:07:15.921632 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf15f872-d89b-46c0-b6e7-f4803f5eb6fd None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:07:15.921900 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf15f872-d89b-46c0-b6e7-f4803f5eb6fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:07:15.934998 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:07:15.984826 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.auth.core [None req-bf15f872-d89b-46c0-b6e7-f4803f5eb6fd None None] MFA Rules not processed for user `49f280ab086b4610855c4ffa378517d9`. Rule list: `[]` (Enabled: `True`). {{(pid=81399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 03:07:15.986067 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-bf15f872-d89b-46c0-b6e7-f4803f5eb6fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:07:16.119358 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:07:16.150998 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 81/167] 10.222.0.18 () {58 vars in 1024 bytes} [Tue Sep 13 03:07:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 230 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 13 03:07:16.151675 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf15f872-d89b-46c0-b6e7-f4803f5eb6fd None None] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:07:16.161363 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c187c2e7-f301-4152-99ea-8e4e4a5bcc77 None None] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:07:16.249641 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f33d205d-2b97-4e63-98d1-f400f900baf4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:07:16.250639 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f33d205d-2b97-4e63-98d1-f400f900baf4 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:07:16.251671 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f33d205d-2b97-4e63-98d1-f400f900baf4 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:07:16.252154 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f33d205d-2b97-4e63-98d1-f400f900baf4 None admin] PATH_INFO: `/v3/roles` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:07:16.253033 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f33d205d-2b97-4e63-98d1-f400f900baf4 None admin] RBAC: Adding query filter params (name=member) {{(pid=81400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 03:07:16.253555 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f33d205d-2b97-4e63-98d1-f400f900baf4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:07:16.255180 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f33d205d-2b97-4e63-98d1-f400f900baf4 None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:07:16.278214 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 87/168] 10.222.0.18 () {56 vars in 1101 bytes} [Tue Sep 13 03:07:15 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 437 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 03:07:16.284183 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f33d205d-2b97-4e63-98d1-f400f900baf4 None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:07:16.300683 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf15f872-d89b-46c0-b6e7-f4803f5eb6fd None None] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:07:16.344686 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebdd009c-3b8f-4a15-bdf6-d324f6dd5368 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:07:16.345462 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebdd009c-3b8f-4a15-bdf6-d324f6dd5368 None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:07:16.345747 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebdd009c-3b8f-4a15-bdf6-d324f6dd5368 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:07:16.346012 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebdd009c-3b8f-4a15-bdf6-d324f6dd5368 None admin] PATH_INFO: `/v3/projects/83aeaa07fb5d40d3b285aeb01d1252b1` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:07:16.347169 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebdd009c-3b8f-4a15-bdf6-d324f6dd5368 None admin] RBAC: Authorizing `identity:get_project(project_id=83aeaa07fb5d40d3b285aeb01d1252b1)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:07:16.348609 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebdd009c-3b8f-4a15-bdf6-d324f6dd5368 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:07:16.350265 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 82/169] 10.222.0.18 () {56 vars in 1186 bytes} [Tue Sep 13 03:07:16 2022] GET /identity/v3/projects/83aeaa07fb5d40d3b285aeb01d1252b1 => generated 298 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 03:07:16.362530 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebdd009c-3b8f-4a15-bdf6-d324f6dd5368 None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:07:16.441036 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:07:16.467952 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:07:16.492975 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f33d205d-2b97-4e63-98d1-f400f900baf4 None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:07:16.555749 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebdd009c-3b8f-4a15-bdf6-d324f6dd5368 None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:07:16.563511 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81f84449-72b0-4e0d-8999-713a209fd4ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:07:16.564918 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81f84449-72b0-4e0d-8999-713a209fd4ca None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:07:16.565410 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81f84449-72b0-4e0d-8999-713a209fd4ca None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:07:16.565946 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81f84449-72b0-4e0d-8999-713a209fd4ca None admin] PATH_INFO: `/v3/users/06632bd878b1474f8d5432f99e899155` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:07:16.567410 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81f84449-72b0-4e0d-8999-713a209fd4ca None admin] RBAC: Authorizing `identity:get_user(user_id=06632bd878b1474f8d5432f99e899155)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:07:16.569074 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81f84449-72b0-4e0d-8999-713a209fd4ca None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:07:16.570915 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 88/170] 10.222.0.18 () {56 vars in 1177 bytes} [Tue Sep 13 03:07:16 2022] GET /identity/v3/users/06632bd878b1474f8d5432f99e899155 => generated 275 bytes in 287 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 03:07:16.576548 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81f84449-72b0-4e0d-8999-713a209fd4ca None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:07:16.611096 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d60a29ea-9586-4de3-ad11-9c878862f6e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:07:16.611096 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d60a29ea-9586-4de3-ad11-9c878862f6e4 None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:07:16.611096 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d60a29ea-9586-4de3-ad11-9c878862f6e4 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:07:16.611096 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d60a29ea-9586-4de3-ad11-9c878862f6e4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:07:16.611465 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d60a29ea-9586-4de3-ad11-9c878862f6e4 None admin] RBAC: Adding query filter params (role.id=425d4b481c7846e0bc07e7e805d5b15f, scope.project.id=83aeaa07fb5d40d3b285aeb01d1252b1, user.id=49f280ab086b4610855c4ffa378517d9) {{(pid=81399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 03:07:16.611465 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d60a29ea-9586-4de3-ad11-9c878862f6e4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:07:16.612369 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d60a29ea-9586-4de3-ad11-9c878862f6e4 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:07:16.624622 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 83/171] 10.222.0.18 () {56 vars in 1374 bytes} [Tue Sep 13 03:07:16 2022] GET /identity/v3/role_assignments?user.id=49f280ab086b4610855c4ffa378517d9&scope.project.id=83aeaa07fb5d40d3b285aeb01d1252b1&role.id=425d4b481c7846e0bc07e7e805d5b15f => generated 608 bytes in 262 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 03:07:16.737127 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81f84449-72b0-4e0d-8999-713a209fd4ca None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:07:16.789246 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d917d9e-c6f2-4d4f-82e2-57806a19c0b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:07:16.790299 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d917d9e-c6f2-4d4f-82e2-57806a19c0b6 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:07:16.790823 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d917d9e-c6f2-4d4f-82e2-57806a19c0b6 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:07:16.791308 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d917d9e-c6f2-4d4f-82e2-57806a19c0b6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:07:16.834983 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:07:16.880125 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d917d9e-c6f2-4d4f-82e2-57806a19c0b6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:07:16.881556 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d917d9e-c6f2-4d4f-82e2-57806a19c0b6 None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:07:16.935889 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d60a29ea-9586-4de3-ad11-9c878862f6e4 None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:07:17.004573 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 89/172] 10.222.0.18 () {58 vars in 1282 bytes} [Tue Sep 13 03:07:16 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 428 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Sep 13 03:07:17.011480 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d917d9e-c6f2-4d4f-82e2-57806a19c0b6 None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:07:17.083052 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:07:17.125609 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d917d9e-c6f2-4d4f-82e2-57806a19c0b6 None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:07:17.167201 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d60a29ea-9586-4de3-ad11-9c878862f6e4 None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:07:17.172262 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59ff70fa-954e-4b8a-a785-cd3301267e58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:07:17.172947 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ff70fa-954e-4b8a-a785-cd3301267e58 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:07:17.173235 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ff70fa-954e-4b8a-a785-cd3301267e58 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:07:17.173517 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ff70fa-954e-4b8a-a785-cd3301267e58 None admin] PATH_INFO: `/v3/projects/83aeaa07fb5d40d3b285aeb01d1252b1` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:07:17.174748 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59ff70fa-954e-4b8a-a785-cd3301267e58 None admin] RBAC: Authorizing `identity:get_project(project_id=83aeaa07fb5d40d3b285aeb01d1252b1)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:07:17.176439 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59ff70fa-954e-4b8a-a785-cd3301267e58 None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:07:17.178380 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 90/173] 10.222.0.18 () {56 vars in 1186 bytes} [Tue Sep 13 03:07:17 2022] GET /identity/v3/projects/83aeaa07fb5d40d3b285aeb01d1252b1 => generated 298 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 03:07:17.179543 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd27c464-d166-4816-a227-04086576a71d None None] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:07:17.179793 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd27c464-d166-4816-a227-04086576a71d None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:07:17.180358 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:07:17.181147 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd27c464-d166-4816-a227-04086576a71d None None] PATH_INFO: `/` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:07:17.183697 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 91/174] 10.222.0.18 () {54 vars in 927 bytes} [Tue Sep 13 03:07:17 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 03:07:17.185165 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0fa0d4a-400f-4d6d-aa60-72fb47c3481b None None] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:07:17.185611 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0fa0d4a-400f-4d6d-aa60-72fb47c3481b None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:07:17.186033 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0fa0d4a-400f-4d6d-aa60-72fb47c3481b None None] PATH_INFO: `/` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:07:17.187807 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 92/175] 10.222.0.18 () {54 vars in 927 bytes} [Tue Sep 13 03:07:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 03:07:17.188724 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0fa0d4a-400f-4d6d-aa60-72fb47c3481b None None] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:07:17.205074 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2662ca16-9f1e-46b1-ae23-dee9750aab56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:07:17.205711 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2662ca16-9f1e-46b1-ae23-dee9750aab56 None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:07:17.205985 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2662ca16-9f1e-46b1-ae23-dee9750aab56 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:07:17.206252 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2662ca16-9f1e-46b1-ae23-dee9750aab56 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:07:17.220633 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2662ca16-9f1e-46b1-ae23-dee9750aab56 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 03:07:17.226142 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application [None req-2662ca16-9f1e-46b1-ae23-dee9750aab56 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 13 03:07:17.226142 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 03:07:17.226142 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 03:07:17.226142 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 03:07:17.226142 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 03:07:17.226142 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 03:07:17.226142 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 03:07:17.226142 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 03:07:17.226142 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 03:07:17.226142 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 03:07:17.226142 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 03:07:17.226142 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 03:07:17.226142 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 03:07:17.226142 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 03:07:17.226142 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 03:07:17.226142 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 03:07:17.227251 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 03:07:17.227251 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 03:07:17.227251 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 03:07:17.227251 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 03:07:17.227251 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 03:07:17.227251 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 03:07:17.227251 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 03:07:17.227251 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 03:07:17.227251 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 03:07:17.227251 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 03:07:17.227251 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 03:07:17.227251 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application with Lock( Sep 13 03:07:17.227251 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 03:07:17.227251 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._enter() Sep 13 03:07:17.227251 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 03:07:17.227251 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 03:07:17.228243 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 03:07:17.228243 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.creator() Sep 13 03:07:17.228243 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 03:07:17.228243 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application created_value = creator( Sep 13 03:07:17.228243 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 03:07:17.228243 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 03:07:17.228243 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 03:07:17.228243 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 03:07:17.228243 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 03:07:17.228243 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 03:07:17.228243 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 13 03:07:17.228243 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Sep 13 03:07:17.229571 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 84/176] 10.222.0.18 () {56 vars in 1099 bytes} [Tue Sep 13 03:07:16 2022] GET /identity/v3/roles/member => generated 84 bytes in 294 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 13 03:07:17.230808 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9095ecd7-2680-4459-9fc3-5f4e85e61b33 None None] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:07:17.231087 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9095ecd7-2680-4459-9fc3-5f4e85e61b33 None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:07:17.231353 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9095ecd7-2680-4459-9fc3-5f4e85e61b33 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:07:17.255769 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.auth.core [None req-9095ecd7-2680-4459-9fc3-5f4e85e61b33 None None] MFA Rules not processed for user `49f280ab086b4610855c4ffa378517d9`. Rule list: `[]` (Enabled: `True`). {{(pid=81399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 03:07:17.256992 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-9095ecd7-2680-4459-9fc3-5f4e85e61b33 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:07:17.322404 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0fa0d4a-400f-4d6d-aa60-72fb47c3481b None None] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:07:17.337743 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 85/177] 10.222.0.18 () {58 vars in 1024 bytes} [Tue Sep 13 03:07:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 13 03:07:17.345877 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ac41eab-e605-4699-9640-1b57dd94febf None None] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:07:17.345877 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ac41eab-e605-4699-9640-1b57dd94febf None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:07:17.345877 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ac41eab-e605-4699-9640-1b57dd94febf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:07:17.359003 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.auth.core [None req-6ac41eab-e605-4699-9640-1b57dd94febf None None] MFA Rules not processed for user `49f280ab086b4610855c4ffa378517d9`. Rule list: `[]` (Enabled: `True`). {{(pid=81399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 03:07:17.359178 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:07:17.360389 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-6ac41eab-e605-4699-9640-1b57dd94febf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:07:17.371219 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3820e72-27dc-4357-91d1-f76239f5c2b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:07:17.371914 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3820e72-27dc-4357-91d1-f76239f5c2b2 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:07:17.372191 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3820e72-27dc-4357-91d1-f76239f5c2b2 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:07:17.372512 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3820e72-27dc-4357-91d1-f76239f5c2b2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:07:17.373229 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3820e72-27dc-4357-91d1-f76239f5c2b2 None admin] RBAC: Adding query filter params (role.id=1eb7a5b5203945c29e6d22d653f47847, scope.project.id=83aeaa07fb5d40d3b285aeb01d1252b1, user.id=06632bd878b1474f8d5432f99e899155) {{(pid=81400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 03:07:17.373585 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3820e72-27dc-4357-91d1-f76239f5c2b2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:07:17.375089 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3820e72-27dc-4357-91d1-f76239f5c2b2 None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:07:17.381927 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 93/178] 10.222.0.18 () {56 vars in 1374 bytes} [Tue Sep 13 03:07:17 2022] GET /identity/v3/role_assignments?user.id=06632bd878b1474f8d5432f99e899155&scope.project.id=83aeaa07fb5d40d3b285aeb01d1252b1&role.id=1eb7a5b5203945c29e6d22d653f47847 => generated 608 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 03:07:17.382799 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3820e72-27dc-4357-91d1-f76239f5c2b2 None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:07:17.444693 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 86/179] 10.222.0.18 () {58 vars in 1024 bytes} [Tue Sep 13 03:07:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 13 03:07:17.447231 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d3b03c-b269-41aa-b8da-75496530665b None None] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:07:17.447680 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d3b03c-b269-41aa-b8da-75496530665b None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:07:17.448089 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d3b03c-b269-41aa-b8da-75496530665b None None] PATH_INFO: `/` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:07:17.450097 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 87/180] 10.222.0.18 () {54 vars in 927 bytes} [Tue Sep 13 03:07:17 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 03:07:17.456593 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe672057-ddd8-440d-b347-443f17f225e3 None None] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:07:17.457093 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe672057-ddd8-440d-b347-443f17f225e3 None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:07:17.457502 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe672057-ddd8-440d-b347-443f17f225e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:07:17.497557 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.auth.core [None req-fe672057-ddd8-440d-b347-443f17f225e3 None None] MFA Rules not processed for user `49f280ab086b4610855c4ffa378517d9`. Rule list: `[]` (Enabled: `True`). {{(pid=81399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 03:07:17.500662 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-fe672057-ddd8-440d-b347-443f17f225e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:07:17.567539 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:07:17.576368 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3820e72-27dc-4357-91d1-f76239f5c2b2 None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:07:17.625862 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfe6bbf7-37f2-4efe-a16f-e7075737f9f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:07:17.627469 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe6bbf7-37f2-4efe-a16f-e7075737f9f2 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:07:17.628283 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe6bbf7-37f2-4efe-a16f-e7075737f9f2 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:07:17.628699 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe6bbf7-37f2-4efe-a16f-e7075737f9f2 None admin] PATH_INFO: `/v3/roles` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:07:17.629813 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfe6bbf7-37f2-4efe-a16f-e7075737f9f2 None admin] RBAC: Adding query filter params (name=member) {{(pid=81400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 03:07:17.630349 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfe6bbf7-37f2-4efe-a16f-e7075737f9f2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:07:17.632564 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfe6bbf7-37f2-4efe-a16f-e7075737f9f2 None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:07:17.638033 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 88/181] 10.222.0.18 () {58 vars in 1024 bytes} [Tue Sep 13 03:07:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 13 03:07:17.648108 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 94/182] 10.222.0.18 () {56 vars in 1101 bytes} [Tue Sep 13 03:07:17 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 266 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 03:07:17.654043 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe672057-ddd8-440d-b347-443f17f225e3 None None] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:07:17.744185 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e187c8-c9cd-4520-bffe-1ac6a252f636 None None] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:07:17.744353 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e187c8-c9cd-4520-bffe-1ac6a252f636 None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:07:17.744604 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e187c8-c9cd-4520-bffe-1ac6a252f636 None None] PATH_INFO: `/` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:07:17.745959 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 95/183] 10.222.0.18 () {54 vars in 927 bytes} [Tue Sep 13 03:07:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 03:07:17.752811 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-430fcc89-961c-4857-af4a-c04a723c77eb None None] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:07:17.753094 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-430fcc89-961c-4857-af4a-c04a723c77eb None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:07:17.753590 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-430fcc89-961c-4857-af4a-c04a723c77eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:07:17.763679 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe672057-ddd8-440d-b347-443f17f225e3 None None] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:07:17.774348 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.auth.core [None req-430fcc89-961c-4857-af4a-c04a723c77eb None None] MFA Rules not processed for user `49f280ab086b4610855c4ffa378517d9`. Rule list: `[]` (Enabled: `True`). {{(pid=81400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 03:07:17.775708 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-430fcc89-961c-4857-af4a-c04a723c77eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:07:17.810289 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f72c740-3044-4967-80f0-969340f52e10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:07:17.810985 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f72c740-3044-4967-80f0-969340f52e10 None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:07:17.811262 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f72c740-3044-4967-80f0-969340f52e10 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:07:17.811529 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f72c740-3044-4967-80f0-969340f52e10 None admin] PATH_INFO: `/v3/users/06632bd878b1474f8d5432f99e899155` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:07:17.814277 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f72c740-3044-4967-80f0-969340f52e10 None admin] RBAC: Authorizing `identity:get_user(user_id=06632bd878b1474f8d5432f99e899155)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:07:17.815772 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f72c740-3044-4967-80f0-969340f52e10 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:07:17.817652 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 89/184] 10.222.0.18 () {56 vars in 1177 bytes} [Tue Sep 13 03:07:17 2022] GET /identity/v3/users/06632bd878b1474f8d5432f99e899155 => generated 275 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 03:07:17.824156 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f72c740-3044-4967-80f0-969340f52e10 None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:07:17.843422 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:07:17.864863 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:07:17.875442 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 96/185] 10.222.0.18 () {58 vars in 1024 bytes} [Tue Sep 13 03:07:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 13 03:07:17.876921 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-430fcc89-961c-4857-af4a-c04a723c77eb None None] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:07:17.945897 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f72c740-3044-4967-80f0-969340f52e10 None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:07:17.981145 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-618d5675-f694-42b8-963b-61999a579295 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:07:17.981890 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-618d5675-f694-42b8-963b-61999a579295 None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:07:17.982161 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-618d5675-f694-42b8-963b-61999a579295 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:07:17.982426 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-618d5675-f694-42b8-963b-61999a579295 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:07:17.984187 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-430fcc89-961c-4857-af4a-c04a723c77eb None None] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:07:18.020820 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6945e5fb-e8f9-4eea-9304-c03063aacdad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:07:18.021375 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6945e5fb-e8f9-4eea-9304-c03063aacdad None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:07:18.021655 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6945e5fb-e8f9-4eea-9304-c03063aacdad None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:07:18.021929 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6945e5fb-e8f9-4eea-9304-c03063aacdad None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:07:18.031858 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6945e5fb-e8f9-4eea-9304-c03063aacdad None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 03:07:18.036923 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application [None req-6945e5fb-e8f9-4eea-9304-c03063aacdad None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 13 03:07:18.036923 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 03:07:18.036923 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 03:07:18.036923 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 03:07:18.036923 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 03:07:18.036923 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 03:07:18.036923 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 03:07:18.036923 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 03:07:18.036923 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 03:07:18.036923 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 03:07:18.036923 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 03:07:18.036923 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 03:07:18.036923 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 03:07:18.036923 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 03:07:18.036923 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 03:07:18.036923 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 03:07:18.037803 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 03:07:18.037803 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 03:07:18.037803 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 03:07:18.037803 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 03:07:18.037803 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 03:07:18.037803 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 03:07:18.037803 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 03:07:18.037803 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 03:07:18.037803 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 03:07:18.037803 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 03:07:18.037803 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 03:07:18.037803 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application with Lock( Sep 13 03:07:18.037803 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 03:07:18.037803 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._enter() Sep 13 03:07:18.037803 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 03:07:18.037803 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 03:07:18.039268 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 03:07:18.039268 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.creator() Sep 13 03:07:18.039268 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 03:07:18.039268 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application created_value = creator( Sep 13 03:07:18.039268 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 03:07:18.039268 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 03:07:18.039268 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 03:07:18.039268 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 03:07:18.039268 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 03:07:18.039268 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 03:07:18.039268 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 13 03:07:18.039268 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Sep 13 03:07:18.039268 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 97/186] 10.222.0.18 () {56 vars in 1096 bytes} [Tue Sep 13 03:07:17 2022] GET /identity/v3/roles/admin => generated 83 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 03:07:18.039268 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6945e5fb-e8f9-4eea-9304-c03063aacdad None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:07:18.061476 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-618d5675-f694-42b8-963b-61999a579295 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:07:18.063353 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-618d5675-f694-42b8-963b-61999a579295 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:07:18.090398 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:07:18.096812 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:07:18.148670 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6945e5fb-e8f9-4eea-9304-c03063aacdad None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:07:18.178733 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 90/187] 10.222.0.18 () {58 vars in 1282 bytes} [Tue Sep 13 03:07:17 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 351 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 13 03:07:18.178733 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-618d5675-f694-42b8-963b-61999a579295 None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:07:18.187858 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9507d4bd-7aa7-4279-b3f0-b8f7d63c0963 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:07:18.188489 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9507d4bd-7aa7-4279-b3f0-b8f7d63c0963 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:07:18.188759 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9507d4bd-7aa7-4279-b3f0-b8f7d63c0963 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:07:18.189023 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9507d4bd-7aa7-4279-b3f0-b8f7d63c0963 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:07:18.199100 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9507d4bd-7aa7-4279-b3f0-b8f7d63c0963 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 03:07:18.203921 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application [None req-9507d4bd-7aa7-4279-b3f0-b8f7d63c0963 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 13 03:07:18.203921 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 03:07:18.203921 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 03:07:18.203921 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 03:07:18.203921 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 03:07:18.203921 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 03:07:18.203921 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 03:07:18.203921 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 03:07:18.203921 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 03:07:18.203921 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 03:07:18.203921 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 03:07:18.203921 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 03:07:18.203921 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 03:07:18.203921 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 03:07:18.203921 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 03:07:18.203921 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 03:07:18.204921 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 03:07:18.204921 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 03:07:18.204921 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 03:07:18.204921 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 03:07:18.204921 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 03:07:18.204921 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 03:07:18.204921 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 03:07:18.204921 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 03:07:18.204921 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 03:07:18.204921 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 03:07:18.204921 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 03:07:18.204921 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application with Lock( Sep 13 03:07:18.204921 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 03:07:18.204921 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._enter() Sep 13 03:07:18.204921 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 03:07:18.204921 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 03:07:18.205715 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 03:07:18.205715 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.creator() Sep 13 03:07:18.205715 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 03:07:18.205715 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application created_value = creator( Sep 13 03:07:18.205715 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 03:07:18.205715 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 03:07:18.205715 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 03:07:18.205715 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 03:07:18.205715 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 03:07:18.205715 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 03:07:18.205715 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 13 03:07:18.205715 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Sep 13 03:07:18.205715 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 98/188] 10.222.0.18 () {56 vars in 1114 bytes} [Tue Sep 13 03:07:18 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 03:07:18.205715 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9507d4bd-7aa7-4279-b3f0-b8f7d63c0963 None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:07:18.276304 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-618d5675-f694-42b8-963b-61999a579295 None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:07:18.302143 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9507d4bd-7aa7-4279-b3f0-b8f7d63c0963 None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:07:18.303650 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:07:18.305480 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:07:18.311051 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c04f277-0f98-4de2-ad77-3173608b01e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:07:18.311827 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c04f277-0f98-4de2-ad77-3173608b01e9 None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:07:18.312272 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c04f277-0f98-4de2-ad77-3173608b01e9 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:07:18.312539 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c04f277-0f98-4de2-ad77-3173608b01e9 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:07:18.322961 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c04f277-0f98-4de2-ad77-3173608b01e9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 03:07:18.327993 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application [None req-9c04f277-0f98-4de2-ad77-3173608b01e9 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 13 03:07:18.327993 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 03:07:18.327993 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 03:07:18.327993 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 03:07:18.327993 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 03:07:18.327993 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 03:07:18.327993 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 03:07:18.327993 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 03:07:18.327993 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 03:07:18.327993 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 03:07:18.327993 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 03:07:18.327993 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 03:07:18.327993 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 03:07:18.327993 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 03:07:18.327993 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 03:07:18.327993 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 03:07:18.328887 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 03:07:18.328887 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 03:07:18.328887 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 03:07:18.328887 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 03:07:18.328887 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 03:07:18.328887 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 03:07:18.328887 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 03:07:18.328887 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 03:07:18.328887 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 03:07:18.328887 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 03:07:18.328887 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 03:07:18.328887 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application with Lock( Sep 13 03:07:18.328887 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 03:07:18.328887 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._enter() Sep 13 03:07:18.328887 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 03:07:18.328887 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 03:07:18.329936 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 03:07:18.329936 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.creator() Sep 13 03:07:18.329936 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 03:07:18.329936 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application created_value = creator( Sep 13 03:07:18.329936 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 03:07:18.329936 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 03:07:18.329936 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 03:07:18.329936 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 03:07:18.329936 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 03:07:18.329936 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 03:07:18.329936 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 13 03:07:18.329936 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Sep 13 03:07:18.329936 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 91/189] 10.222.0.18 () {56 vars in 1099 bytes} [Tue Sep 13 03:07:18 2022] GET /identity/v3/roles/reader => generated 84 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 03:07:18.329936 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c04f277-0f98-4de2-ad77-3173608b01e9 None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:07:18.338584 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c402d99-75ff-40bb-84a4-360e949f6966 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:07:18.339208 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c402d99-75ff-40bb-84a4-360e949f6966 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:07:18.339482 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c402d99-75ff-40bb-84a4-360e949f6966 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:07:18.339747 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c402d99-75ff-40bb-84a4-360e949f6966 None admin] PATH_INFO: `/v3/roles` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:07:18.340412 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c402d99-75ff-40bb-84a4-360e949f6966 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 03:07:18.340707 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c402d99-75ff-40bb-84a4-360e949f6966 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:07:18.342088 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c402d99-75ff-40bb-84a4-360e949f6966 None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:07:18.351507 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 99/190] 10.222.0.18 () {56 vars in 1099 bytes} [Tue Sep 13 03:07:18 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 03:07:18.351507 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c402d99-75ff-40bb-84a4-360e949f6966 None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:07:18.434738 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c04f277-0f98-4de2-ad77-3173608b01e9 None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:07:18.452973 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c402d99-75ff-40bb-84a4-360e949f6966 None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:07:18.467350 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93dba64e-c623-4dea-a570-a2fed00ed499 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:07:18.467966 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93dba64e-c623-4dea-a570-a2fed00ed499 None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:07:18.468238 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93dba64e-c623-4dea-a570-a2fed00ed499 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:07:18.468506 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93dba64e-c623-4dea-a570-a2fed00ed499 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:07:18.469590 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93dba64e-c623-4dea-a570-a2fed00ed499 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:07:18.471578 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93dba64e-c623-4dea-a570-a2fed00ed499 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:07:18.473129 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 92/191] 10.222.0.18 () {56 vars in 1108 bytes} [Tue Sep 13 03:07:18 2022] GET /identity/v3/domains/default => generated 199 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 03:07:18.474745 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93dba64e-c623-4dea-a570-a2fed00ed499 None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:07:18.486742 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67d3e51f-5704-4ba5-9a1f-24abc0139a5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:07:18.487374 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d3e51f-5704-4ba5-9a1f-24abc0139a5b None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:07:18.487649 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d3e51f-5704-4ba5-9a1f-24abc0139a5b None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:07:18.487918 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d3e51f-5704-4ba5-9a1f-24abc0139a5b None admin] PATH_INFO: `/v3/projects/d730110113be4b61aed1777ca25db340` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:07:18.489026 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67d3e51f-5704-4ba5-9a1f-24abc0139a5b None admin] RBAC: Authorizing `identity:get_project(project_id=d730110113be4b61aed1777ca25db340)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:07:18.490459 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67d3e51f-5704-4ba5-9a1f-24abc0139a5b None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:07:18.492243 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 100/192] 10.222.0.18 () {56 vars in 1186 bytes} [Tue Sep 13 03:07:18 2022] GET /identity/v3/projects/d730110113be4b61aed1777ca25db340 => generated 312 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 03:07:18.492924 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67d3e51f-5704-4ba5-9a1f-24abc0139a5b None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:07:18.523484 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:07:18.536947 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:07:18.578097 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93dba64e-c623-4dea-a570-a2fed00ed499 None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:07:18.590861 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67d3e51f-5704-4ba5-9a1f-24abc0139a5b None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:07:18.611040 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eafb42d-6b0e-4e53-85e6-34a8d565de33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:07:18.611636 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eafb42d-6b0e-4e53-85e6-34a8d565de33 None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:07:18.611908 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eafb42d-6b0e-4e53-85e6-34a8d565de33 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:07:18.612174 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eafb42d-6b0e-4e53-85e6-34a8d565de33 None admin] PATH_INFO: `/v3/roles` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:07:18.612819 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eafb42d-6b0e-4e53-85e6-34a8d565de33 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 03:07:18.613118 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eafb42d-6b0e-4e53-85e6-34a8d565de33 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:07:18.614482 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eafb42d-6b0e-4e53-85e6-34a8d565de33 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:07:18.622046 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 93/193] 10.222.0.18 () {56 vars in 1111 bytes} [Tue Sep 13 03:07:18 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 03:07:18.627663 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eafb42d-6b0e-4e53-85e6-34a8d565de33 None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:07:18.627771 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a36d712-be66-4961-af86-eac227a89a10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:07:18.627771 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a36d712-be66-4961-af86-eac227a89a10 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:07:18.627771 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a36d712-be66-4961-af86-eac227a89a10 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:07:18.627771 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a36d712-be66-4961-af86-eac227a89a10 None admin] PATH_INFO: `/v3/roles` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:07:18.628102 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a36d712-be66-4961-af86-eac227a89a10 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 03:07:18.628102 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a36d712-be66-4961-af86-eac227a89a10 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:07:18.628102 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a36d712-be66-4961-af86-eac227a89a10 None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:07:18.635083 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 101/194] 10.222.0.18 () {56 vars in 1101 bytes} [Tue Sep 13 03:07:18 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 03:07:18.635829 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a36d712-be66-4961-af86-eac227a89a10 None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:07:18.727413 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eafb42d-6b0e-4e53-85e6-34a8d565de33 None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:07:18.729287 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:07:18.736845 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a36d712-be66-4961-af86-eac227a89a10 None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:07:18.747750 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:07:18.762660 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-960ba551-6675-489a-9535-ff2e79d7e3bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:07:18.763307 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-960ba551-6675-489a-9535-ff2e79d7e3bb None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:07:18.763592 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-960ba551-6675-489a-9535-ff2e79d7e3bb None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:07:18.763865 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-960ba551-6675-489a-9535-ff2e79d7e3bb None admin] PATH_INFO: `/v3/users/86fc8d5c47e944ec8f7e33a2d6c291da` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:07:18.769944 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3287c299-1f68-4577-86da-d57fa79f7e92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:07:18.770536 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3287c299-1f68-4577-86da-d57fa79f7e92 None admin] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:07:18.770841 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3287c299-1f68-4577-86da-d57fa79f7e92 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:07:18.771111 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3287c299-1f68-4577-86da-d57fa79f7e92 None admin] PATH_INFO: `/v3/users` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:07:18.771754 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3287c299-1f68-4577-86da-d57fa79f7e92 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:07:18.773198 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3287c299-1f68-4577-86da-d57fa79f7e92 None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:07:18.777054 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-960ba551-6675-489a-9535-ff2e79d7e3bb None admin] RBAC: Authorizing `identity:get_user(user_id=86fc8d5c47e944ec8f7e33a2d6c291da)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:07:18.778494 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-960ba551-6675-489a-9535-ff2e79d7e3bb None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:07:18.780418 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 94/195] 10.222.0.18 () {56 vars in 1177 bytes} [Tue Sep 13 03:07:18 2022] GET /identity/v3/users/86fc8d5c47e944ec8f7e33a2d6c291da => generated 283 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 03:07:18.781170 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-960ba551-6675-489a-9535-ff2e79d7e3bb None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:07:18.793724 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 102/196] 10.222.0.18 () {60 vars in 1132 bytes} [Tue Sep 13 03:07:18 2022] POST /identity/v3/users => generated 297 bytes in 158 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 03:07:18.794415 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3287c299-1f68-4577-86da-d57fa79f7e92 None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:07:18.895899 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-960ba551-6675-489a-9535-ff2e79d7e3bb None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:07:18.926253 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3287c299-1f68-4577-86da-d57fa79f7e92 None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:07:18.936340 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48442999-32c2-4deb-a03e-4db270300d51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:07:18.936967 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48442999-32c2-4deb-a03e-4db270300d51 None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:07:18.937247 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48442999-32c2-4deb-a03e-4db270300d51 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:07:18.937520 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48442999-32c2-4deb-a03e-4db270300d51 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:07:18.938264 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48442999-32c2-4deb-a03e-4db270300d51 None admin] RBAC: Adding query filter params (role.id=1eb7a5b5203945c29e6d22d653f47847, scope.project.id=d730110113be4b61aed1777ca25db340, user.id=06632bd878b1474f8d5432f99e899155) {{(pid=81399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 03:07:18.938632 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48442999-32c2-4deb-a03e-4db270300d51 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:07:18.940194 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48442999-32c2-4deb-a03e-4db270300d51 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:07:18.947386 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 95/197] 10.222.0.18 () {56 vars in 1374 bytes} [Tue Sep 13 03:07:18 2022] GET /identity/v3/role_assignments?user.id=06632bd878b1474f8d5432f99e899155&scope.project.id=d730110113be4b61aed1777ca25db340&role.id=1eb7a5b5203945c29e6d22d653f47847 => generated 608 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 03:07:18.948112 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48442999-32c2-4deb-a03e-4db270300d51 None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:07:18.980819 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0666b43-58df-4e71-8f0e-921ae54e519f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:07:18.981490 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0666b43-58df-4e71-8f0e-921ae54e519f None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:07:18.981762 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0666b43-58df-4e71-8f0e-921ae54e519f None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:07:18.982029 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0666b43-58df-4e71-8f0e-921ae54e519f None admin] PATH_INFO: `/v3/users/86fc8d5c47e944ec8f7e33a2d6c291da` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:07:18.983237 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0666b43-58df-4e71-8f0e-921ae54e519f None admin] RBAC: Authorizing `identity:get_user(user_id=86fc8d5c47e944ec8f7e33a2d6c291da)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:07:18.984737 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0666b43-58df-4e71-8f0e-921ae54e519f None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:07:18.990658 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 103/198] 10.222.0.18 () {56 vars in 1177 bytes} [Tue Sep 13 03:07:18 2022] GET /identity/v3/users/86fc8d5c47e944ec8f7e33a2d6c291da => generated 283 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 03:07:18.990658 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0666b43-58df-4e71-8f0e-921ae54e519f None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:07:19.014289 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:07:19.016809 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:07:19.106051 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48442999-32c2-4deb-a03e-4db270300d51 None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:07:19.131809 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0666b43-58df-4e71-8f0e-921ae54e519f None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:07:19.158142 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-361206fa-d9bd-41ea-a1b1-a7f792fa35b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:07:19.159126 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-361206fa-d9bd-41ea-a1b1-a7f792fa35b3 None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:07:19.159640 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-361206fa-d9bd-41ea-a1b1-a7f792fa35b3 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:07:19.160082 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-361206fa-d9bd-41ea-a1b1-a7f792fa35b3 None admin] PATH_INFO: `/v3/users/cfb765821b764d438d4fcad2b5d6007e` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:07:19.162459 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-361206fa-d9bd-41ea-a1b1-a7f792fa35b3 None admin] RBAC: Authorizing `identity:get_user(user_id=cfb765821b764d438d4fcad2b5d6007e)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:07:19.164256 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-361206fa-d9bd-41ea-a1b1-a7f792fa35b3 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:07:19.166070 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 96/199] 10.222.0.18 () {56 vars in 1177 bytes} [Tue Sep 13 03:07:18 2022] GET /identity/v3/users/cfb765821b764d438d4fcad2b5d6007e => generated 289 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 03:07:19.167040 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-361206fa-d9bd-41ea-a1b1-a7f792fa35b3 None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:07:19.177663 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17396e58-20b6-4726-9279-45834a0a1074 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:07:19.178099 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17396e58-20b6-4726-9279-45834a0a1074 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:07:19.178371 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17396e58-20b6-4726-9279-45834a0a1074 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:07:19.178907 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17396e58-20b6-4726-9279-45834a0a1074 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:07:19.270282 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17396e58-20b6-4726-9279-45834a0a1074 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:07:19.271795 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17396e58-20b6-4726-9279-45834a0a1074 None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:07:19.291425 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:07:19.305620 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-361206fa-d9bd-41ea-a1b1-a7f792fa35b3 None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:07:19.319429 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:07:19.352708 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07f6dbe8-eec6-470b-80a6-7cb95f5ad596 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:07:19.354163 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07f6dbe8-eec6-470b-80a6-7cb95f5ad596 None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:07:19.354636 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07f6dbe8-eec6-470b-80a6-7cb95f5ad596 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:07:19.355106 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07f6dbe8-eec6-470b-80a6-7cb95f5ad596 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:07:19.382841 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 104/200] 10.222.0.18 () {58 vars in 1282 bytes} [Tue Sep 13 03:07:18 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 395 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 13 03:07:19.388471 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17396e58-20b6-4726-9279-45834a0a1074 None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:07:19.427560 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07f6dbe8-eec6-470b-80a6-7cb95f5ad596 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:07:19.429305 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07f6dbe8-eec6-470b-80a6-7cb95f5ad596 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:07:19.496430 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17396e58-20b6-4726-9279-45834a0a1074 None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:07:19.514969 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:07:19.537518 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2d130a3-8a11-4b5c-bdcc-ee6f1ef94bc3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:07:19.538126 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d130a3-8a11-4b5c-bdcc-ee6f1ef94bc3 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:07:19.538404 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d130a3-8a11-4b5c-bdcc-ee6f1ef94bc3 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:07:19.538699 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d130a3-8a11-4b5c-bdcc-ee6f1ef94bc3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:07:19.545450 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 97/201] 10.222.0.18 () {58 vars in 1282 bytes} [Tue Sep 13 03:07:19 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 379 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 13 03:07:19.546286 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07f6dbe8-eec6-470b-80a6-7cb95f5ad596 None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:07:19.558861 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:07:19.629674 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2d130a3-8a11-4b5c-bdcc-ee6f1ef94bc3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:07:19.631166 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2d130a3-8a11-4b5c-bdcc-ee6f1ef94bc3 None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:07:19.666436 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07f6dbe8-eec6-470b-80a6-7cb95f5ad596 None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:07:19.714149 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f3747aa-02aa-43fe-89d2-2ff6c6219b5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:07:19.715437 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f3747aa-02aa-43fe-89d2-2ff6c6219b5b None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:07:19.716180 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f3747aa-02aa-43fe-89d2-2ff6c6219b5b None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:07:19.716671 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f3747aa-02aa-43fe-89d2-2ff6c6219b5b None admin] PATH_INFO: `/v3/projects/54999ea237f64c5f972ef79e765facd0` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:07:19.718386 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f3747aa-02aa-43fe-89d2-2ff6c6219b5b None admin] RBAC: Authorizing `identity:get_project(project_id=54999ea237f64c5f972ef79e765facd0)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:07:19.720614 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f3747aa-02aa-43fe-89d2-2ff6c6219b5b None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:07:19.723039 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 98/202] 10.222.0.18 () {56 vars in 1186 bytes} [Tue Sep 13 03:07:19 2022] GET /identity/v3/projects/54999ea237f64c5f972ef79e765facd0 => generated 302 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 03:07:19.724388 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f3747aa-02aa-43fe-89d2-2ff6c6219b5b None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:07:19.737506 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 105/203] 10.222.0.18 () {58 vars in 1282 bytes} [Tue Sep 13 03:07:19 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 354 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 13 03:07:19.742047 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2d130a3-8a11-4b5c-bdcc-ee6f1ef94bc3 None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:07:19.766417 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:07:19.822471 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:07:19.848296 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f3747aa-02aa-43fe-89d2-2ff6c6219b5b None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:07:19.885012 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e586042-e9db-4eed-8ac4-a065f8432ee9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:07:19.885517 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2d130a3-8a11-4b5c-bdcc-ee6f1ef94bc3 None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:07:19.885633 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e586042-e9db-4eed-8ac4-a065f8432ee9 None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:07:19.885848 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e586042-e9db-4eed-8ac4-a065f8432ee9 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:07:19.886113 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e586042-e9db-4eed-8ac4-a065f8432ee9 None admin] PATH_INFO: `/v3/projects/54999ea237f64c5f972ef79e765facd0` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:07:19.887307 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e586042-e9db-4eed-8ac4-a065f8432ee9 None admin] RBAC: Authorizing `identity:get_project(project_id=54999ea237f64c5f972ef79e765facd0)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:07:19.889455 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e586042-e9db-4eed-8ac4-a065f8432ee9 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:07:19.890690 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 99/204] 10.222.0.18 () {56 vars in 1186 bytes} [Tue Sep 13 03:07:19 2022] GET /identity/v3/projects/54999ea237f64c5f972ef79e765facd0 => generated 302 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 03:07:19.891538 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e586042-e9db-4eed-8ac4-a065f8432ee9 None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:07:19.928356 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-557f8df6-961b-489f-9571-4e84c75b2e3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:07:19.928971 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-557f8df6-961b-489f-9571-4e84c75b2e3f None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:07:19.929243 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-557f8df6-961b-489f-9571-4e84c75b2e3f None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:07:19.929510 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-557f8df6-961b-489f-9571-4e84c75b2e3f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:07:19.930500 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-557f8df6-961b-489f-9571-4e84c75b2e3f None admin] RBAC: Adding query filter params (role.id=425d4b481c7846e0bc07e7e805d5b15f, scope.project.id=54999ea237f64c5f972ef79e765facd0, user.id=86fc8d5c47e944ec8f7e33a2d6c291da) {{(pid=81400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 03:07:19.930968 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-557f8df6-961b-489f-9571-4e84c75b2e3f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:07:19.932463 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-557f8df6-961b-489f-9571-4e84c75b2e3f None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:07:19.939818 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 106/205] 10.222.0.18 () {56 vars in 1374 bytes} [Tue Sep 13 03:07:19 2022] GET /identity/v3/role_assignments?user.id=86fc8d5c47e944ec8f7e33a2d6c291da&scope.project.id=54999ea237f64c5f972ef79e765facd0&role.id=425d4b481c7846e0bc07e7e805d5b15f => generated 259 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 03:07:19.940503 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-557f8df6-961b-489f-9571-4e84c75b2e3f None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:07:20.022295 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e586042-e9db-4eed-8ac4-a065f8432ee9 None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:07:20.068906 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc57f367-9854-459a-aca2-b8f207e4bc02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:07:20.070471 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc57f367-9854-459a-aca2-b8f207e4bc02 None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:07:20.072022 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc57f367-9854-459a-aca2-b8f207e4bc02 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:07:20.072581 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc57f367-9854-459a-aca2-b8f207e4bc02 None admin] PATH_INFO: `/v3/projects/83aeaa07fb5d40d3b285aeb01d1252b1` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:07:20.082639 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc57f367-9854-459a-aca2-b8f207e4bc02 None admin] RBAC: Authorizing `identity:get_project(project_id=83aeaa07fb5d40d3b285aeb01d1252b1)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:07:20.082639 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc57f367-9854-459a-aca2-b8f207e4bc02 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:07:20.084305 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 100/206] 10.222.0.18 () {56 vars in 1186 bytes} [Tue Sep 13 03:07:19 2022] GET /identity/v3/projects/83aeaa07fb5d40d3b285aeb01d1252b1 => generated 298 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 03:07:20.090348 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc57f367-9854-459a-aca2-b8f207e4bc02 None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:07:20.128683 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:07:20.132838 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:07:20.169863 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-557f8df6-961b-489f-9571-4e84c75b2e3f None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:07:20.223424 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d264d53-3d15-4dbf-9ccd-a3a11d1b5e3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:07:20.224539 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d264d53-3d15-4dbf-9ccd-a3a11d1b5e3a None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:07:20.225130 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d264d53-3d15-4dbf-9ccd-a3a11d1b5e3a None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:07:20.225627 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d264d53-3d15-4dbf-9ccd-a3a11d1b5e3a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:07:20.226542 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d264d53-3d15-4dbf-9ccd-a3a11d1b5e3a None admin] RBAC: Adding query filter params (role.id=a9aa35ebda4d4aff80e62f750e9104ab, scope.project.id=54999ea237f64c5f972ef79e765facd0, user.id=86fc8d5c47e944ec8f7e33a2d6c291da) {{(pid=81400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 03:07:20.227088 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d264d53-3d15-4dbf-9ccd-a3a11d1b5e3a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:07:20.228799 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d264d53-3d15-4dbf-9ccd-a3a11d1b5e3a None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:07:20.236601 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 107/207] 10.222.0.18 () {56 vars in 1374 bytes} [Tue Sep 13 03:07:19 2022] GET /identity/v3/role_assignments?user.id=86fc8d5c47e944ec8f7e33a2d6c291da&scope.project.id=54999ea237f64c5f972ef79e765facd0&role.id=a9aa35ebda4d4aff80e62f750e9104ab => generated 259 bytes in 296 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 03:07:20.309542 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc57f367-9854-459a-aca2-b8f207e4bc02 None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:07:20.346147 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a77a630b-18f4-4b05-9848-06ab696f4f03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:07:20.346793 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a77a630b-18f4-4b05-9848-06ab696f4f03 None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:07:20.347121 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a77a630b-18f4-4b05-9848-06ab696f4f03 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:07:20.347399 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a77a630b-18f4-4b05-9848-06ab696f4f03 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:07:20.348142 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a77a630b-18f4-4b05-9848-06ab696f4f03 None admin] RBAC: Adding query filter params (role.id=502e6c11e747432988d7f723cee0af64, scope.project.id=83aeaa07fb5d40d3b285aeb01d1252b1, user.id=cfb765821b764d438d4fcad2b5d6007e) {{(pid=81399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 03:07:20.348451 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a77a630b-18f4-4b05-9848-06ab696f4f03 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:07:20.349932 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a77a630b-18f4-4b05-9848-06ab696f4f03 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:07:20.357043 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 101/208] 10.222.0.18 () {56 vars in 1374 bytes} [Tue Sep 13 03:07:20 2022] GET /identity/v3/role_assignments?user.id=cfb765821b764d438d4fcad2b5d6007e&scope.project.id=83aeaa07fb5d40d3b285aeb01d1252b1&role.id=502e6c11e747432988d7f723cee0af64 => generated 608 bytes in 267 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 03:07:21.279543 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4c897a8-c790-4eeb-ac10-3724d966202f None None] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:07:21.280554 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4c897a8-c790-4eeb-ac10-3724d966202f None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:07:21.281049 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4c897a8-c790-4eeb-ac10-3724d966202f None None] PATH_INFO: `/` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:07:21.282773 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 108/209] 10.222.0.18 () {54 vars in 927 bytes} [Tue Sep 13 03:07:21 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 03:07:21.288850 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b26e01a-7075-4190-b548-b4ec228427de None None] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:07:21.289352 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b26e01a-7075-4190-b548-b4ec228427de None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:07:21.289424 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b26e01a-7075-4190-b548-b4ec228427de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:07:21.308974 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.auth.core [None req-0b26e01a-7075-4190-b548-b4ec228427de None None] MFA Rules not processed for user `49f280ab086b4610855c4ffa378517d9`. Rule list: `[]` (Enabled: `True`). {{(pid=81399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 03:07:21.310193 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-0b26e01a-7075-4190-b548-b4ec228427de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:07:21.382402 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 102/210] 10.222.0.18 () {58 vars in 1024 bytes} [Tue Sep 13 03:07:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 13 03:07:21.393142 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-691485dc-bf43-4ee0-94de-00e1d769e225 None None] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:07:21.393429 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-691485dc-bf43-4ee0-94de-00e1d769e225 None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:07:21.393703 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-691485dc-bf43-4ee0-94de-00e1d769e225 None None] PATH_INFO: `/` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:07:21.396344 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 109/211] 10.222.0.18 () {54 vars in 927 bytes} [Tue Sep 13 03:07:21 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 03:07:21.400536 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fdcb8e5-89b0-4464-a212-0b4e96f96303 None None] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:07:21.400825 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fdcb8e5-89b0-4464-a212-0b4e96f96303 None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:07:21.401096 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fdcb8e5-89b0-4464-a212-0b4e96f96303 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:07:21.419205 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.auth.core [None req-5fdcb8e5-89b0-4464-a212-0b4e96f96303 None None] MFA Rules not processed for user `49f280ab086b4610855c4ffa378517d9`. Rule list: `[]` (Enabled: `True`). {{(pid=81399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 03:07:21.420384 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-5fdcb8e5-89b0-4464-a212-0b4e96f96303 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:07:21.422343 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:07:21.488008 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 103/212] 10.222.0.18 () {58 vars in 1024 bytes} [Tue Sep 13 03:07:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 13 03:07:21.661421 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-691485dc-bf43-4ee0-94de-00e1d769e225 None None] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:07:21.765187 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fdcb8e5-89b0-4464-a212-0b4e96f96303 None None] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:07:21.768505 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-691485dc-bf43-4ee0-94de-00e1d769e225 None None] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:07:21.804752 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ed4399c-0293-4559-86a5-b8fcf46c73a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:07:21.805379 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ed4399c-0293-4559-86a5-b8fcf46c73a6 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:07:21.805651 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ed4399c-0293-4559-86a5-b8fcf46c73a6 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:07:21.806171 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ed4399c-0293-4559-86a5-b8fcf46c73a6 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:07:21.817818 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ed4399c-0293-4559-86a5-b8fcf46c73a6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 03:07:21.823642 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application [None req-2ed4399c-0293-4559-86a5-b8fcf46c73a6 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 13 03:07:21.823642 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 03:07:21.823642 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 03:07:21.823642 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 03:07:21.823642 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 03:07:21.823642 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 03:07:21.823642 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 03:07:21.823642 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 03:07:21.823642 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 03:07:21.823642 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 03:07:21.823642 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 03:07:21.823642 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 03:07:21.823642 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 03:07:21.823642 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 03:07:21.823642 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 03:07:21.823642 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 03:07:21.824554 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 03:07:21.824554 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 03:07:21.824554 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 03:07:21.824554 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 03:07:21.824554 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 03:07:21.824554 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 03:07:21.824554 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 03:07:21.824554 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 03:07:21.824554 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 03:07:21.824554 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 03:07:21.824554 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 03:07:21.824554 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application with Lock( Sep 13 03:07:21.824554 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 03:07:21.824554 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._enter() Sep 13 03:07:21.824554 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 03:07:21.824554 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 03:07:21.825776 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 03:07:21.825776 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.creator() Sep 13 03:07:21.825776 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 03:07:21.825776 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application created_value = creator( Sep 13 03:07:21.825776 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 03:07:21.825776 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 03:07:21.825776 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 03:07:21.825776 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 03:07:21.825776 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 03:07:21.825776 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 03:07:21.825776 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 13 03:07:21.825776 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Sep 13 03:07:21.825776 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 110/213] 10.222.0.18 () {56 vars in 1099 bytes} [Tue Sep 13 03:07:21 2022] GET /identity/v3/roles/member => generated 84 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 03:07:21.830839 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ed4399c-0293-4559-86a5-b8fcf46c73a6 None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:07:21.840610 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:07:21.890736 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fdcb8e5-89b0-4464-a212-0b4e96f96303 None None] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:07:21.937537 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10e109dd-da51-4c87-b712-5eb91434e1eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:07:21.939122 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10e109dd-da51-4c87-b712-5eb91434e1eb None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:07:21.939756 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10e109dd-da51-4c87-b712-5eb91434e1eb None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:07:21.940228 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10e109dd-da51-4c87-b712-5eb91434e1eb None admin] PATH_INFO: `/v3/domains/default` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:07:21.942426 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10e109dd-da51-4c87-b712-5eb91434e1eb None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:07:21.944124 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10e109dd-da51-4c87-b712-5eb91434e1eb None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:07:21.944231 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ed4399c-0293-4559-86a5-b8fcf46c73a6 None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:07:21.947277 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 104/214] 10.222.0.18 () {56 vars in 1108 bytes} [Tue Sep 13 03:07:21 2022] GET /identity/v3/domains/default => generated 199 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 03:07:21.953520 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10e109dd-da51-4c87-b712-5eb91434e1eb None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:07:21.964860 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:07:21.983029 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99bc993f-7916-4570-a13d-500a8fa0f922 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:07:21.983737 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99bc993f-7916-4570-a13d-500a8fa0f922 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:07:21.984016 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99bc993f-7916-4570-a13d-500a8fa0f922 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:07:21.984285 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99bc993f-7916-4570-a13d-500a8fa0f922 None admin] PATH_INFO: `/v3/roles` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:07:21.984938 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99bc993f-7916-4570-a13d-500a8fa0f922 None admin] RBAC: Adding query filter params (name=member) {{(pid=81400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 03:07:21.985259 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99bc993f-7916-4570-a13d-500a8fa0f922 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:07:21.987229 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99bc993f-7916-4570-a13d-500a8fa0f922 None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:07:21.996219 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 111/215] 10.222.0.18 () {56 vars in 1101 bytes} [Tue Sep 13 03:07:21 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 03:07:22.001790 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99bc993f-7916-4570-a13d-500a8fa0f922 None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:07:22.091004 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:07:22.093869 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10e109dd-da51-4c87-b712-5eb91434e1eb None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:07:22.127893 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99bc993f-7916-4570-a13d-500a8fa0f922 None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:07:22.141809 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb386d2b-5285-4adb-a284-bf93f5b7f45a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:07:22.142824 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb386d2b-5285-4adb-a284-bf93f5b7f45a None admin] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:07:22.143518 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb386d2b-5285-4adb-a284-bf93f5b7f45a None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:07:22.143979 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb386d2b-5285-4adb-a284-bf93f5b7f45a None admin] PATH_INFO: `/v3/users` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:07:22.144813 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb386d2b-5285-4adb-a284-bf93f5b7f45a None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:07:22.147093 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb386d2b-5285-4adb-a284-bf93f5b7f45a None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:07:22.166426 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c71d8ba-c282-4115-b3bf-dd4329fa7641 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:07:22.167182 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c71d8ba-c282-4115-b3bf-dd4329fa7641 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:07:22.167461 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c71d8ba-c282-4115-b3bf-dd4329fa7641 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:07:22.167916 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c71d8ba-c282-4115-b3bf-dd4329fa7641 None admin] PATH_INFO: `/v3/users/cbbc542ba2bb42deb9b1ad183d118075` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:07:22.177352 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 105/216] 10.222.0.18 () {60 vars in 1132 bytes} [Tue Sep 13 03:07:21 2022] POST /identity/v3/users => generated 297 bytes in 224 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 13 03:07:22.193380 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c71d8ba-c282-4115-b3bf-dd4329fa7641 None admin] RBAC: Authorizing `identity:get_user(user_id=cbbc542ba2bb42deb9b1ad183d118075)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:07:22.195533 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c71d8ba-c282-4115-b3bf-dd4329fa7641 None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:07:22.197732 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 112/217] 10.222.0.18 () {56 vars in 1177 bytes} [Tue Sep 13 03:07:22 2022] GET /identity/v3/users/cbbc542ba2bb42deb9b1ad183d118075 => generated 297 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 03:07:22.204280 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb386d2b-5285-4adb-a284-bf93f5b7f45a None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:07:22.278876 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:07:22.315155 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e24c9056-1314-4987-a08e-c8defaa92073 None None] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:07:22.316453 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e24c9056-1314-4987-a08e-c8defaa92073 None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:07:22.317235 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e24c9056-1314-4987-a08e-c8defaa92073 None None] PATH_INFO: `/` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:07:22.319023 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 113/218] 10.222.0.18 () {54 vars in 927 bytes} [Tue Sep 13 03:07:22 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 13 03:07:22.324838 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d153b88e-098e-4319-839d-65574243add3 None None] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:07:22.325155 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d153b88e-098e-4319-839d-65574243add3 None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:07:22.325377 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d153b88e-098e-4319-839d-65574243add3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:07:22.331466 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb386d2b-5285-4adb-a284-bf93f5b7f45a None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:07:22.347388 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.auth.core [None req-d153b88e-098e-4319-839d-65574243add3 None None] MFA Rules not processed for user `49f280ab086b4610855c4ffa378517d9`. Rule list: `[]` (Enabled: `True`). {{(pid=81400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 03:07:22.348584 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-d153b88e-098e-4319-839d-65574243add3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:07:22.379735 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ede87c28-e21c-447a-bc7f-a53dfc9fbcf7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:07:22.380464 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ede87c28-e21c-447a-bc7f-a53dfc9fbcf7 None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:07:22.380740 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ede87c28-e21c-447a-bc7f-a53dfc9fbcf7 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:07:22.381117 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ede87c28-e21c-447a-bc7f-a53dfc9fbcf7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:07:22.438011 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 114/219] 10.222.0.18 () {58 vars in 1024 bytes} [Tue Sep 13 03:07:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 13 03:07:22.464439 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ede87c28-e21c-447a-bc7f-a53dfc9fbcf7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:07:22.465853 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ede87c28-e21c-447a-bc7f-a53dfc9fbcf7 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:07:22.532149 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:07:22.576382 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 106/220] 10.222.0.18 () {58 vars in 1282 bytes} [Tue Sep 13 03:07:22 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 373 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 13 03:07:22.582814 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d153b88e-098e-4319-839d-65574243add3 None None] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:07:22.645265 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:07:22.691933 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00123311-3e8c-48a7-94b9-46ed9f8401ef None None] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:07:22.692257 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00123311-3e8c-48a7-94b9-46ed9f8401ef None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:07:22.692492 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00123311-3e8c-48a7-94b9-46ed9f8401ef None None] PATH_INFO: `/` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:07:22.694038 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 107/221] 10.222.0.18 () {54 vars in 927 bytes} [Tue Sep 13 03:07:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 13 03:07:22.698231 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d153b88e-098e-4319-839d-65574243add3 None None] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:07:22.700238 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06fa35d5-82b5-47a0-894e-991403fbdf8d None None] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:07:22.700730 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06fa35d5-82b5-47a0-894e-991403fbdf8d None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:07:22.701216 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06fa35d5-82b5-47a0-894e-991403fbdf8d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:07:22.723609 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.auth.core [None req-06fa35d5-82b5-47a0-894e-991403fbdf8d None None] MFA Rules not processed for user `49f280ab086b4610855c4ffa378517d9`. Rule list: `[]` (Enabled: `True`). {{(pid=81399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 03:07:22.724812 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-06fa35d5-82b5-47a0-894e-991403fbdf8d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:07:22.744636 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cab8bb2-9c92-421d-9e41-2674c8b60529 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:07:22.745528 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cab8bb2-9c92-421d-9e41-2674c8b60529 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:07:22.745811 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cab8bb2-9c92-421d-9e41-2674c8b60529 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:07:22.746473 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cab8bb2-9c92-421d-9e41-2674c8b60529 None admin] PATH_INFO: `/v3/projects/54999ea237f64c5f972ef79e765facd0` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:07:22.747649 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cab8bb2-9c92-421d-9e41-2674c8b60529 None admin] RBAC: Authorizing `identity:get_project(project_id=54999ea237f64c5f972ef79e765facd0)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:07:22.749643 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cab8bb2-9c92-421d-9e41-2674c8b60529 None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:07:22.752638 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 115/222] 10.222.0.18 () {56 vars in 1186 bytes} [Tue Sep 13 03:07:22 2022] GET /identity/v3/projects/54999ea237f64c5f972ef79e765facd0 => generated 302 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 03:07:22.753844 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cab8bb2-9c92-421d-9e41-2674c8b60529 None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:07:22.808298 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 108/223] 10.222.0.18 () {58 vars in 1024 bytes} [Tue Sep 13 03:07:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 13 03:07:22.809536 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06fa35d5-82b5-47a0-894e-991403fbdf8d None None] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:07:22.910424 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cab8bb2-9c92-421d-9e41-2674c8b60529 None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:07:22.935288 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:07:22.960091 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71c382ce-8cbf-453a-b688-bd9c843c719e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:07:22.960752 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c382ce-8cbf-453a-b688-bd9c843c719e None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:07:22.961115 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c382ce-8cbf-453a-b688-bd9c843c719e None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:07:22.961550 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c382ce-8cbf-453a-b688-bd9c843c719e None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:07:22.965909 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06fa35d5-82b5-47a0-894e-991403fbdf8d None None] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:07:22.974086 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:07:23.000327 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71c382ce-8cbf-453a-b688-bd9c843c719e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 03:07:23.003983 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d02682aa-fe78-4475-8417-a687e22ea7d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:07:23.004757 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d02682aa-fe78-4475-8417-a687e22ea7d5 None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:07:23.005179 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d02682aa-fe78-4475-8417-a687e22ea7d5 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:07:23.005616 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d02682aa-fe78-4475-8417-a687e22ea7d5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:07:23.006112 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application [None req-71c382ce-8cbf-453a-b688-bd9c843c719e None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 13 03:07:23.006112 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 03:07:23.006112 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 03:07:23.006112 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 03:07:23.006112 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 03:07:23.006112 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 03:07:23.006112 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 03:07:23.006112 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 03:07:23.006112 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 03:07:23.006112 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 03:07:23.006112 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 03:07:23.006112 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 03:07:23.006112 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 03:07:23.006112 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 03:07:23.006112 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 03:07:23.006112 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 03:07:23.007194 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 03:07:23.007194 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 03:07:23.007194 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 03:07:23.007194 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 03:07:23.007194 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 03:07:23.007194 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 03:07:23.007194 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 03:07:23.007194 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 03:07:23.007194 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 03:07:23.007194 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 03:07:23.007194 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 03:07:23.007194 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application with Lock( Sep 13 03:07:23.007194 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 03:07:23.007194 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._enter() Sep 13 03:07:23.007194 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 03:07:23.007194 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 03:07:23.008201 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 03:07:23.008201 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.creator() Sep 13 03:07:23.008201 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 03:07:23.008201 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application created_value = creator( Sep 13 03:07:23.008201 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 03:07:23.008201 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 03:07:23.008201 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 03:07:23.008201 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 03:07:23.008201 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 03:07:23.008201 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 03:07:23.008201 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 13 03:07:23.008201 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Sep 13 03:07:23.009392 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d02682aa-fe78-4475-8417-a687e22ea7d5 None admin] RBAC: Adding query filter params (role.id=1eb7a5b5203945c29e6d22d653f47847, scope.project.id=54999ea237f64c5f972ef79e765facd0, user.id=cbbc542ba2bb42deb9b1ad183d118075) {{(pid=81399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 03:07:23.009843 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d02682aa-fe78-4475-8417-a687e22ea7d5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:07:23.011500 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d02682aa-fe78-4475-8417-a687e22ea7d5 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:07:23.015517 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 116/224] 10.222.0.18 () {56 vars in 1096 bytes} [Tue Sep 13 03:07:22 2022] GET /identity/v3/roles/admin => generated 83 bytes in 262 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 03:07:23.018445 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 109/225] 10.222.0.18 () {56 vars in 1374 bytes} [Tue Sep 13 03:07:22 2022] GET /identity/v3/role_assignments?user.id=cbbc542ba2bb42deb9b1ad183d118075&scope.project.id=54999ea237f64c5f972ef79e765facd0&role.id=1eb7a5b5203945c29e6d22d653f47847 => generated 259 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 03:07:23.022237 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71c382ce-8cbf-453a-b688-bd9c843c719e None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:07:23.125090 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d02682aa-fe78-4475-8417-a687e22ea7d5 None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:07:23.197183 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71c382ce-8cbf-453a-b688-bd9c843c719e None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:07:23.233804 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d646f540-7804-4786-a7f2-dad9b2d81a22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:07:23.234341 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d646f540-7804-4786-a7f2-dad9b2d81a22 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:07:23.234641 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d646f540-7804-4786-a7f2-dad9b2d81a22 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:07:23.234932 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d646f540-7804-4786-a7f2-dad9b2d81a22 None admin] PATH_INFO: `/v3/roles` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:07:23.236158 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d646f540-7804-4786-a7f2-dad9b2d81a22 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 03:07:23.236463 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d646f540-7804-4786-a7f2-dad9b2d81a22 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:07:23.237863 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d646f540-7804-4786-a7f2-dad9b2d81a22 None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:07:23.252166 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 117/226] 10.222.0.18 () {56 vars in 1099 bytes} [Tue Sep 13 03:07:23 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 03:07:23.253589 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d646f540-7804-4786-a7f2-dad9b2d81a22 None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:07:23.266585 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d02682aa-fe78-4475-8417-a687e22ea7d5 None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:07:23.306027 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:07:23.315478 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97c2e541-a0d9-4bc7-9408-70889ff325ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:07:23.316120 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c2e541-a0d9-4bc7-9408-70889ff325ba None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:07:23.316398 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c2e541-a0d9-4bc7-9408-70889ff325ba None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:07:23.316664 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c2e541-a0d9-4bc7-9408-70889ff325ba None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:07:23.328072 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97c2e541-a0d9-4bc7-9408-70889ff325ba None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 03:07:23.333611 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application [None req-97c2e541-a0d9-4bc7-9408-70889ff325ba None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 13 03:07:23.333611 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 03:07:23.333611 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 03:07:23.333611 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 03:07:23.333611 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 03:07:23.333611 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 03:07:23.333611 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 03:07:23.333611 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 03:07:23.333611 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 03:07:23.333611 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 03:07:23.333611 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 03:07:23.333611 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 03:07:23.333611 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 03:07:23.333611 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 03:07:23.333611 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 03:07:23.333611 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 03:07:23.335988 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 03:07:23.335988 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 03:07:23.335988 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 03:07:23.335988 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 03:07:23.335988 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 03:07:23.335988 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 03:07:23.335988 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 03:07:23.335988 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 03:07:23.335988 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 03:07:23.335988 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 03:07:23.335988 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 03:07:23.335988 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application with Lock( Sep 13 03:07:23.335988 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 03:07:23.335988 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._enter() Sep 13 03:07:23.335988 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 03:07:23.335988 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 03:07:23.337013 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 03:07:23.337013 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.creator() Sep 13 03:07:23.337013 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 03:07:23.337013 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application created_value = creator( Sep 13 03:07:23.337013 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 03:07:23.337013 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 03:07:23.337013 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 03:07:23.337013 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 03:07:23.337013 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 03:07:23.337013 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 03:07:23.337013 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 13 03:07:23.337013 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Sep 13 03:07:23.337013 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 110/227] 10.222.0.18 () {56 vars in 1114 bytes} [Tue Sep 13 03:07:23 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 03:07:23.340721 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97c2e541-a0d9-4bc7-9408-70889ff325ba None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:07:23.362375 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:07:23.372548 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d646f540-7804-4786-a7f2-dad9b2d81a22 None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:07:23.423365 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6410069-facb-4c11-98fd-3bdf5576d591 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:07:23.424256 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6410069-facb-4c11-98fd-3bdf5576d591 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:07:23.424810 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6410069-facb-4c11-98fd-3bdf5576d591 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:07:23.425223 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6410069-facb-4c11-98fd-3bdf5576d591 None admin] PATH_INFO: `/v3/users/86fc8d5c47e944ec8f7e33a2d6c291da` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:07:23.428996 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6410069-facb-4c11-98fd-3bdf5576d591 None admin] RBAC: Authorizing `identity:get_user(user_id=86fc8d5c47e944ec8f7e33a2d6c291da)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:07:23.430987 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6410069-facb-4c11-98fd-3bdf5576d591 None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:07:23.434250 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 118/228] 10.222.0.18 () {56 vars in 1177 bytes} [Tue Sep 13 03:07:23 2022] GET /identity/v3/users/86fc8d5c47e944ec8f7e33a2d6c291da => generated 283 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 03:07:23.439172 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6410069-facb-4c11-98fd-3bdf5576d591 None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:07:23.450675 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97c2e541-a0d9-4bc7-9408-70889ff325ba None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:07:23.487654 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10f35269-2642-4d5e-881e-1462416fd53e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:07:23.488161 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f35269-2642-4d5e-881e-1462416fd53e None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:07:23.488439 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f35269-2642-4d5e-881e-1462416fd53e None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:07:23.488709 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f35269-2642-4d5e-881e-1462416fd53e None admin] PATH_INFO: `/v3/roles` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:07:23.489373 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10f35269-2642-4d5e-881e-1462416fd53e None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 03:07:23.489676 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10f35269-2642-4d5e-881e-1462416fd53e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:07:23.491206 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10f35269-2642-4d5e-881e-1462416fd53e None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:07:23.499664 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 111/229] 10.222.0.18 () {56 vars in 1111 bytes} [Tue Sep 13 03:07:23 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 03:07:23.507487 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10f35269-2642-4d5e-881e-1462416fd53e None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:07:23.607022 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:07:23.608795 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6410069-facb-4c11-98fd-3bdf5576d591 None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:07:23.625457 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:07:23.645496 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10f35269-2642-4d5e-881e-1462416fd53e None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:07:23.648797 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75b29ce0-f78d-4fc0-9bfe-4f98e4a2125a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:07:23.649742 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b29ce0-f78d-4fc0-9bfe-4f98e4a2125a None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:07:23.650161 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b29ce0-f78d-4fc0-9bfe-4f98e4a2125a None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:07:23.650585 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b29ce0-f78d-4fc0-9bfe-4f98e4a2125a None admin] PATH_INFO: `/v3/projects/54999ea237f64c5f972ef79e765facd0` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:07:23.651909 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75b29ce0-f78d-4fc0-9bfe-4f98e4a2125a None admin] RBAC: Authorizing `identity:get_project(project_id=54999ea237f64c5f972ef79e765facd0)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:07:23.653635 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75b29ce0-f78d-4fc0-9bfe-4f98e4a2125a None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:07:23.660790 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 119/230] 10.222.0.18 () {56 vars in 1186 bytes} [Tue Sep 13 03:07:23 2022] GET /identity/v3/projects/54999ea237f64c5f972ef79e765facd0 => generated 302 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 03:07:23.662296 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75b29ce0-f78d-4fc0-9bfe-4f98e4a2125a None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:07:23.693095 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1068836-e375-4dc3-aa5b-064ea420ca7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:07:23.693734 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1068836-e375-4dc3-aa5b-064ea420ca7c None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:07:23.694009 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1068836-e375-4dc3-aa5b-064ea420ca7c None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:07:23.694278 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1068836-e375-4dc3-aa5b-064ea420ca7c None admin] PATH_INFO: `/v3/users/86fc8d5c47e944ec8f7e33a2d6c291da` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:07:23.695465 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1068836-e375-4dc3-aa5b-064ea420ca7c None admin] RBAC: Authorizing `identity:get_user(user_id=86fc8d5c47e944ec8f7e33a2d6c291da)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:07:23.696900 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1068836-e375-4dc3-aa5b-064ea420ca7c None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:07:23.698555 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 112/231] 10.222.0.18 () {56 vars in 1177 bytes} [Tue Sep 13 03:07:23 2022] GET /identity/v3/users/86fc8d5c47e944ec8f7e33a2d6c291da => generated 283 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 03:07:23.704574 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1068836-e375-4dc3-aa5b-064ea420ca7c None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:07:23.807630 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1068836-e375-4dc3-aa5b-064ea420ca7c None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:07:23.869894 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75b29ce0-f78d-4fc0-9bfe-4f98e4a2125a None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:07:23.873259 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99942089-e6f1-427d-bdad-c65da4e4142a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:07:23.873908 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99942089-e6f1-427d-bdad-c65da4e4142a None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:07:23.874799 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99942089-e6f1-427d-bdad-c65da4e4142a None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:07:23.875083 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99942089-e6f1-427d-bdad-c65da4e4142a None admin] PATH_INFO: `/v3/projects/54999ea237f64c5f972ef79e765facd0` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:07:23.879812 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99942089-e6f1-427d-bdad-c65da4e4142a None admin] RBAC: Authorizing `identity:get_project(project_id=54999ea237f64c5f972ef79e765facd0)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:07:23.881284 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99942089-e6f1-427d-bdad-c65da4e4142a None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:07:23.885512 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 113/232] 10.222.0.18 () {56 vars in 1186 bytes} [Tue Sep 13 03:07:23 2022] GET /identity/v3/projects/54999ea237f64c5f972ef79e765facd0 => generated 302 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 03:07:23.891014 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99942089-e6f1-427d-bdad-c65da4e4142a None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:07:23.922976 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:07:23.940495 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c959fbd-625b-4b70-b3b1-ecc7f7ac8f3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:07:23.941327 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c959fbd-625b-4b70-b3b1-ecc7f7ac8f3f None admin] REQUEST_METHOD: `PUT` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:07:23.941609 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c959fbd-625b-4b70-b3b1-ecc7f7ac8f3f None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:07:23.941879 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c959fbd-625b-4b70-b3b1-ecc7f7ac8f3f None admin] PATH_INFO: `/v3/projects/54999ea237f64c5f972ef79e765facd0/users/86fc8d5c47e944ec8f7e33a2d6c291da/roles/425d4b481c7846e0bc07e7e805d5b15f` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:07:23.947180 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c959fbd-625b-4b70-b3b1-ecc7f7ac8f3f None admin] RBAC: Authorizing `identity:create_grant(project_id=54999ea237f64c5f972ef79e765facd0, user_id=86fc8d5c47e944ec8f7e33a2d6c291da, role_id=425d4b481c7846e0bc07e7e805d5b15f)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:07:23.949103 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c959fbd-625b-4b70-b3b1-ecc7f7ac8f3f None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:07:23.975698 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 120/233] 10.222.0.18 () {58 vars in 1439 bytes} [Tue Sep 13 03:07:23 2022] PUT /identity/v3/projects/54999ea237f64c5f972ef79e765facd0/users/86fc8d5c47e944ec8f7e33a2d6c291da/roles/425d4b481c7846e0bc07e7e805d5b15f => generated 0 bytes in 313 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 03:07:24.042627 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99942089-e6f1-427d-bdad-c65da4e4142a None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:07:24.101147 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe3c74d7-0e63-4d69-ba1e-31235f238eb7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:07:24.101834 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe3c74d7-0e63-4d69-ba1e-31235f238eb7 None admin] REQUEST_METHOD: `PUT` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:07:24.102117 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe3c74d7-0e63-4d69-ba1e-31235f238eb7 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:07:24.102400 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe3c74d7-0e63-4d69-ba1e-31235f238eb7 None admin] PATH_INFO: `/v3/projects/54999ea237f64c5f972ef79e765facd0/users/86fc8d5c47e944ec8f7e33a2d6c291da/roles/a9aa35ebda4d4aff80e62f750e9104ab` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:07:24.108163 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe3c74d7-0e63-4d69-ba1e-31235f238eb7 None admin] RBAC: Authorizing `identity:create_grant(project_id=54999ea237f64c5f972ef79e765facd0, user_id=86fc8d5c47e944ec8f7e33a2d6c291da, role_id=a9aa35ebda4d4aff80e62f750e9104ab)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:07:24.109654 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe3c74d7-0e63-4d69-ba1e-31235f238eb7 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:07:24.140725 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 114/234] 10.222.0.18 () {58 vars in 1439 bytes} [Tue Sep 13 03:07:23 2022] PUT /identity/v3/projects/54999ea237f64c5f972ef79e765facd0/users/86fc8d5c47e944ec8f7e33a2d6c291da/roles/a9aa35ebda4d4aff80e62f750e9104ab => generated 0 bytes in 250 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 03:07:25.029786 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6828da86-d2c7-4735-b834-1fa65850c4aa None None] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:07:25.030291 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6828da86-d2c7-4735-b834-1fa65850c4aa None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:07:25.030367 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6828da86-d2c7-4735-b834-1fa65850c4aa None None] PATH_INFO: `/` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:07:25.031970 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 121/235] 10.222.0.18 () {54 vars in 927 bytes} [Tue Sep 13 03:07:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 03:07:25.041407 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45cc6cd2-4df3-4947-8a2b-b3b35d738c75 None None] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:07:25.041813 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45cc6cd2-4df3-4947-8a2b-b3b35d738c75 None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:07:25.041984 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45cc6cd2-4df3-4947-8a2b-b3b35d738c75 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:07:25.068429 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.auth.core [None req-45cc6cd2-4df3-4947-8a2b-b3b35d738c75 None None] MFA Rules not processed for user `49f280ab086b4610855c4ffa378517d9`. Rule list: `[]` (Enabled: `True`). {{(pid=81399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 03:07:25.070029 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-45cc6cd2-4df3-4947-8a2b-b3b35d738c75 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:07:25.119550 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d2856a5-e905-48b8-b309-262a0965d174 None None] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:07:25.120162 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d2856a5-e905-48b8-b309-262a0965d174 None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:07:25.120633 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d2856a5-e905-48b8-b309-262a0965d174 None None] PATH_INFO: `/` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:07:25.123540 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 122/236] 10.222.0.18 () {54 vars in 927 bytes} [Tue Sep 13 03:07:25 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 03:07:25.127910 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea9a69fa-aaf8-4e08-9ea3-4c90e9a1b01d None None] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:07:25.128423 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea9a69fa-aaf8-4e08-9ea3-4c90e9a1b01d None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:07:25.128892 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea9a69fa-aaf8-4e08-9ea3-4c90e9a1b01d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:07:25.155860 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.auth.core [None req-ea9a69fa-aaf8-4e08-9ea3-4c90e9a1b01d None None] MFA Rules not processed for user `49f280ab086b4610855c4ffa378517d9`. Rule list: `[]` (Enabled: `True`). {{(pid=81400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 03:07:25.158911 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-ea9a69fa-aaf8-4e08-9ea3-4c90e9a1b01d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:07:25.183365 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 115/237] 10.222.0.18 () {58 vars in 1024 bytes} [Tue Sep 13 03:07:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 13 03:07:25.214638 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:07:25.259256 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 123/238] 10.222.0.18 () {58 vars in 1024 bytes} [Tue Sep 13 03:07:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 13 03:07:25.572734 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45cc6cd2-4df3-4947-8a2b-b3b35d738c75 None None] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:07:25.576740 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:07:25.609699 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea9a69fa-aaf8-4e08-9ea3-4c90e9a1b01d None None] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:07:25.744602 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45cc6cd2-4df3-4947-8a2b-b3b35d738c75 None None] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:07:25.790764 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea9a69fa-aaf8-4e08-9ea3-4c90e9a1b01d None None] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:07:25.813459 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e36b7c35-53cc-49e4-99d6-2776e20d8e23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:07:25.814108 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e36b7c35-53cc-49e4-99d6-2776e20d8e23 None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:07:25.814389 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e36b7c35-53cc-49e4-99d6-2776e20d8e23 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:07:25.817404 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e36b7c35-53cc-49e4-99d6-2776e20d8e23 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:07:25.839138 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e36b7c35-53cc-49e4-99d6-2776e20d8e23 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 03:07:25.850984 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application [None req-e36b7c35-53cc-49e4-99d6-2776e20d8e23 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 13 03:07:25.850984 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 03:07:25.850984 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 03:07:25.850984 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 03:07:25.850984 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 03:07:25.850984 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 03:07:25.850984 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 03:07:25.850984 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 03:07:25.850984 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 03:07:25.850984 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 03:07:25.850984 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 03:07:25.850984 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 03:07:25.850984 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 03:07:25.850984 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 03:07:25.850984 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 03:07:25.850984 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 03:07:25.851893 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 03:07:25.851893 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 03:07:25.851893 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 03:07:25.851893 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 03:07:25.851893 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 03:07:25.851893 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 03:07:25.851893 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 03:07:25.851893 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 03:07:25.851893 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 03:07:25.851893 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 03:07:25.851893 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 03:07:25.851893 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application with Lock( Sep 13 03:07:25.851893 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 03:07:25.851893 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._enter() Sep 13 03:07:25.851893 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 03:07:25.851893 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 03:07:25.852707 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 03:07:25.852707 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.creator() Sep 13 03:07:25.852707 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 03:07:25.852707 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application created_value = creator( Sep 13 03:07:25.852707 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 03:07:25.852707 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 03:07:25.852707 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 03:07:25.852707 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 03:07:25.852707 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 03:07:25.852707 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 03:07:25.852707 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 13 03:07:25.852707 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Sep 13 03:07:25.854873 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 116/239] 10.222.0.18 () {56 vars in 1099 bytes} [Tue Sep 13 03:07:25 2022] GET /identity/v3/roles/reader => generated 84 bytes in 282 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 03:07:25.859816 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e36b7c35-53cc-49e4-99d6-2776e20d8e23 None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:07:25.865700 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d800227f-f850-4aec-95fe-ef6b76a05e3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:07:25.866338 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d800227f-f850-4aec-95fe-ef6b76a05e3c None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:07:25.866641 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d800227f-f850-4aec-95fe-ef6b76a05e3c None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:07:25.866912 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d800227f-f850-4aec-95fe-ef6b76a05e3c None admin] PATH_INFO: `/v3/domains/default` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:07:25.868019 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d800227f-f850-4aec-95fe-ef6b76a05e3c None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:07:25.869461 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d800227f-f850-4aec-95fe-ef6b76a05e3c None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:07:25.871127 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 124/240] 10.222.0.18 () {56 vars in 1108 bytes} [Tue Sep 13 03:07:25 2022] GET /identity/v3/domains/default => generated 199 bytes in 262 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 03:07:25.872350 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4318f17f-6dbf-4a01-a183-22fba77b14f0 None None] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:07:25.872621 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4318f17f-6dbf-4a01-a183-22fba77b14f0 None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:07:25.872883 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4318f17f-6dbf-4a01-a183-22fba77b14f0 None None] PATH_INFO: `/` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:07:25.874306 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 125/241] 10.222.0.18 () {54 vars in 927 bytes} [Tue Sep 13 03:07:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 03:07:25.878835 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4318f17f-6dbf-4a01-a183-22fba77b14f0 None None] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:07:25.930367 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:07:25.942154 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:07:25.989917 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e36b7c35-53cc-49e4-99d6-2776e20d8e23 None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:07:26.001462 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4318f17f-6dbf-4a01-a183-22fba77b14f0 None None] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:07:26.040087 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cc79347-e790-4871-bee9-0825e3ac7e8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:07:26.041543 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cc79347-e790-4871-bee9-0825e3ac7e8a None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:07:26.042572 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cc79347-e790-4871-bee9-0825e3ac7e8a None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:07:26.043090 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cc79347-e790-4871-bee9-0825e3ac7e8a None admin] PATH_INFO: `/v3/roles` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:07:26.044070 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cc79347-e790-4871-bee9-0825e3ac7e8a None admin] RBAC: Adding query filter params (name=reader) {{(pid=81399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 03:07:26.044682 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cc79347-e790-4871-bee9-0825e3ac7e8a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:07:26.045785 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9974ba21-d183-4257-82a7-ea3582b25c45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:07:26.046520 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9974ba21-d183-4257-82a7-ea3582b25c45 None admin] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:07:26.046945 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9974ba21-d183-4257-82a7-ea3582b25c45 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:07:26.047074 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cc79347-e790-4871-bee9-0825e3ac7e8a None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:07:26.047391 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9974ba21-d183-4257-82a7-ea3582b25c45 None admin] PATH_INFO: `/v3/users` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:07:26.048168 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9974ba21-d183-4257-82a7-ea3582b25c45 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:07:26.049932 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9974ba21-d183-4257-82a7-ea3582b25c45 None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:07:26.061522 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 117/242] 10.222.0.18 () {56 vars in 1101 bytes} [Tue Sep 13 03:07:25 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 03:07:26.061522 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d38e73f-1456-4827-9216-7c6b87af6026 None None] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:07:26.061522 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d38e73f-1456-4827-9216-7c6b87af6026 None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:07:26.061522 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d38e73f-1456-4827-9216-7c6b87af6026 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:07:26.087389 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 126/243] 10.222.0.18 () {60 vars in 1132 bytes} [Tue Sep 13 03:07:25 2022] POST /identity/v3/users => generated 293 bytes in 204 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 03:07:26.087389 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9974ba21-d183-4257-82a7-ea3582b25c45 None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:07:26.106286 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.auth.core [None req-4d38e73f-1456-4827-9216-7c6b87af6026 None None] MFA Rules not processed for user `49f280ab086b4610855c4ffa378517d9`. Rule list: `[]` (Enabled: `True`). {{(pid=81399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 03:07:26.107468 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-4d38e73f-1456-4827-9216-7c6b87af6026 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:07:26.205361 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 118/244] 10.222.0.18 () {58 vars in 1024 bytes} [Tue Sep 13 03:07:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 13 03:07:26.293763 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:07:26.300342 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9974ba21-d183-4257-82a7-ea3582b25c45 None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:07:26.336142 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-526c9470-9690-4742-b341-5ecc423f4171 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:07:26.336770 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-526c9470-9690-4742-b341-5ecc423f4171 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:07:26.337046 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-526c9470-9690-4742-b341-5ecc423f4171 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:07:26.337315 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-526c9470-9690-4742-b341-5ecc423f4171 None admin] PATH_INFO: `/v3/users/46f0efbf2eed4631b420782e6d2992b4` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:07:26.352376 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-526c9470-9690-4742-b341-5ecc423f4171 None admin] RBAC: Authorizing `identity:get_user(user_id=46f0efbf2eed4631b420782e6d2992b4)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:07:26.353908 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-526c9470-9690-4742-b341-5ecc423f4171 None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:07:26.355899 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 127/245] 10.222.0.18 () {56 vars in 1177 bytes} [Tue Sep 13 03:07:26 2022] GET /identity/v3/users/46f0efbf2eed4631b420782e6d2992b4 => generated 297 bytes in 272 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 03:07:26.361774 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d38e73f-1456-4827-9216-7c6b87af6026 None None] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:07:26.430850 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:07:26.483572 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d38e73f-1456-4827-9216-7c6b87af6026 None None] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:07:26.523910 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-091d5504-d04d-4b6d-b8bc-fc786e5c9a07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:07:26.524582 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-091d5504-d04d-4b6d-b8bc-fc786e5c9a07 None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:07:26.524868 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-091d5504-d04d-4b6d-b8bc-fc786e5c9a07 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:07:26.525140 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-091d5504-d04d-4b6d-b8bc-fc786e5c9a07 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:07:26.599320 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-526c9470-9690-4742-b341-5ecc423f4171 None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:07:26.610204 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-091d5504-d04d-4b6d-b8bc-fc786e5c9a07 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:07:26.610717 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-091d5504-d04d-4b6d-b8bc-fc786e5c9a07 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:07:26.678700 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:07:26.748061 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 119/246] 10.222.0.18 () {58 vars in 1282 bytes} [Tue Sep 13 03:07:26 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 384 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 13 03:07:26.748271 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-526c9470-9690-4742-b341-5ecc423f4171 None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:07:26.752816 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-091d5504-d04d-4b6d-b8bc-fc786e5c9a07 None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:07:26.796681 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b2d190b-36f4-4d55-a888-4104400f5aa5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:07:26.798199 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b2d190b-36f4-4d55-a888-4104400f5aa5 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:07:26.798703 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b2d190b-36f4-4d55-a888-4104400f5aa5 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:07:26.799174 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b2d190b-36f4-4d55-a888-4104400f5aa5 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:07:26.814384 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b2d190b-36f4-4d55-a888-4104400f5aa5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 03:07:26.821023 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application [None req-3b2d190b-36f4-4d55-a888-4104400f5aa5 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 13 03:07:26.821023 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 03:07:26.821023 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 03:07:26.821023 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 03:07:26.821023 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 03:07:26.821023 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 03:07:26.821023 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 03:07:26.821023 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 03:07:26.821023 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 03:07:26.821023 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 03:07:26.821023 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 03:07:26.821023 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 03:07:26.821023 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 03:07:26.821023 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 03:07:26.821023 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 03:07:26.821023 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 03:07:26.821935 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 03:07:26.821935 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 03:07:26.821935 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 03:07:26.821935 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 03:07:26.821935 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 03:07:26.821935 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 03:07:26.821935 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 03:07:26.821935 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 03:07:26.821935 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 03:07:26.821935 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 03:07:26.821935 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 03:07:26.821935 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application with Lock( Sep 13 03:07:26.821935 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 03:07:26.821935 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._enter() Sep 13 03:07:26.821935 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 03:07:26.821935 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 03:07:26.822779 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 03:07:26.822779 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.creator() Sep 13 03:07:26.822779 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 03:07:26.822779 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application created_value = creator( Sep 13 03:07:26.822779 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 03:07:26.822779 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 03:07:26.822779 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 03:07:26.822779 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 03:07:26.822779 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 03:07:26.822779 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 03:07:26.822779 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 13 03:07:26.822779 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Sep 13 03:07:26.826681 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 128/247] 10.222.0.18 () {56 vars in 1099 bytes} [Tue Sep 13 03:07:26 2022] GET /identity/v3/roles/member => generated 84 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 13 03:07:26.826681 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a54684-69cd-48ad-af03-71556fdf8dd2 None None] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:07:26.826681 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a54684-69cd-48ad-af03-71556fdf8dd2 None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:07:26.826681 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a54684-69cd-48ad-af03-71556fdf8dd2 None None] PATH_INFO: `/` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:07:26.828167 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 129/248] 10.222.0.18 () {54 vars in 927 bytes} [Tue Sep 13 03:07:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 03:07:26.829972 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32a54684-69cd-48ad-af03-71556fdf8dd2 None None] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:07:26.852247 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:07:26.877986 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-091d5504-d04d-4b6d-b8bc-fc786e5c9a07 None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:07:26.913766 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd42b2a5-bbda-42a0-a3d7-563f0267dbf7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:07:26.914725 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd42b2a5-bbda-42a0-a3d7-563f0267dbf7 None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:07:26.915192 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd42b2a5-bbda-42a0-a3d7-563f0267dbf7 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:07:26.915573 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd42b2a5-bbda-42a0-a3d7-563f0267dbf7 None admin] PATH_INFO: `/v3/projects/54999ea237f64c5f972ef79e765facd0` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:07:26.916813 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd42b2a5-bbda-42a0-a3d7-563f0267dbf7 None admin] RBAC: Authorizing `identity:get_project(project_id=54999ea237f64c5f972ef79e765facd0)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:07:26.918364 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd42b2a5-bbda-42a0-a3d7-563f0267dbf7 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:07:26.920452 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 120/249] 10.222.0.18 () {56 vars in 1186 bytes} [Tue Sep 13 03:07:26 2022] GET /identity/v3/projects/54999ea237f64c5f972ef79e765facd0 => generated 302 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 03:07:26.921724 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec3b7cd-7a13-4570-8d68-07eb42eeb3f1 None None] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:07:26.922103 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec3b7cd-7a13-4570-8d68-07eb42eeb3f1 None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:07:26.922478 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec3b7cd-7a13-4570-8d68-07eb42eeb3f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:07:26.942584 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.auth.core [None req-0ec3b7cd-7a13-4570-8d68-07eb42eeb3f1 None None] MFA Rules not processed for user `49f280ab086b4610855c4ffa378517d9`. Rule list: `[]` (Enabled: `True`). {{(pid=81399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 03:07:26.943857 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-0ec3b7cd-7a13-4570-8d68-07eb42eeb3f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:07:26.962789 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:07:26.977376 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32a54684-69cd-48ad-af03-71556fdf8dd2 None None] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:07:27.018145 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 121/250] 10.222.0.18 () {58 vars in 1024 bytes} [Tue Sep 13 03:07:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 13 03:07:27.019762 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d75ed3-f705-4cd6-992a-6af10adf1980 None None] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:07:27.020147 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d75ed3-f705-4cd6-992a-6af10adf1980 None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:07:27.020519 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d75ed3-f705-4cd6-992a-6af10adf1980 None None] PATH_INFO: `/` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:07:27.021888 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 122/251] 10.222.0.18 () {54 vars in 927 bytes} [Tue Sep 13 03:07:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 03:07:27.022400 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d44d5f82-26a0-4c9d-8cc9-04ccd64266a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:07:27.023037 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43d75ed3-f705-4cd6-992a-6af10adf1980 None None] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:07:27.023735 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d44d5f82-26a0-4c9d-8cc9-04ccd64266a3 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:07:27.024402 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d44d5f82-26a0-4c9d-8cc9-04ccd64266a3 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:07:27.025070 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d44d5f82-26a0-4c9d-8cc9-04ccd64266a3 None admin] PATH_INFO: `/v3/roles` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:07:27.026986 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d44d5f82-26a0-4c9d-8cc9-04ccd64266a3 None admin] RBAC: Adding query filter params (name=member) {{(pid=81400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 03:07:27.027848 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d44d5f82-26a0-4c9d-8cc9-04ccd64266a3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:07:27.030766 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d44d5f82-26a0-4c9d-8cc9-04ccd64266a3 None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:07:27.047225 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 130/252] 10.222.0.18 () {56 vars in 1101 bytes} [Tue Sep 13 03:07:26 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 03:07:27.050582 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18cb51f2-3101-4165-81e2-dec5ceb82b80 None None] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:07:27.051422 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18cb51f2-3101-4165-81e2-dec5ceb82b80 None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:07:27.052221 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18cb51f2-3101-4165-81e2-dec5ceb82b80 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:07:27.088963 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.auth.core [None req-18cb51f2-3101-4165-81e2-dec5ceb82b80 None None] MFA Rules not processed for user `49f280ab086b4610855c4ffa378517d9`. Rule list: `[]` (Enabled: `True`). {{(pid=81400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 03:07:27.090136 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-18cb51f2-3101-4165-81e2-dec5ceb82b80 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:07:27.208539 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43d75ed3-f705-4cd6-992a-6af10adf1980 None None] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:07:27.228330 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 131/253] 10.222.0.18 () {58 vars in 1024 bytes} [Tue Sep 13 03:07:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 13 03:07:27.229168 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18cb51f2-3101-4165-81e2-dec5ceb82b80 None None] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:07:27.254258 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:07:27.278440 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07a9783d-48b4-414c-9ded-17a9dc60831c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:07:27.279081 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a9783d-48b4-414c-9ded-17a9dc60831c None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:07:27.279355 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a9783d-48b4-414c-9ded-17a9dc60831c None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:07:27.279621 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a9783d-48b4-414c-9ded-17a9dc60831c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:07:27.280340 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07a9783d-48b4-414c-9ded-17a9dc60831c None admin] RBAC: Adding query filter params (role.id=502e6c11e747432988d7f723cee0af64, scope.project.id=54999ea237f64c5f972ef79e765facd0, user.id=46f0efbf2eed4631b420782e6d2992b4) {{(pid=81399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 03:07:27.280641 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07a9783d-48b4-414c-9ded-17a9dc60831c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:07:27.284621 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07a9783d-48b4-414c-9ded-17a9dc60831c None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:07:27.295861 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 123/254] 10.222.0.18 () {56 vars in 1374 bytes} [Tue Sep 13 03:07:27 2022] GET /identity/v3/role_assignments?user.id=46f0efbf2eed4631b420782e6d2992b4&scope.project.id=54999ea237f64c5f972ef79e765facd0&role.id=502e6c11e747432988d7f723cee0af64 => generated 259 bytes in 273 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 03:07:27.332625 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07a9783d-48b4-414c-9ded-17a9dc60831c None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:07:27.376729 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:07:27.463019 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18cb51f2-3101-4165-81e2-dec5ceb82b80 None None] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:07:27.553785 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69842cf9-9ad4-421f-adff-f0fc6812d2bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:07:27.554976 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69842cf9-9ad4-421f-adff-f0fc6812d2bf None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:07:27.555441 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69842cf9-9ad4-421f-adff-f0fc6812d2bf None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:07:27.556107 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69842cf9-9ad4-421f-adff-f0fc6812d2bf None admin] PATH_INFO: `/v3/users/cbbc542ba2bb42deb9b1ad183d118075` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:07:27.557564 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69842cf9-9ad4-421f-adff-f0fc6812d2bf None admin] RBAC: Authorizing `identity:get_user(user_id=cbbc542ba2bb42deb9b1ad183d118075)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:07:27.559272 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69842cf9-9ad4-421f-adff-f0fc6812d2bf None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:07:27.562623 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 132/255] 10.222.0.18 () {56 vars in 1177 bytes} [Tue Sep 13 03:07:27 2022] GET /identity/v3/users/cbbc542ba2bb42deb9b1ad183d118075 => generated 297 bytes in 333 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 03:07:27.573712 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69842cf9-9ad4-421f-adff-f0fc6812d2bf None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:07:27.607557 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07a9783d-48b4-414c-9ded-17a9dc60831c None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:07:27.649644 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f246b270-8ac9-47c8-8964-cdff8c2f36bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:07:27.650297 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f246b270-8ac9-47c8-8964-cdff8c2f36bd None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:07:27.650575 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f246b270-8ac9-47c8-8964-cdff8c2f36bd None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:07:27.651163 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f246b270-8ac9-47c8-8964-cdff8c2f36bd None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:07:27.668030 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:07:27.673893 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f246b270-8ac9-47c8-8964-cdff8c2f36bd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 03:07:27.679821 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application [None req-f246b270-8ac9-47c8-8964-cdff8c2f36bd None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 13 03:07:27.679821 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 03:07:27.679821 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 03:07:27.679821 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 03:07:27.679821 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 03:07:27.679821 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 03:07:27.679821 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 03:07:27.679821 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 03:07:27.679821 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 03:07:27.679821 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 03:07:27.679821 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 03:07:27.679821 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 03:07:27.679821 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 03:07:27.679821 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 03:07:27.679821 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 03:07:27.679821 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 03:07:27.681063 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 03:07:27.681063 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 03:07:27.681063 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 03:07:27.681063 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 03:07:27.681063 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 03:07:27.681063 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 03:07:27.681063 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 03:07:27.681063 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 03:07:27.681063 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 03:07:27.681063 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 03:07:27.681063 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 03:07:27.681063 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application with Lock( Sep 13 03:07:27.681063 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 03:07:27.681063 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._enter() Sep 13 03:07:27.681063 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 03:07:27.681063 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 03:07:27.681848 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 03:07:27.681848 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.creator() Sep 13 03:07:27.681848 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 03:07:27.681848 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application created_value = creator( Sep 13 03:07:27.681848 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 03:07:27.681848 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 03:07:27.681848 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 03:07:27.681848 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 03:07:27.681848 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 03:07:27.681848 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 03:07:27.681848 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 13 03:07:27.681848 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Sep 13 03:07:27.681848 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 124/256] 10.222.0.18 () {56 vars in 1114 bytes} [Tue Sep 13 03:07:27 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 349 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 13 03:07:27.682525 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f246b270-8ac9-47c8-8964-cdff8c2f36bd None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:07:27.712350 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69842cf9-9ad4-421f-adff-f0fc6812d2bf None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:07:27.783227 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-313b873b-688f-4117-903a-ae7ca28e85ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:07:27.783771 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-313b873b-688f-4117-903a-ae7ca28e85ed None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:07:27.784046 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-313b873b-688f-4117-903a-ae7ca28e85ed None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:07:27.784313 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-313b873b-688f-4117-903a-ae7ca28e85ed None admin] PATH_INFO: `/v3/projects/54999ea237f64c5f972ef79e765facd0` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:07:27.785550 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-313b873b-688f-4117-903a-ae7ca28e85ed None admin] RBAC: Authorizing `identity:get_project(project_id=54999ea237f64c5f972ef79e765facd0)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:07:27.788150 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-313b873b-688f-4117-903a-ae7ca28e85ed None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:07:27.789804 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 133/257] 10.222.0.18 () {56 vars in 1186 bytes} [Tue Sep 13 03:07:27 2022] GET /identity/v3/projects/54999ea237f64c5f972ef79e765facd0 => generated 302 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 03:07:27.790550 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-313b873b-688f-4117-903a-ae7ca28e85ed None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:07:27.832963 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:07:27.874443 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f246b270-8ac9-47c8-8964-cdff8c2f36bd None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:07:27.940244 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c0951c9-5589-459b-89b2-9f80a53ae292 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:07:27.941250 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c0951c9-5589-459b-89b2-9f80a53ae292 None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:07:27.941744 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c0951c9-5589-459b-89b2-9f80a53ae292 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:07:27.942005 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c0951c9-5589-459b-89b2-9f80a53ae292 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:07:27.966879 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c0951c9-5589-459b-89b2-9f80a53ae292 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 03:07:27.974428 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application [None req-5c0951c9-5589-459b-89b2-9f80a53ae292 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 13 03:07:27.974428 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 03:07:27.974428 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 03:07:27.974428 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 03:07:27.974428 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 03:07:27.974428 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 03:07:27.974428 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 03:07:27.974428 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 03:07:27.974428 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 03:07:27.974428 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 03:07:27.974428 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 03:07:27.974428 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 03:07:27.974428 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 03:07:27.974428 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 03:07:27.974428 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 03:07:27.974428 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 03:07:27.975391 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 03:07:27.975391 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 03:07:27.975391 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 03:07:27.975391 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 03:07:27.975391 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 03:07:27.975391 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 03:07:27.975391 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 03:07:27.975391 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 03:07:27.975391 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 03:07:27.975391 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 03:07:27.975391 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 03:07:27.975391 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application with Lock( Sep 13 03:07:27.975391 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 03:07:27.975391 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._enter() Sep 13 03:07:27.975391 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 03:07:27.975391 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 03:07:27.976189 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 03:07:27.976189 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.creator() Sep 13 03:07:27.976189 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 03:07:27.976189 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application created_value = creator( Sep 13 03:07:27.976189 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 03:07:27.976189 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 03:07:27.976189 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 03:07:27.976189 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 03:07:27.976189 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 03:07:27.976189 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 03:07:27.976189 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 13 03:07:27.976189 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Sep 13 03:07:27.983097 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 125/258] 10.222.0.18 () {56 vars in 1096 bytes} [Tue Sep 13 03:07:27 2022] GET /identity/v3/roles/admin => generated 83 bytes in 301 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 03:07:27.985547 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c0951c9-5589-459b-89b2-9f80a53ae292 None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:07:27.987885 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-313b873b-688f-4117-903a-ae7ca28e85ed None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:07:28.029823 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1e273fd-efd0-4fd3-9f96-44fc046c0ffc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:07:28.030838 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e273fd-efd0-4fd3-9f96-44fc046c0ffc None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:07:28.031326 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e273fd-efd0-4fd3-9f96-44fc046c0ffc None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:07:28.032035 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e273fd-efd0-4fd3-9f96-44fc046c0ffc None admin] PATH_INFO: `/v3/roles` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:07:28.032909 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1e273fd-efd0-4fd3-9f96-44fc046c0ffc None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 03:07:28.033475 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1e273fd-efd0-4fd3-9f96-44fc046c0ffc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:07:28.035096 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1e273fd-efd0-4fd3-9f96-44fc046c0ffc None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:07:28.044780 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 134/259] 10.222.0.18 () {56 vars in 1111 bytes} [Tue Sep 13 03:07:27 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 254 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 03:07:28.046090 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1e273fd-efd0-4fd3-9f96-44fc046c0ffc None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:07:28.050249 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:07:28.193090 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c0951c9-5589-459b-89b2-9f80a53ae292 None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:07:28.246197 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1e273fd-efd0-4fd3-9f96-44fc046c0ffc None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:07:28.268522 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e4bcd6b-0be2-43ed-bece-aa22ba1739b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:07:28.269195 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e4bcd6b-0be2-43ed-bece-aa22ba1739b4 None admin] REQUEST_METHOD: `PUT` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:07:28.269477 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e4bcd6b-0be2-43ed-bece-aa22ba1739b4 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:07:28.269749 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e4bcd6b-0be2-43ed-bece-aa22ba1739b4 None admin] PATH_INFO: `/v3/projects/54999ea237f64c5f972ef79e765facd0/users/cbbc542ba2bb42deb9b1ad183d118075/roles/1eb7a5b5203945c29e6d22d653f47847` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:07:28.272103 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e4bcd6b-0be2-43ed-bece-aa22ba1739b4 None admin] RBAC: Authorizing `identity:create_grant(project_id=54999ea237f64c5f972ef79e765facd0, user_id=cbbc542ba2bb42deb9b1ad183d118075, role_id=1eb7a5b5203945c29e6d22d653f47847)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:07:28.275042 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e4bcd6b-0be2-43ed-bece-aa22ba1739b4 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:07:28.299336 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:07:28.304228 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 126/260] 10.222.0.18 () {58 vars in 1439 bytes} [Tue Sep 13 03:07:27 2022] PUT /identity/v3/projects/54999ea237f64c5f972ef79e765facd0/users/cbbc542ba2bb42deb9b1ad183d118075/roles/1eb7a5b5203945c29e6d22d653f47847 => generated 0 bytes in 320 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 03:07:28.305018 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e4bcd6b-0be2-43ed-bece-aa22ba1739b4 None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:07:28.318217 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d6642b6-5272-4b6a-8523-1c4d5fd206dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:07:28.318912 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d6642b6-5272-4b6a-8523-1c4d5fd206dc None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:07:28.319194 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d6642b6-5272-4b6a-8523-1c4d5fd206dc None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:07:28.319468 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d6642b6-5272-4b6a-8523-1c4d5fd206dc None admin] PATH_INFO: `/v3/roles` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:07:28.320696 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d6642b6-5272-4b6a-8523-1c4d5fd206dc None admin] RBAC: Adding query filter params (name=admin) {{(pid=81400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 03:07:28.321008 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d6642b6-5272-4b6a-8523-1c4d5fd206dc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:07:28.322486 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d6642b6-5272-4b6a-8523-1c4d5fd206dc None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:07:28.343928 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 135/261] 10.222.0.18 () {56 vars in 1099 bytes} [Tue Sep 13 03:07:28 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 298 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 03:07:28.350320 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d6642b6-5272-4b6a-8523-1c4d5fd206dc None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:07:28.451525 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e4bcd6b-0be2-43ed-bece-aa22ba1739b4 None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:07:28.467587 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:07:28.493085 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af3b7a00-f109-4c25-a810-608aefe8d5fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:07:28.494077 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af3b7a00-f109-4c25-a810-608aefe8d5fb None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:07:28.494367 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af3b7a00-f109-4c25-a810-608aefe8d5fb None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:07:28.494664 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af3b7a00-f109-4c25-a810-608aefe8d5fb None admin] PATH_INFO: `/v3/users/86fc8d5c47e944ec8f7e33a2d6c291da` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:07:28.495850 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af3b7a00-f109-4c25-a810-608aefe8d5fb None admin] RBAC: Authorizing `identity:get_user(user_id=86fc8d5c47e944ec8f7e33a2d6c291da)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:07:28.497308 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af3b7a00-f109-4c25-a810-608aefe8d5fb None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:07:28.499772 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 127/262] 10.222.0.18 () {56 vars in 1177 bytes} [Tue Sep 13 03:07:28 2022] GET /identity/v3/users/86fc8d5c47e944ec8f7e33a2d6c291da => generated 283 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 03:07:28.507129 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af3b7a00-f109-4c25-a810-608aefe8d5fb None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:07:28.512811 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d6642b6-5272-4b6a-8523-1c4d5fd206dc None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:07:28.552177 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45b93fba-c058-4490-b551-78053cf7574a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:07:28.553035 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b93fba-c058-4490-b551-78053cf7574a None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:07:28.553473 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b93fba-c058-4490-b551-78053cf7574a None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:07:28.553887 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b93fba-c058-4490-b551-78053cf7574a None admin] PATH_INFO: `/v3/users/86fc8d5c47e944ec8f7e33a2d6c291da` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:07:28.555307 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45b93fba-c058-4490-b551-78053cf7574a None admin] RBAC: Authorizing `identity:get_user(user_id=86fc8d5c47e944ec8f7e33a2d6c291da)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:07:28.557252 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45b93fba-c058-4490-b551-78053cf7574a None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:07:28.559648 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 136/263] 10.222.0.18 () {56 vars in 1177 bytes} [Tue Sep 13 03:07:28 2022] GET /identity/v3/users/86fc8d5c47e944ec8f7e33a2d6c291da => generated 283 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 03:07:28.567777 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45b93fba-c058-4490-b551-78053cf7574a None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:07:28.586123 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:07:28.629777 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af3b7a00-f109-4c25-a810-608aefe8d5fb None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:07:28.670230 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-694fee4a-d4e3-4fad-9c72-c5de327ac101 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:07:28.670230 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-694fee4a-d4e3-4fad-9c72-c5de327ac101 None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:07:28.670230 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-694fee4a-d4e3-4fad-9c72-c5de327ac101 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:07:28.670230 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-694fee4a-d4e3-4fad-9c72-c5de327ac101 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:07:28.699129 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45b93fba-c058-4490-b551-78053cf7574a None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:07:28.756298 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a24901b-6d74-4465-83f6-e81725304dbe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:07:28.757184 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a24901b-6d74-4465-83f6-e81725304dbe None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:07:28.758576 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a24901b-6d74-4465-83f6-e81725304dbe None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:07:28.759006 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a24901b-6d74-4465-83f6-e81725304dbe None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:07:28.764208 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:07:28.792012 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-694fee4a-d4e3-4fad-9c72-c5de327ac101 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:07:28.793483 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-694fee4a-d4e3-4fad-9c72-c5de327ac101 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:07:28.915412 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a24901b-6d74-4465-83f6-e81725304dbe None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:07:28.918210 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a24901b-6d74-4465-83f6-e81725304dbe None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:07:28.937787 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:07:29.016884 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 128/264] 10.222.0.18 () {58 vars in 1282 bytes} [Tue Sep 13 03:07:28 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 511 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 13 03:07:29.022642 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-694fee4a-d4e3-4fad-9c72-c5de327ac101 None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:07:29.123748 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 137/265] 10.222.0.18 () {58 vars in 1282 bytes} [Tue Sep 13 03:07:28 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 556 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 13 03:07:29.130585 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a24901b-6d74-4465-83f6-e81725304dbe None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:07:29.211526 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:07:29.262337 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-694fee4a-d4e3-4fad-9c72-c5de327ac101 None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:07:29.333641 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-032a90c2-3feb-4df0-a194-72e28a4da3a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:07:29.334297 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-032a90c2-3feb-4df0-a194-72e28a4da3a8 None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:07:29.334570 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-032a90c2-3feb-4df0-a194-72e28a4da3a8 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:07:29.334847 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-032a90c2-3feb-4df0-a194-72e28a4da3a8 None admin] PATH_INFO: `/v3/projects/54999ea237f64c5f972ef79e765facd0` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:07:29.336329 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-032a90c2-3feb-4df0-a194-72e28a4da3a8 None admin] RBAC: Authorizing `identity:get_project(project_id=54999ea237f64c5f972ef79e765facd0)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:07:29.343835 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-032a90c2-3feb-4df0-a194-72e28a4da3a8 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:07:29.346580 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 129/266] 10.222.0.18 () {56 vars in 1186 bytes} [Tue Sep 13 03:07:29 2022] GET /identity/v3/projects/54999ea237f64c5f972ef79e765facd0 => generated 302 bytes in 324 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 03:07:29.352439 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-032a90c2-3feb-4df0-a194-72e28a4da3a8 None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:07:29.399095 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:07:29.401546 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a24901b-6d74-4465-83f6-e81725304dbe None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:07:29.457130 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-032a90c2-3feb-4df0-a194-72e28a4da3a8 None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:07:29.466869 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-636c7e06-9ed0-4868-83cf-1a5dbba6a27c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:07:29.467528 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-636c7e06-9ed0-4868-83cf-1a5dbba6a27c None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:07:29.467809 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-636c7e06-9ed0-4868-83cf-1a5dbba6a27c None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:07:29.468081 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-636c7e06-9ed0-4868-83cf-1a5dbba6a27c None admin] PATH_INFO: `/v3/projects/54999ea237f64c5f972ef79e765facd0` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:07:29.469208 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-636c7e06-9ed0-4868-83cf-1a5dbba6a27c None admin] RBAC: Authorizing `identity:get_project(project_id=54999ea237f64c5f972ef79e765facd0)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:07:29.471244 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-636c7e06-9ed0-4868-83cf-1a5dbba6a27c None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:07:29.473204 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 138/267] 10.222.0.18 () {56 vars in 1186 bytes} [Tue Sep 13 03:07:29 2022] GET /identity/v3/projects/54999ea237f64c5f972ef79e765facd0 => generated 302 bytes in 344 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 03:07:29.474463 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ca760cd-b07f-43f7-a0a0-ed2c8e17fe5a None None] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:07:29.474777 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ca760cd-b07f-43f7-a0a0-ed2c8e17fe5a None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:07:29.475050 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ca760cd-b07f-43f7-a0a0-ed2c8e17fe5a None None] PATH_INFO: `/` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:07:29.476403 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 139/268] 10.222.0.18 () {54 vars in 927 bytes} [Tue Sep 13 03:07:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 03:07:29.477254 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44ac2c86-9f21-4e1e-ba9c-25c59f2912fa None None] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:07:29.477530 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44ac2c86-9f21-4e1e-ba9c-25c59f2912fa None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:07:29.477936 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44ac2c86-9f21-4e1e-ba9c-25c59f2912fa None None] PATH_INFO: `/` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:07:29.479195 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 140/269] 10.222.0.18 () {54 vars in 927 bytes} [Tue Sep 13 03:07:29 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 03:07:29.479764 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44ac2c86-9f21-4e1e-ba9c-25c59f2912fa None None] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:07:29.496624 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1b95732-d891-4713-8208-c964045c4820 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:07:29.497228 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b95732-d891-4713-8208-c964045c4820 None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:07:29.497501 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b95732-d891-4713-8208-c964045c4820 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:07:29.497769 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b95732-d891-4713-8208-c964045c4820 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:07:29.498485 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1b95732-d891-4713-8208-c964045c4820 None admin] RBAC: Adding query filter params (role.id=a9aa35ebda4d4aff80e62f750e9104ab, scope.project.id=54999ea237f64c5f972ef79e765facd0, user.id=86fc8d5c47e944ec8f7e33a2d6c291da) {{(pid=81399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 03:07:29.498814 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1b95732-d891-4713-8208-c964045c4820 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:07:29.500243 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1b95732-d891-4713-8208-c964045c4820 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:07:29.506950 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 130/270] 10.222.0.18 () {56 vars in 1374 bytes} [Tue Sep 13 03:07:29 2022] GET /identity/v3/role_assignments?user.id=86fc8d5c47e944ec8f7e33a2d6c291da&scope.project.id=54999ea237f64c5f972ef79e765facd0&role.id=a9aa35ebda4d4aff80e62f750e9104ab => generated 608 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 03:07:29.508097 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8dcec94-f5a4-4d8c-a476-b05514f4a3f6 None None] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:07:29.508373 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8dcec94-f5a4-4d8c-a476-b05514f4a3f6 None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:07:29.508643 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8dcec94-f5a4-4d8c-a476-b05514f4a3f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:07:29.528163 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.auth.core [None req-b8dcec94-f5a4-4d8c-a476-b05514f4a3f6 None None] MFA Rules not processed for user `49f280ab086b4610855c4ffa378517d9`. Rule list: `[]` (Enabled: `True`). {{(pid=81399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 03:07:29.529359 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-b8dcec94-f5a4-4d8c-a476-b05514f4a3f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:07:29.614937 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44ac2c86-9f21-4e1e-ba9c-25c59f2912fa None None] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:07:29.616835 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:07:29.629593 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 131/271] 10.222.0.18 () {58 vars in 1024 bytes} [Tue Sep 13 03:07:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 13 03:07:29.630724 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-269c779c-9c5c-4b8c-8ca8-7092a6c49976 None None] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:07:29.631010 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-269c779c-9c5c-4b8c-8ca8-7092a6c49976 None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:07:29.631283 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-269c779c-9c5c-4b8c-8ca8-7092a6c49976 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:07:29.668650 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.auth.core [None req-269c779c-9c5c-4b8c-8ca8-7092a6c49976 None None] MFA Rules not processed for user `49f280ab086b4610855c4ffa378517d9`. Rule list: `[]` (Enabled: `True`). {{(pid=81399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 03:07:29.669871 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-269c779c-9c5c-4b8c-8ca8-7092a6c49976 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:07:29.690412 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0ad444a-880d-4a8a-a987-2562780619da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:07:29.693074 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0ad444a-880d-4a8a-a987-2562780619da None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:07:29.693503 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0ad444a-880d-4a8a-a987-2562780619da None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:07:29.693782 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0ad444a-880d-4a8a-a987-2562780619da None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:07:29.694764 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0ad444a-880d-4a8a-a987-2562780619da None admin] RBAC: Adding query filter params (role.id=425d4b481c7846e0bc07e7e805d5b15f, scope.project.id=54999ea237f64c5f972ef79e765facd0, user.id=86fc8d5c47e944ec8f7e33a2d6c291da) {{(pid=81400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 03:07:29.695080 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0ad444a-880d-4a8a-a987-2562780619da None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:07:29.698830 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0ad444a-880d-4a8a-a987-2562780619da None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:07:29.716123 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 141/272] 10.222.0.18 () {56 vars in 1374 bytes} [Tue Sep 13 03:07:29 2022] GET /identity/v3/role_assignments?user.id=86fc8d5c47e944ec8f7e33a2d6c291da&scope.project.id=54999ea237f64c5f972ef79e765facd0&role.id=425d4b481c7846e0bc07e7e805d5b15f => generated 608 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 03:07:29.719396 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:07:29.772391 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 132/273] 10.222.0.18 () {58 vars in 1024 bytes} [Tue Sep 13 03:07:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 13 03:07:29.967243 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0ad444a-880d-4a8a-a987-2562780619da None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:07:30.095825 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0ad444a-880d-4a8a-a987-2562780619da None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:07:30.125266 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-269c779c-9c5c-4b8c-8ca8-7092a6c49976 None None] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:07:30.136414 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88ed853c-2202-4c5c-9773-f1133d22aff8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:07:30.137739 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ed853c-2202-4c5c-9773-f1133d22aff8 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:07:30.138194 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ed853c-2202-4c5c-9773-f1133d22aff8 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:07:30.138671 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ed853c-2202-4c5c-9773-f1133d22aff8 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:07:30.140095 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88ed853c-2202-4c5c-9773-f1133d22aff8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:07:30.141917 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88ed853c-2202-4c5c-9773-f1133d22aff8 None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:07:30.144055 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 142/274] 10.222.0.18 () {56 vars in 1108 bytes} [Tue Sep 13 03:07:29 2022] GET /identity/v3/domains/default => generated 199 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 03:07:30.149837 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88ed853c-2202-4c5c-9773-f1133d22aff8 None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:07:30.191079 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:07:30.251763 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-269c779c-9c5c-4b8c-8ca8-7092a6c49976 None None] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:07:30.267662 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88ed853c-2202-4c5c-9773-f1133d22aff8 None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:07:30.293292 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41871993-baa5-4046-9375-566881f4b3ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:07:30.294415 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41871993-baa5-4046-9375-566881f4b3ca None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:07:30.295237 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41871993-baa5-4046-9375-566881f4b3ca None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:07:30.295693 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41871993-baa5-4046-9375-566881f4b3ca None admin] PATH_INFO: `/v3/roles/member` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:07:30.308216 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41871993-baa5-4046-9375-566881f4b3ca None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 03:07:30.308476 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01eeee75-6a5f-4e36-a51e-f2ae2111ee4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:07:30.309153 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01eeee75-6a5f-4e36-a51e-f2ae2111ee4d None admin] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:07:30.309436 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01eeee75-6a5f-4e36-a51e-f2ae2111ee4d None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:07:30.309711 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01eeee75-6a5f-4e36-a51e-f2ae2111ee4d None admin] PATH_INFO: `/v3/users` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:07:30.310366 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01eeee75-6a5f-4e36-a51e-f2ae2111ee4d None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:07:30.311838 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01eeee75-6a5f-4e36-a51e-f2ae2111ee4d None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:07:30.314764 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application [None req-41871993-baa5-4046-9375-566881f4b3ca None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 13 03:07:30.314764 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 03:07:30.314764 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 03:07:30.314764 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 03:07:30.314764 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 03:07:30.314764 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 03:07:30.314764 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 03:07:30.314764 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 03:07:30.314764 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 03:07:30.314764 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 03:07:30.314764 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 03:07:30.314764 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 03:07:30.314764 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 03:07:30.314764 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 03:07:30.314764 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 03:07:30.314764 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 03:07:30.315634 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 03:07:30.315634 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 03:07:30.315634 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 03:07:30.315634 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 03:07:30.315634 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 03:07:30.315634 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 03:07:30.315634 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 03:07:30.315634 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 03:07:30.315634 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 03:07:30.315634 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 03:07:30.315634 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 03:07:30.315634 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application with Lock( Sep 13 03:07:30.315634 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 03:07:30.315634 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._enter() Sep 13 03:07:30.315634 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 03:07:30.315634 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 03:07:30.316519 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 03:07:30.316519 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.creator() Sep 13 03:07:30.316519 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 03:07:30.316519 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application created_value = creator( Sep 13 03:07:30.316519 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 03:07:30.316519 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 03:07:30.316519 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 03:07:30.316519 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 03:07:30.316519 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 03:07:30.316519 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 03:07:30.316519 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 13 03:07:30.316519 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Sep 13 03:07:30.322778 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 133/275] 10.222.0.18 () {56 vars in 1099 bytes} [Tue Sep 13 03:07:30 2022] GET /identity/v3/roles/member => generated 84 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 13 03:07:30.326118 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41871993-baa5-4046-9375-566881f4b3ca None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:07:30.347517 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 143/276] 10.222.0.18 () {60 vars in 1132 bytes} [Tue Sep 13 03:07:30 2022] POST /identity/v3/users => generated 293 bytes in 198 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 13 03:07:30.351957 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:07:30.376031 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fb2106e-3333-4057-8647-5a1893369f40 None None] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:07:30.376395 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fb2106e-3333-4057-8647-5a1893369f40 None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:07:30.376752 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fb2106e-3333-4057-8647-5a1893369f40 None None] PATH_INFO: `/` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:07:30.378394 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 144/277] 10.222.0.18 () {54 vars in 927 bytes} [Tue Sep 13 03:07:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 13 03:07:30.383965 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a0690d-0175-41a6-8e97-e1426ca512d9 None None] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:07:30.384236 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a0690d-0175-41a6-8e97-e1426ca512d9 None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:07:30.384506 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a0690d-0175-41a6-8e97-e1426ca512d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:07:30.405631 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.auth.core [None req-10a0690d-0175-41a6-8e97-e1426ca512d9 None None] MFA Rules not processed for user `49f280ab086b4610855c4ffa378517d9`. Rule list: `[]` (Enabled: `True`). {{(pid=81400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 03:07:30.407450 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-10a0690d-0175-41a6-8e97-e1426ca512d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:07:30.448319 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41871993-baa5-4046-9375-566881f4b3ca None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:07:30.482865 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 145/278] 10.222.0.18 () {58 vars in 1024 bytes} [Tue Sep 13 03:07:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 13 03:07:30.491471 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fcc4383-2146-427f-a4fa-729499abcba7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:07:30.492391 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fcc4383-2146-427f-a4fa-729499abcba7 None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:07:30.492848 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fcc4383-2146-427f-a4fa-729499abcba7 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:07:30.493321 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fcc4383-2146-427f-a4fa-729499abcba7 None admin] PATH_INFO: `/v3/roles` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:07:30.494173 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fcc4383-2146-427f-a4fa-729499abcba7 None admin] RBAC: Adding query filter params (name=member) {{(pid=81399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 03:07:30.494708 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fcc4383-2146-427f-a4fa-729499abcba7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:07:30.496318 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fcc4383-2146-427f-a4fa-729499abcba7 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:07:30.506146 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 134/279] 10.222.0.18 () {56 vars in 1101 bytes} [Tue Sep 13 03:07:30 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 03:07:30.512539 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10a0690d-0175-41a6-8e97-e1426ca512d9 None None] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:07:30.539023 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:07:30.626647 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10a0690d-0175-41a6-8e97-e1426ca512d9 None None] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:07:30.663996 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a930bdfc-172f-49ff-a53d-7be7f053cb81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:07:30.664917 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a930bdfc-172f-49ff-a53d-7be7f053cb81 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:07:30.665369 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a930bdfc-172f-49ff-a53d-7be7f053cb81 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:07:30.665642 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a930bdfc-172f-49ff-a53d-7be7f053cb81 None admin] PATH_INFO: `/v3/users/2d48d7cb2b294c6ea181c169ad83c520` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:07:30.679720 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a930bdfc-172f-49ff-a53d-7be7f053cb81 None admin] RBAC: Authorizing `identity:get_user(user_id=2d48d7cb2b294c6ea181c169ad83c520)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:07:30.681442 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a930bdfc-172f-49ff-a53d-7be7f053cb81 None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:07:30.686730 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 146/280] 10.222.0.18 () {56 vars in 1177 bytes} [Tue Sep 13 03:07:30 2022] GET /identity/v3/users/2d48d7cb2b294c6ea181c169ad83c520 => generated 293 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 03:07:30.689408 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fcc4383-2146-427f-a4fa-729499abcba7 None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:07:30.750575 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a930bdfc-172f-49ff-a53d-7be7f053cb81 None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:07:30.799239 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:07:30.827806 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fcc4383-2146-427f-a4fa-729499abcba7 None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:07:30.864980 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:07:30.881605 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00e57062-1b16-48d1-a8da-d1f51b6a3b5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:07:30.882545 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e57062-1b16-48d1-a8da-d1f51b6a3b5e None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:07:30.883007 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e57062-1b16-48d1-a8da-d1f51b6a3b5e None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:07:30.883459 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e57062-1b16-48d1-a8da-d1f51b6a3b5e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:07:30.884317 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00e57062-1b16-48d1-a8da-d1f51b6a3b5e None admin] RBAC: Adding query filter params (role.id=1eb7a5b5203945c29e6d22d653f47847, user.id=2d48d7cb2b294c6ea181c169ad83c520, scope.system=all) {{(pid=81399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 03:07:30.884762 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00e57062-1b16-48d1-a8da-d1f51b6a3b5e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:07:30.886749 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00e57062-1b16-48d1-a8da-d1f51b6a3b5e None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:07:30.889897 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a930bdfc-172f-49ff-a53d-7be7f053cb81 None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:07:30.894767 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 135/281] 10.222.0.18 () {56 vars in 1308 bytes} [Tue Sep 13 03:07:30 2022] GET /identity/v3/role_assignments?user.id=2d48d7cb2b294c6ea181c169ad83c520&scope.system=all&role.id=1eb7a5b5203945c29e6d22d653f47847 => generated 226 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 03:07:30.937618 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-199bc95a-cadf-437d-a491-6604c28a4dbd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:07:30.939152 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-199bc95a-cadf-437d-a491-6604c28a4dbd None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:07:30.939595 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-199bc95a-cadf-437d-a491-6604c28a4dbd None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:07:30.940012 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-199bc95a-cadf-437d-a491-6604c28a4dbd None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:07:30.952157 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-199bc95a-cadf-437d-a491-6604c28a4dbd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 03:07:30.958125 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application [None req-199bc95a-cadf-437d-a491-6604c28a4dbd None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 13 03:07:30.958125 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 03:07:30.958125 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 03:07:30.958125 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 03:07:30.958125 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 03:07:30.958125 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 03:07:30.958125 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 03:07:30.958125 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 03:07:30.958125 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 03:07:30.958125 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 03:07:30.958125 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 03:07:30.958125 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 03:07:30.958125 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 03:07:30.958125 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 03:07:30.958125 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 03:07:30.958125 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 03:07:30.959310 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 03:07:30.959310 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 03:07:30.959310 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 03:07:30.959310 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 03:07:30.959310 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 03:07:30.959310 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 03:07:30.959310 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 03:07:30.959310 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 03:07:30.959310 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 03:07:30.959310 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 03:07:30.959310 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 03:07:30.959310 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application with Lock( Sep 13 03:07:30.959310 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 03:07:30.959310 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._enter() Sep 13 03:07:30.959310 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 03:07:30.959310 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 03:07:30.960322 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 03:07:30.960322 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.creator() Sep 13 03:07:30.960322 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 03:07:30.960322 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application created_value = creator( Sep 13 03:07:30.960322 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 03:07:30.960322 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 03:07:30.960322 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 03:07:30.960322 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 03:07:30.960322 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 03:07:30.960322 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 03:07:30.960322 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 13 03:07:30.960322 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Sep 13 03:07:30.962714 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 147/282] 10.222.0.18 () {56 vars in 1099 bytes} [Tue Sep 13 03:07:30 2022] GET /identity/v3/roles/reader => generated 84 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 13 03:07:30.968780 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00e57062-1b16-48d1-a8da-d1f51b6a3b5e None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:07:31.091781 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00e57062-1b16-48d1-a8da-d1f51b6a3b5e None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:07:31.130781 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4980052-4e1b-46b8-a0be-116d29686ae4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:07:31.131448 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4980052-4e1b-46b8-a0be-116d29686ae4 None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:07:31.131778 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4980052-4e1b-46b8-a0be-116d29686ae4 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:07:31.131996 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4980052-4e1b-46b8-a0be-116d29686ae4 None admin] PATH_INFO: `/v3/roles` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:07:31.132654 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4980052-4e1b-46b8-a0be-116d29686ae4 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 03:07:31.132957 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4980052-4e1b-46b8-a0be-116d29686ae4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:07:31.134347 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4980052-4e1b-46b8-a0be-116d29686ae4 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:07:31.143667 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 136/283] 10.222.0.18 () {56 vars in 1101 bytes} [Tue Sep 13 03:07:30 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 03:07:31.149316 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-199bc95a-cadf-437d-a491-6604c28a4dbd None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:07:31.215294 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46196a25-cbcf-457e-b6d7-a6ed507faf5b None None] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:07:31.215617 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46196a25-cbcf-457e-b6d7-a6ed507faf5b None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:07:31.215885 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46196a25-cbcf-457e-b6d7-a6ed507faf5b None None] PATH_INFO: `/` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:07:31.220545 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 137/284] 10.222.0.18 () {54 vars in 927 bytes} [Tue Sep 13 03:07:31 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 03:07:31.226867 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d7cb1c5-162c-44c0-93a6-df328686fd57 None None] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:07:31.227151 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d7cb1c5-162c-44c0-93a6-df328686fd57 None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:07:31.227420 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d7cb1c5-162c-44c0-93a6-df328686fd57 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:07:31.251378 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:07:31.264663 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.auth.core [None req-2d7cb1c5-162c-44c0-93a6-df328686fd57 None None] MFA Rules not processed for user `49f280ab086b4610855c4ffa378517d9`. Rule list: `[]` (Enabled: `True`). {{(pid=81399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 03:07:31.269094 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-2d7cb1c5-162c-44c0-93a6-df328686fd57 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:07:31.313122 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-199bc95a-cadf-437d-a491-6604c28a4dbd None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:07:31.367750 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dba643d-d395-4c18-bba6-c0581dbea1ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:07:31.368383 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dba643d-d395-4c18-bba6-c0581dbea1ef None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:07:31.368656 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dba643d-d395-4c18-bba6-c0581dbea1ef None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:07:31.368925 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dba643d-d395-4c18-bba6-c0581dbea1ef None admin] PATH_INFO: `/v3/users/46f0efbf2eed4631b420782e6d2992b4` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:07:31.371059 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dba643d-d395-4c18-bba6-c0581dbea1ef None admin] RBAC: Authorizing `identity:get_user(user_id=46f0efbf2eed4631b420782e6d2992b4)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:07:31.372510 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dba643d-d395-4c18-bba6-c0581dbea1ef None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:07:31.375130 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 148/285] 10.222.0.18 () {56 vars in 1177 bytes} [Tue Sep 13 03:07:31 2022] GET /identity/v3/users/46f0efbf2eed4631b420782e6d2992b4 => generated 297 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 03:07:31.381504 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dba643d-d395-4c18-bba6-c0581dbea1ef None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:07:31.386211 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 138/286] 10.222.0.18 () {58 vars in 1024 bytes} [Tue Sep 13 03:07:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 13 03:07:31.399481 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:07:31.525744 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dba643d-d395-4c18-bba6-c0581dbea1ef None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:07:31.576243 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27780ecb-22eb-4a99-834f-9cd8afff336f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:07:31.576877 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27780ecb-22eb-4a99-834f-9cd8afff336f None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:07:31.577148 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27780ecb-22eb-4a99-834f-9cd8afff336f None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:07:31.577415 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27780ecb-22eb-4a99-834f-9cd8afff336f None admin] PATH_INFO: `/v3/projects/54999ea237f64c5f972ef79e765facd0` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:07:31.578541 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27780ecb-22eb-4a99-834f-9cd8afff336f None admin] RBAC: Authorizing `identity:get_project(project_id=54999ea237f64c5f972ef79e765facd0)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:07:31.580068 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27780ecb-22eb-4a99-834f-9cd8afff336f None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:07:31.581991 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 149/287] 10.222.0.18 () {56 vars in 1186 bytes} [Tue Sep 13 03:07:31 2022] GET /identity/v3/projects/54999ea237f64c5f972ef79e765facd0 => generated 302 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 03:07:31.587736 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d7cb1c5-162c-44c0-93a6-df328686fd57 None None] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:07:31.730535 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27780ecb-22eb-4a99-834f-9cd8afff336f None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:07:31.742393 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d7cb1c5-162c-44c0-93a6-df328686fd57 None None] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:07:31.817213 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:07:31.820250 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07135816-cf49-4abc-bb60-da9cebc94415 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:07:31.820933 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07135816-cf49-4abc-bb60-da9cebc94415 None admin] REQUEST_METHOD: `PUT` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:07:31.821219 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07135816-cf49-4abc-bb60-da9cebc94415 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:07:31.821485 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07135816-cf49-4abc-bb60-da9cebc94415 None admin] PATH_INFO: `/v3/projects/54999ea237f64c5f972ef79e765facd0/users/46f0efbf2eed4631b420782e6d2992b4/roles/502e6c11e747432988d7f723cee0af64` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:07:31.827890 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07135816-cf49-4abc-bb60-da9cebc94415 None admin] RBAC: Authorizing `identity:create_grant(project_id=54999ea237f64c5f972ef79e765facd0, user_id=46f0efbf2eed4631b420782e6d2992b4, role_id=502e6c11e747432988d7f723cee0af64)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:07:31.829407 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07135816-cf49-4abc-bb60-da9cebc94415 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:07:31.836037 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:07:31.849685 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27780ecb-22eb-4a99-834f-9cd8afff336f None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:07:31.854839 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 139/288] 10.222.0.18 () {58 vars in 1439 bytes} [Tue Sep 13 03:07:31 2022] PUT /identity/v3/projects/54999ea237f64c5f972ef79e765facd0/users/46f0efbf2eed4631b420782e6d2992b4/roles/502e6c11e747432988d7f723cee0af64 => generated 0 bytes in 267 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 03:07:31.890212 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bae5864-84c8-4e7d-a296-80dfb3359a2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:07:31.891242 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bae5864-84c8-4e7d-a296-80dfb3359a2b None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:07:31.891697 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bae5864-84c8-4e7d-a296-80dfb3359a2b None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:07:31.892116 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bae5864-84c8-4e7d-a296-80dfb3359a2b None admin] PATH_INFO: `/v3/roles/member` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:07:31.904422 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bae5864-84c8-4e7d-a296-80dfb3359a2b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 03:07:31.910465 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application [None req-1bae5864-84c8-4e7d-a296-80dfb3359a2b None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 13 03:07:31.910465 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 03:07:31.910465 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 03:07:31.910465 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 03:07:31.910465 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 03:07:31.910465 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 03:07:31.910465 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 03:07:31.910465 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 03:07:31.910465 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 03:07:31.910465 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 03:07:31.910465 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 03:07:31.910465 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 03:07:31.910465 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 03:07:31.910465 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 03:07:31.910465 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 03:07:31.910465 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 03:07:31.911639 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 03:07:31.911639 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 03:07:31.911639 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 03:07:31.911639 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 03:07:31.911639 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 03:07:31.911639 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 03:07:31.911639 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 03:07:31.911639 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 03:07:31.911639 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 03:07:31.911639 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 03:07:31.911639 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 03:07:31.911639 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application with Lock( Sep 13 03:07:31.911639 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 03:07:31.911639 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._enter() Sep 13 03:07:31.911639 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 03:07:31.911639 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 03:07:31.912651 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 03:07:31.912651 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.creator() Sep 13 03:07:31.912651 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 03:07:31.912651 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application created_value = creator( Sep 13 03:07:31.912651 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 03:07:31.912651 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 03:07:31.912651 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 03:07:31.912651 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 03:07:31.912651 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 03:07:31.912651 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 03:07:31.912651 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 13 03:07:31.912651 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Sep 13 03:07:31.915388 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 150/289] 10.222.0.18 () {56 vars in 1099 bytes} [Tue Sep 13 03:07:31 2022] GET /identity/v3/roles/member => generated 84 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 13 03:07:31.921591 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07135816-cf49-4abc-bb60-da9cebc94415 None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:07:32.083903 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07135816-cf49-4abc-bb60-da9cebc94415 None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:07:32.123774 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-066c007b-51d8-46fb-a46e-ab48984fa426 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:07:32.124395 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-066c007b-51d8-46fb-a46e-ab48984fa426 None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:07:32.124666 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-066c007b-51d8-46fb-a46e-ab48984fa426 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:07:32.124942 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-066c007b-51d8-46fb-a46e-ab48984fa426 None admin] PATH_INFO: `/v3/roles` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:07:32.125641 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-066c007b-51d8-46fb-a46e-ab48984fa426 None admin] RBAC: Adding query filter params (name=member) {{(pid=81399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 03:07:32.125948 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-066c007b-51d8-46fb-a46e-ab48984fa426 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:07:32.127412 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-066c007b-51d8-46fb-a46e-ab48984fa426 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:07:32.136156 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 140/290] 10.222.0.18 () {56 vars in 1101 bytes} [Tue Sep 13 03:07:31 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 03:07:32.143588 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bae5864-84c8-4e7d-a296-80dfb3359a2b None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:07:32.304662 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bae5864-84c8-4e7d-a296-80dfb3359a2b None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:07:32.350957 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e981c830-acae-4eb7-b3e1-d0e9bb6bd494 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:07:32.351803 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e981c830-acae-4eb7-b3e1-d0e9bb6bd494 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:07:32.352224 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e981c830-acae-4eb7-b3e1-d0e9bb6bd494 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:07:32.352638 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e981c830-acae-4eb7-b3e1-d0e9bb6bd494 None admin] PATH_INFO: `/v3/users/cbbc542ba2bb42deb9b1ad183d118075` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:07:32.353886 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e981c830-acae-4eb7-b3e1-d0e9bb6bd494 None admin] RBAC: Authorizing `identity:get_user(user_id=cbbc542ba2bb42deb9b1ad183d118075)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:07:32.355497 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e981c830-acae-4eb7-b3e1-d0e9bb6bd494 None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:07:32.357247 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 151/291] 10.222.0.18 () {56 vars in 1177 bytes} [Tue Sep 13 03:07:32 2022] GET /identity/v3/users/cbbc542ba2bb42deb9b1ad183d118075 => generated 297 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 03:07:32.362936 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-066c007b-51d8-46fb-a46e-ab48984fa426 None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:07:32.480212 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:07:32.531145 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-066c007b-51d8-46fb-a46e-ab48984fa426 None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:07:32.564602 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f98c6d40-b3c8-4322-af48-faf6b9300830 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:07:32.565217 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f98c6d40-b3c8-4322-af48-faf6b9300830 None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:07:32.565497 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f98c6d40-b3c8-4322-af48-faf6b9300830 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:07:32.565769 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f98c6d40-b3c8-4322-af48-faf6b9300830 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:07:32.652182 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f98c6d40-b3c8-4322-af48-faf6b9300830 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:07:32.653583 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f98c6d40-b3c8-4322-af48-faf6b9300830 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:07:32.731094 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:07:32.777029 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 141/292] 10.222.0.18 () {58 vars in 1282 bytes} [Tue Sep 13 03:07:32 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 414 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 13 03:07:32.782978 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e981c830-acae-4eb7-b3e1-d0e9bb6bd494 None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:07:32.786022 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:07:32.940377 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e981c830-acae-4eb7-b3e1-d0e9bb6bd494 None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:07:32.985607 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53fbd7d8-89c8-45ee-a8be-7d4bc6b9c342 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:07:32.986344 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53fbd7d8-89c8-45ee-a8be-7d4bc6b9c342 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:07:32.986637 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53fbd7d8-89c8-45ee-a8be-7d4bc6b9c342 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:07:32.986910 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53fbd7d8-89c8-45ee-a8be-7d4bc6b9c342 None admin] PATH_INFO: `/v3/projects/54999ea237f64c5f972ef79e765facd0` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:07:32.991290 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53fbd7d8-89c8-45ee-a8be-7d4bc6b9c342 None admin] RBAC: Authorizing `identity:get_project(project_id=54999ea237f64c5f972ef79e765facd0)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:07:32.992747 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53fbd7d8-89c8-45ee-a8be-7d4bc6b9c342 None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:07:32.994564 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 152/293] 10.222.0.18 () {56 vars in 1186 bytes} [Tue Sep 13 03:07:32 2022] GET /identity/v3/projects/54999ea237f64c5f972ef79e765facd0 => generated 302 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 03:07:33.001179 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f98c6d40-b3c8-4322-af48-faf6b9300830 None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:07:33.146476 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f98c6d40-b3c8-4322-af48-faf6b9300830 None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:07:33.187229 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13665aa8-efdf-42b6-bcaa-4d8d75e3c1ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:07:33.187857 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13665aa8-efdf-42b6-bcaa-4d8d75e3c1ab None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:07:33.188243 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13665aa8-efdf-42b6-bcaa-4d8d75e3c1ab None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:07:33.188515 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13665aa8-efdf-42b6-bcaa-4d8d75e3c1ab None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:07:33.189259 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13665aa8-efdf-42b6-bcaa-4d8d75e3c1ab None admin] RBAC: Adding query filter params (role.id=1eb7a5b5203945c29e6d22d653f47847, scope.project.id=54999ea237f64c5f972ef79e765facd0, user.id=cbbc542ba2bb42deb9b1ad183d118075) {{(pid=81399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 03:07:33.189563 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13665aa8-efdf-42b6-bcaa-4d8d75e3c1ab None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:07:33.191151 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13665aa8-efdf-42b6-bcaa-4d8d75e3c1ab None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:07:33.205137 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 142/294] 10.222.0.18 () {56 vars in 1374 bytes} [Tue Sep 13 03:07:32 2022] GET /identity/v3/role_assignments?user.id=cbbc542ba2bb42deb9b1ad183d118075&scope.project.id=54999ea237f64c5f972ef79e765facd0&role.id=1eb7a5b5203945c29e6d22d653f47847 => generated 608 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 03:07:33.401610 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e8c96e-3a44-4873-a812-7003323fc81c None None] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:07:33.402116 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e8c96e-3a44-4873-a812-7003323fc81c None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:07:33.402223 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e8c96e-3a44-4873-a812-7003323fc81c None None] PATH_INFO: `/` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:07:33.403917 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 153/295] 10.222.0.18 () {54 vars in 927 bytes} [Tue Sep 13 03:07:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 03:07:33.412340 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c52ddb5-6dea-4574-9259-c7ff918d0938 None None] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:07:33.412921 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c52ddb5-6dea-4574-9259-c7ff918d0938 None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:07:33.413388 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c52ddb5-6dea-4574-9259-c7ff918d0938 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:07:33.424771 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-825fa182-93d2-4b0e-aee8-0738c284230d None None] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:07:33.425102 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-825fa182-93d2-4b0e-aee8-0738c284230d None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:07:33.425324 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-825fa182-93d2-4b0e-aee8-0738c284230d None None] PATH_INFO: `/` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:07:33.427021 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 154/296] 10.222.0.18 () {54 vars in 927 bytes} [Tue Sep 13 03:07:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 03:07:33.437778 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf61fa5d-0db3-4669-82b0-fd2d7d2eb432 None None] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:07:33.438160 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf61fa5d-0db3-4669-82b0-fd2d7d2eb432 None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:07:33.438378 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf61fa5d-0db3-4669-82b0-fd2d7d2eb432 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:07:33.441474 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.auth.core [None req-0c52ddb5-6dea-4574-9259-c7ff918d0938 None None] MFA Rules not processed for user `49f280ab086b4610855c4ffa378517d9`. Rule list: `[]` (Enabled: `True`). {{(pid=81399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 03:07:33.443000 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-0c52ddb5-6dea-4574-9259-c7ff918d0938 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:07:33.473351 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.auth.core [None req-bf61fa5d-0db3-4669-82b0-fd2d7d2eb432 None None] MFA Rules not processed for user `49f280ab086b4610855c4ffa378517d9`. Rule list: `[]` (Enabled: `True`). {{(pid=81400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 03:07:33.475865 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-bf61fa5d-0db3-4669-82b0-fd2d7d2eb432 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:07:33.480740 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:07:33.554000 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 143/297] 10.222.0.18 () {58 vars in 1024 bytes} [Tue Sep 13 03:07:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 13 03:07:33.577823 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:07:33.590707 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 155/298] 10.222.0.18 () {58 vars in 1024 bytes} [Tue Sep 13 03:07:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 13 03:07:33.683253 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b2e8cb-e170-403d-89d5-8c1c37ef52f1 None None] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:07:33.683538 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b2e8cb-e170-403d-89d5-8c1c37ef52f1 None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:07:33.683815 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b2e8cb-e170-403d-89d5-8c1c37ef52f1 None None] PATH_INFO: `/` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:07:33.685423 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 144/299] 10.222.0.18 () {54 vars in 927 bytes} [Tue Sep 13 03:07:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 03:07:33.690592 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a4b69a-ed0c-4830-a3af-4d3f581f7762 None None] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:07:33.690907 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a4b69a-ed0c-4830-a3af-4d3f581f7762 None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:07:33.691178 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a4b69a-ed0c-4830-a3af-4d3f581f7762 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:07:33.710049 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.auth.core [None req-48a4b69a-ed0c-4830-a3af-4d3f581f7762 None None] MFA Rules not processed for user `49f280ab086b4610855c4ffa378517d9`. Rule list: `[]` (Enabled: `True`). {{(pid=81400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 03:07:33.711189 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-48a4b69a-ed0c-4830-a3af-4d3f581f7762 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:07:33.788238 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 156/300] 10.222.0.18 () {58 vars in 1024 bytes} [Tue Sep 13 03:07:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 13 03:07:33.867862 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86b2e8cb-e170-403d-89d5-8c1c37ef52f1 None None] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:07:33.891795 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48a4b69a-ed0c-4830-a3af-4d3f581f7762 None None] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:07:33.986242 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86b2e8cb-e170-403d-89d5-8c1c37ef52f1 None None] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:07:34.022252 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48a4b69a-ed0c-4830-a3af-4d3f581f7762 None None] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:07:34.029435 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0f7c9e1-eb6b-445e-a89b-bc352e13e70e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:07:34.030152 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f7c9e1-eb6b-445e-a89b-bc352e13e70e None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:07:34.030498 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f7c9e1-eb6b-445e-a89b-bc352e13e70e None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:07:34.030860 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f7c9e1-eb6b-445e-a89b-bc352e13e70e None admin] PATH_INFO: `/v3/domains/default` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:07:34.032608 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0f7c9e1-eb6b-445e-a89b-bc352e13e70e None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:07:34.034754 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0f7c9e1-eb6b-445e-a89b-bc352e13e70e None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:07:34.037422 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 145/301] 10.222.0.18 () {56 vars in 1108 bytes} [Tue Sep 13 03:07:33 2022] GET /identity/v3/domains/default => generated 199 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 03:07:34.043262 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0f7c9e1-eb6b-445e-a89b-bc352e13e70e None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:07:34.056686 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:07:34.065109 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e13e43c1-9be5-438f-a459-5045491d86ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:07:34.066053 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e13e43c1-9be5-438f-a459-5045491d86ef None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:07:34.066548 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e13e43c1-9be5-438f-a459-5045491d86ef None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:07:34.067029 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e13e43c1-9be5-438f-a459-5045491d86ef None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:07:34.071755 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:07:34.080638 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e13e43c1-9be5-438f-a459-5045491d86ef None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 03:07:34.086439 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application [None req-e13e43c1-9be5-438f-a459-5045491d86ef None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 13 03:07:34.086439 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 03:07:34.086439 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 03:07:34.086439 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 03:07:34.086439 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 03:07:34.086439 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 03:07:34.086439 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 03:07:34.086439 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 03:07:34.086439 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 03:07:34.086439 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 03:07:34.086439 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 03:07:34.086439 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 03:07:34.086439 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 03:07:34.086439 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 03:07:34.086439 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 03:07:34.086439 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 03:07:34.087362 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 03:07:34.087362 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 03:07:34.087362 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 03:07:34.087362 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 03:07:34.087362 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 03:07:34.087362 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 03:07:34.087362 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 03:07:34.087362 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 03:07:34.087362 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 03:07:34.087362 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 03:07:34.087362 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 03:07:34.087362 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application with Lock( Sep 13 03:07:34.087362 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 03:07:34.087362 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._enter() Sep 13 03:07:34.087362 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 03:07:34.087362 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 03:07:34.088179 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 03:07:34.088179 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.creator() Sep 13 03:07:34.088179 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 03:07:34.088179 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application created_value = creator( Sep 13 03:07:34.088179 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 03:07:34.088179 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 03:07:34.088179 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 03:07:34.088179 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 03:07:34.088179 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 03:07:34.088179 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 03:07:34.088179 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 13 03:07:34.088179 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Sep 13 03:07:34.090510 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 157/302] 10.222.0.18 () {56 vars in 1099 bytes} [Tue Sep 13 03:07:33 2022] GET /identity/v3/roles/reader => generated 84 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 13 03:07:34.096052 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e13e43c1-9be5-438f-a459-5045491d86ef None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:07:34.162247 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0f7c9e1-eb6b-445e-a89b-bc352e13e70e None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:07:34.196935 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38f3ff7f-1175-48ff-bcd2-9c32da501f05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:07:34.197536 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38f3ff7f-1175-48ff-bcd2-9c32da501f05 None admin] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:07:34.197811 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38f3ff7f-1175-48ff-bcd2-9c32da501f05 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:07:34.198321 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38f3ff7f-1175-48ff-bcd2-9c32da501f05 None admin] PATH_INFO: `/v3/groups` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:07:34.198728 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38f3ff7f-1175-48ff-bcd2-9c32da501f05 None admin] RBAC: Authorizing `identity:create_group()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:07:34.200151 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38f3ff7f-1175-48ff-bcd2-9c32da501f05 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:07:34.201176 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e13e43c1-9be5-438f-a459-5045491d86ef None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:07:34.220861 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 146/303] 10.222.0.18 () {60 vars in 1134 bytes} [Tue Sep 13 03:07:34 2022] POST /identity/v3/groups => generated 229 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 03:07:34.221986 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38f3ff7f-1175-48ff-bcd2-9c32da501f05 None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:07:34.238551 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-964d3248-97b6-43c3-a748-624f31cea7ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:07:34.239162 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-964d3248-97b6-43c3-a748-624f31cea7ef None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:07:34.239447 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-964d3248-97b6-43c3-a748-624f31cea7ef None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:07:34.239711 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-964d3248-97b6-43c3-a748-624f31cea7ef None admin] PATH_INFO: `/v3/roles/member` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:07:34.256948 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-964d3248-97b6-43c3-a748-624f31cea7ef None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 03:07:34.262156 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application [None req-964d3248-97b6-43c3-a748-624f31cea7ef None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 13 03:07:34.262156 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 03:07:34.262156 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 03:07:34.262156 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 03:07:34.262156 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 03:07:34.262156 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 03:07:34.262156 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 03:07:34.262156 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 03:07:34.262156 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 03:07:34.262156 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 03:07:34.262156 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 03:07:34.262156 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 03:07:34.262156 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 03:07:34.262156 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 03:07:34.262156 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 03:07:34.262156 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 03:07:34.263185 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 03:07:34.263185 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 03:07:34.263185 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 03:07:34.263185 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 03:07:34.263185 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 03:07:34.263185 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 03:07:34.263185 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 03:07:34.263185 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 03:07:34.263185 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 03:07:34.263185 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 03:07:34.263185 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 03:07:34.263185 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application with Lock( Sep 13 03:07:34.263185 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 03:07:34.263185 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._enter() Sep 13 03:07:34.263185 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 03:07:34.263185 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 03:07:34.264102 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 03:07:34.264102 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.creator() Sep 13 03:07:34.264102 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 03:07:34.264102 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application created_value = creator( Sep 13 03:07:34.264102 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 03:07:34.264102 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 03:07:34.264102 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 03:07:34.264102 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 03:07:34.264102 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 03:07:34.264102 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 03:07:34.264102 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 13 03:07:34.264102 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Sep 13 03:07:34.264102 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 158/304] 10.222.0.18 () {56 vars in 1099 bytes} [Tue Sep 13 03:07:34 2022] GET /identity/v3/roles/member => generated 84 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 13 03:07:34.269771 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-964d3248-97b6-43c3-a748-624f31cea7ef None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:07:34.333891 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:07:34.361550 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:07:34.392171 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38f3ff7f-1175-48ff-bcd2-9c32da501f05 None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:07:34.404701 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-964d3248-97b6-43c3-a748-624f31cea7ef None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:07:34.440399 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3df54a70-e011-45b4-8b84-a89f6e9dc7b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:07:34.441363 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df54a70-e011-45b4-8b84-a89f6e9dc7b6 None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:07:34.441644 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df54a70-e011-45b4-8b84-a89f6e9dc7b6 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:07:34.441915 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df54a70-e011-45b4-8b84-a89f6e9dc7b6 None admin] PATH_INFO: `/v3/roles` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:07:34.442638 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3df54a70-e011-45b4-8b84-a89f6e9dc7b6 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 03:07:34.442947 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3df54a70-e011-45b4-8b84-a89f6e9dc7b6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:07:34.444327 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3df54a70-e011-45b4-8b84-a89f6e9dc7b6 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:07:34.444536 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85844e98-5715-4c50-a772-88158a08b96a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:07:34.445142 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85844e98-5715-4c50-a772-88158a08b96a None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:07:34.445412 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85844e98-5715-4c50-a772-88158a08b96a None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:07:34.445781 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85844e98-5715-4c50-a772-88158a08b96a None admin] PATH_INFO: `/v3/roles` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:07:34.446446 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85844e98-5715-4c50-a772-88158a08b96a None admin] RBAC: Adding query filter params (name=member) {{(pid=81400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 03:07:34.446849 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85844e98-5715-4c50-a772-88158a08b96a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:07:34.450216 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85844e98-5715-4c50-a772-88158a08b96a None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:07:34.455673 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 147/305] 10.222.0.18 () {56 vars in 1101 bytes} [Tue Sep 13 03:07:34 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 03:07:34.461279 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 159/306] 10.222.0.18 () {56 vars in 1101 bytes} [Tue Sep 13 03:07:34 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 03:07:34.461843 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3df54a70-e011-45b4-8b84-a89f6e9dc7b6 None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:07:34.466651 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85844e98-5715-4c50-a772-88158a08b96a None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:07:34.573751 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3df54a70-e011-45b4-8b84-a89f6e9dc7b6 None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:07:34.576383 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85844e98-5715-4c50-a772-88158a08b96a None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:07:34.589196 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:07:34.611783 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4099ec56-7b15-49c4-91b3-361fc765bf40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:07:34.612422 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4099ec56-7b15-49c4-91b3-361fc765bf40 None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:07:34.612699 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4099ec56-7b15-49c4-91b3-361fc765bf40 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:07:34.612977 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4099ec56-7b15-49c4-91b3-361fc765bf40 None admin] PATH_INFO: `/v3/users/a8487a20e1a74e4b90d755457519f3ee` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:07:34.613335 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b103eb6-d17d-4e06-a4a1-7c0141708765 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:07:34.614142 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b103eb6-d17d-4e06-a4a1-7c0141708765 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:07:34.614596 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b103eb6-d17d-4e06-a4a1-7c0141708765 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:07:34.615088 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b103eb6-d17d-4e06-a4a1-7c0141708765 None admin] PATH_INFO: `/v3/users/2d48d7cb2b294c6ea181c169ad83c520` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:07:34.616428 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b103eb6-d17d-4e06-a4a1-7c0141708765 None admin] RBAC: Authorizing `identity:get_user(user_id=2d48d7cb2b294c6ea181c169ad83c520)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:07:34.618048 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b103eb6-d17d-4e06-a4a1-7c0141708765 None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:07:34.619878 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 160/307] 10.222.0.18 () {56 vars in 1177 bytes} [Tue Sep 13 03:07:34 2022] GET /identity/v3/users/2d48d7cb2b294c6ea181c169ad83c520 => generated 293 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 03:07:34.621193 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3398517b-a2e3-4bc7-ac4b-9b4add39c77a None None] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:07:34.621526 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3398517b-a2e3-4bc7-ac4b-9b4add39c77a None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:07:34.621802 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3398517b-a2e3-4bc7-ac4b-9b4add39c77a None None] PATH_INFO: `/` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:07:34.625064 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 161/308] 10.222.0.18 () {54 vars in 927 bytes} [Tue Sep 13 03:07:34 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 03:07:34.625185 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3398517b-a2e3-4bc7-ac4b-9b4add39c77a None None] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:07:34.632257 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:07:34.635240 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4099ec56-7b15-49c4-91b3-361fc765bf40 None admin] RBAC: Authorizing `identity:get_user(user_id=a8487a20e1a74e4b90d755457519f3ee)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:07:34.636823 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4099ec56-7b15-49c4-91b3-361fc765bf40 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:07:34.638952 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 148/309] 10.222.0.18 () {56 vars in 1177 bytes} [Tue Sep 13 03:07:34 2022] GET /identity/v3/users/a8487a20e1a74e4b90d755457519f3ee => generated 293 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 03:07:34.640415 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-801b4672-ce25-4415-9ee8-e75092c17da4 None None] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:07:34.641088 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-801b4672-ce25-4415-9ee8-e75092c17da4 None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:07:34.641358 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-801b4672-ce25-4415-9ee8-e75092c17da4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:07:34.661039 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.auth.core [None req-801b4672-ce25-4415-9ee8-e75092c17da4 None None] MFA Rules not processed for user `49f280ab086b4610855c4ffa378517d9`. Rule list: `[]` (Enabled: `True`). {{(pid=81399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 03:07:34.662126 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-801b4672-ce25-4415-9ee8-e75092c17da4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:07:34.742019 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3398517b-a2e3-4bc7-ac4b-9b4add39c77a None None] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:07:34.742403 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 149/310] 10.222.0.18 () {58 vars in 1024 bytes} [Tue Sep 13 03:07:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 13 03:07:34.743519 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-801b4672-ce25-4415-9ee8-e75092c17da4 None None] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:07:34.787825 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74486366-3c16-4e4e-ac27-991295089615 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:07:34.788572 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74486366-3c16-4e4e-ac27-991295089615 None admin] REQUEST_METHOD: `PUT` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:07:34.788913 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74486366-3c16-4e4e-ac27-991295089615 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:07:34.789268 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74486366-3c16-4e4e-ac27-991295089615 None admin] PATH_INFO: `/v3/system/users/2d48d7cb2b294c6ea181c169ad83c520/roles/1eb7a5b5203945c29e6d22d653f47847` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:07:34.790849 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74486366-3c16-4e4e-ac27-991295089615 None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=2d48d7cb2b294c6ea181c169ad83c520, role_id=1eb7a5b5203945c29e6d22d653f47847)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:07:34.792403 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74486366-3c16-4e4e-ac27-991295089615 None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:07:34.810651 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 162/311] 10.222.0.18 () {58 vars in 1334 bytes} [Tue Sep 13 03:07:34 2022] PUT /identity/v3/system/users/2d48d7cb2b294c6ea181c169ad83c520/roles/1eb7a5b5203945c29e6d22d653f47847 => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 03:07:34.867919 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-801b4672-ce25-4415-9ee8-e75092c17da4 None None] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:07:34.902330 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:07:34.910794 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27c2be7a-4766-4c0a-8ce6-7243760b6a37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:07:34.911476 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c2be7a-4766-4c0a-8ce6-7243760b6a37 None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:07:34.911759 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c2be7a-4766-4c0a-8ce6-7243760b6a37 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:07:34.912030 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c2be7a-4766-4c0a-8ce6-7243760b6a37 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:07:34.912762 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27c2be7a-4766-4c0a-8ce6-7243760b6a37 None admin] RBAC: Adding query filter params (role.id=502e6c11e747432988d7f723cee0af64, user.id=a8487a20e1a74e4b90d755457519f3ee, scope.system=all) {{(pid=81399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 03:07:34.913071 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27c2be7a-4766-4c0a-8ce6-7243760b6a37 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:07:34.914511 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27c2be7a-4766-4c0a-8ce6-7243760b6a37 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:07:34.920940 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 150/312] 10.222.0.18 () {56 vars in 1308 bytes} [Tue Sep 13 03:07:34 2022] GET /identity/v3/role_assignments?user.id=a8487a20e1a74e4b90d755457519f3ee&scope.system=all&role.id=502e6c11e747432988d7f723cee0af64 => generated 226 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 03:07:35.030494 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74486366-3c16-4e4e-ac27-991295089615 None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:07:35.087855 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:07:35.151097 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74486366-3c16-4e4e-ac27-991295089615 None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:07:35.186641 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c23b979a-1e50-43e6-8a0e-7cf283ca19e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:07:35.187310 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c23b979a-1e50-43e6-8a0e-7cf283ca19e8 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:07:35.187573 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c23b979a-1e50-43e6-8a0e-7cf283ca19e8 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:07:35.187842 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c23b979a-1e50-43e6-8a0e-7cf283ca19e8 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:07:35.199704 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c23b979a-1e50-43e6-8a0e-7cf283ca19e8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 03:07:35.206107 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application [None req-c23b979a-1e50-43e6-8a0e-7cf283ca19e8 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 13 03:07:35.206107 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 03:07:35.206107 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 03:07:35.206107 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 03:07:35.206107 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 03:07:35.206107 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 03:07:35.206107 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 03:07:35.206107 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 03:07:35.206107 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 03:07:35.206107 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 03:07:35.206107 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 03:07:35.206107 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 03:07:35.206107 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 03:07:35.206107 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 03:07:35.206107 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 03:07:35.206107 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 03:07:35.207129 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 03:07:35.207129 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 03:07:35.207129 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 03:07:35.207129 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 03:07:35.207129 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 03:07:35.207129 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 03:07:35.207129 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 03:07:35.207129 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 03:07:35.207129 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 03:07:35.207129 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 03:07:35.207129 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 03:07:35.207129 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application with Lock( Sep 13 03:07:35.207129 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 03:07:35.207129 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._enter() Sep 13 03:07:35.207129 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 03:07:35.207129 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 03:07:35.207925 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 03:07:35.207925 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.creator() Sep 13 03:07:35.207925 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 03:07:35.207925 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application created_value = creator( Sep 13 03:07:35.207925 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 03:07:35.207925 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 03:07:35.207925 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 03:07:35.207925 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 03:07:35.207925 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 03:07:35.207925 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 03:07:35.207925 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 13 03:07:35.207925 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Sep 13 03:07:35.207925 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 163/313] 10.222.0.18 () {56 vars in 1099 bytes} [Tue Sep 13 03:07:35 2022] GET /identity/v3/roles/reader => generated 84 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 03:07:35.213166 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27c2be7a-4766-4c0a-8ce6-7243760b6a37 None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:07:35.314507 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27c2be7a-4766-4c0a-8ce6-7243760b6a37 None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:07:35.355029 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4f5102e-a9fb-4ee0-b76b-7d775c3fa6eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:07:35.355642 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4f5102e-a9fb-4ee0-b76b-7d775c3fa6eb None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:07:35.355919 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4f5102e-a9fb-4ee0-b76b-7d775c3fa6eb None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:07:35.356188 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4f5102e-a9fb-4ee0-b76b-7d775c3fa6eb None admin] PATH_INFO: `/v3/roles` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:07:35.356848 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4f5102e-a9fb-4ee0-b76b-7d775c3fa6eb None admin] RBAC: Adding query filter params (name=reader) {{(pid=81399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 03:07:35.357150 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4f5102e-a9fb-4ee0-b76b-7d775c3fa6eb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:07:35.358517 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4f5102e-a9fb-4ee0-b76b-7d775c3fa6eb None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:07:35.367519 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 151/314] 10.222.0.18 () {56 vars in 1101 bytes} [Tue Sep 13 03:07:35 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 03:07:35.373034 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c23b979a-1e50-43e6-8a0e-7cf283ca19e8 None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:07:35.480228 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c23b979a-1e50-43e6-8a0e-7cf283ca19e8 None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:07:35.483457 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:07:35.516769 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac37736c-9960-449e-a3c1-158c4fe71a74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:07:35.517401 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac37736c-9960-449e-a3c1-158c4fe71a74 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:07:35.517736 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac37736c-9960-449e-a3c1-158c4fe71a74 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:07:35.518010 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac37736c-9960-449e-a3c1-158c4fe71a74 None admin] PATH_INFO: `/v3/users/46f0efbf2eed4631b420782e6d2992b4` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:07:35.519195 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac37736c-9960-449e-a3c1-158c4fe71a74 None admin] RBAC: Authorizing `identity:get_user(user_id=46f0efbf2eed4631b420782e6d2992b4)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:07:35.520691 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac37736c-9960-449e-a3c1-158c4fe71a74 None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:07:35.522658 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 164/315] 10.222.0.18 () {56 vars in 1177 bytes} [Tue Sep 13 03:07:35 2022] GET /identity/v3/users/46f0efbf2eed4631b420782e6d2992b4 => generated 297 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 03:07:35.528165 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4f5102e-a9fb-4ee0-b76b-7d775c3fa6eb None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:07:35.588955 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:07:35.635199 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4f5102e-a9fb-4ee0-b76b-7d775c3fa6eb None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:07:35.677702 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7b8e5dd-1ea6-4262-9807-948ea837ef4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:07:35.678373 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b8e5dd-1ea6-4262-9807-948ea837ef4c None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:07:35.678683 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b8e5dd-1ea6-4262-9807-948ea837ef4c None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:07:35.678959 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b8e5dd-1ea6-4262-9807-948ea837ef4c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:07:35.751270 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7b8e5dd-1ea6-4262-9807-948ea837ef4c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:07:35.752750 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7b8e5dd-1ea6-4262-9807-948ea837ef4c None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:07:35.810179 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:07:35.861413 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 152/316] 10.222.0.18 () {58 vars in 1282 bytes} [Tue Sep 13 03:07:35 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 334 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 13 03:07:35.866777 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac37736c-9960-449e-a3c1-158c4fe71a74 None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:07:35.976545 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac37736c-9960-449e-a3c1-158c4fe71a74 None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:07:36.010690 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce6e3aea-fda7-448d-ac9a-12766de06017 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:07:36.011243 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce6e3aea-fda7-448d-ac9a-12766de06017 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:07:36.011516 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce6e3aea-fda7-448d-ac9a-12766de06017 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:07:36.011784 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce6e3aea-fda7-448d-ac9a-12766de06017 None admin] PATH_INFO: `/v3/projects/54999ea237f64c5f972ef79e765facd0` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:07:36.012883 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce6e3aea-fda7-448d-ac9a-12766de06017 None admin] RBAC: Authorizing `identity:get_project(project_id=54999ea237f64c5f972ef79e765facd0)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:07:36.014331 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce6e3aea-fda7-448d-ac9a-12766de06017 None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:07:36.016928 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 165/317] 10.222.0.18 () {56 vars in 1186 bytes} [Tue Sep 13 03:07:35 2022] GET /identity/v3/projects/54999ea237f64c5f972ef79e765facd0 => generated 302 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 03:07:36.027701 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7b8e5dd-1ea6-4262-9807-948ea837ef4c None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:07:36.140606 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7b8e5dd-1ea6-4262-9807-948ea837ef4c None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:07:36.176100 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97c20b92-7bc4-492d-ae04-fff376b42b4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:07:36.176702 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c20b92-7bc4-492d-ae04-fff376b42b4d None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:07:36.176980 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c20b92-7bc4-492d-ae04-fff376b42b4d None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:07:36.177246 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c20b92-7bc4-492d-ae04-fff376b42b4d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:07:36.177960 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97c20b92-7bc4-492d-ae04-fff376b42b4d None admin] RBAC: Adding query filter params (role.id=502e6c11e747432988d7f723cee0af64, scope.project.id=54999ea237f64c5f972ef79e765facd0, user.id=46f0efbf2eed4631b420782e6d2992b4) {{(pid=81399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 03:07:36.178263 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97c20b92-7bc4-492d-ae04-fff376b42b4d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:07:36.179704 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97c20b92-7bc4-492d-ae04-fff376b42b4d None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:07:36.186158 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 153/318] 10.222.0.18 () {56 vars in 1374 bytes} [Tue Sep 13 03:07:36 2022] GET /identity/v3/role_assignments?user.id=46f0efbf2eed4631b420782e6d2992b4&scope.project.id=54999ea237f64c5f972ef79e765facd0&role.id=502e6c11e747432988d7f723cee0af64 => generated 608 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 03:07:36.503849 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83b31e13-4637-4af1-b589-8ba31504b1b7 None None] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:07:36.504495 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83b31e13-4637-4af1-b589-8ba31504b1b7 None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:07:36.504495 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83b31e13-4637-4af1-b589-8ba31504b1b7 None None] PATH_INFO: `/` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:07:36.505862 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 166/319] 10.222.0.18 () {54 vars in 927 bytes} [Tue Sep 13 03:07:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 03:07:36.510999 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86619129-97be-4831-ae21-a3af77d3c44c None None] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:07:36.511291 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86619129-97be-4831-ae21-a3af77d3c44c None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:07:36.511568 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86619129-97be-4831-ae21-a3af77d3c44c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:07:36.530312 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.auth.core [None req-86619129-97be-4831-ae21-a3af77d3c44c None None] MFA Rules not processed for user `49f280ab086b4610855c4ffa378517d9`. Rule list: `[]` (Enabled: `True`). {{(pid=81399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 03:07:36.531531 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-86619129-97be-4831-ae21-a3af77d3c44c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:07:36.546060 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:07:36.600794 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 154/320] 10.222.0.18 () {58 vars in 1024 bytes} [Tue Sep 13 03:07:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 13 03:07:36.868450 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83b31e13-4637-4af1-b589-8ba31504b1b7 None None] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:07:36.911815 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:07:36.978214 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83b31e13-4637-4af1-b589-8ba31504b1b7 None None] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:07:37.016338 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-310ef7dd-1230-4bdd-9c8c-c58e82bd98c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:07:37.016962 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-310ef7dd-1230-4bdd-9c8c-c58e82bd98c6 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:07:37.017234 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-310ef7dd-1230-4bdd-9c8c-c58e82bd98c6 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:07:37.017503 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-310ef7dd-1230-4bdd-9c8c-c58e82bd98c6 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:07:37.018586 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-310ef7dd-1230-4bdd-9c8c-c58e82bd98c6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:07:37.020073 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-310ef7dd-1230-4bdd-9c8c-c58e82bd98c6 None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:07:37.021780 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 167/321] 10.222.0.18 () {56 vars in 1108 bytes} [Tue Sep 13 03:07:36 2022] GET /identity/v3/domains/default => generated 199 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 03:07:37.027140 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86619129-97be-4831-ae21-a3af77d3c44c None None] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:07:37.056338 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f972b5a-7dc8-41a3-9fbb-74749d40b4b9 None None] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:07:37.056633 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f972b5a-7dc8-41a3-9fbb-74749d40b4b9 None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:07:37.056877 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f972b5a-7dc8-41a3-9fbb-74749d40b4b9 None None] PATH_INFO: `/` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:07:37.060429 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 168/322] 10.222.0.18 () {54 vars in 927 bytes} [Tue Sep 13 03:07:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 03:07:37.063241 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0d3fc25-8126-4144-87b9-b67ee94354c6 None None] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:07:37.063522 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0d3fc25-8126-4144-87b9-b67ee94354c6 None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:07:37.063789 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0d3fc25-8126-4144-87b9-b67ee94354c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:07:37.081828 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.auth.core [None req-a0d3fc25-8126-4144-87b9-b67ee94354c6 None None] MFA Rules not processed for user `49f280ab086b4610855c4ffa378517d9`. Rule list: `[]` (Enabled: `True`). {{(pid=81400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 03:07:37.083035 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-a0d3fc25-8126-4144-87b9-b67ee94354c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:07:37.148294 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86619129-97be-4831-ae21-a3af77d3c44c None None] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:07:37.157207 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 169/323] 10.222.0.18 () {58 vars in 1024 bytes} [Tue Sep 13 03:07:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 13 03:07:37.184066 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e0beeed-ebfa-43da-b94e-bce185723c72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:07:37.184679 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e0beeed-ebfa-43da-b94e-bce185723c72 None admin] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:07:37.184981 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e0beeed-ebfa-43da-b94e-bce185723c72 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:07:37.185260 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e0beeed-ebfa-43da-b94e-bce185723c72 None admin] PATH_INFO: `/v3/groups` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:07:37.185881 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e0beeed-ebfa-43da-b94e-bce185723c72 None admin] RBAC: Authorizing `identity:create_group()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:07:37.187344 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e0beeed-ebfa-43da-b94e-bce185723c72 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:07:37.202083 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 155/324] 10.222.0.18 () {60 vars in 1134 bytes} [Tue Sep 13 03:07:37 2022] POST /identity/v3/groups => generated 226 bytes in 175 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 13 03:07:37.228437 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c15a89ce-f724-4e38-910d-9065d02de2f5 None None] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:07:37.228753 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c15a89ce-f724-4e38-910d-9065d02de2f5 None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:07:37.228989 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c15a89ce-f724-4e38-910d-9065d02de2f5 None None] PATH_INFO: `/` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:07:37.230450 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 170/325] 10.222.0.18 () {54 vars in 927 bytes} [Tue Sep 13 03:07:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 03:07:37.237080 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aed67a4-c1f4-4799-9651-1ebdf9bf6d3e None None] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:07:37.237374 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aed67a4-c1f4-4799-9651-1ebdf9bf6d3e None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:07:37.237629 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aed67a4-c1f4-4799-9651-1ebdf9bf6d3e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:07:37.256645 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.auth.core [None req-0aed67a4-c1f4-4799-9651-1ebdf9bf6d3e None None] MFA Rules not processed for user `49f280ab086b4610855c4ffa378517d9`. Rule list: `[]` (Enabled: `True`). {{(pid=81399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 03:07:37.257908 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-0aed67a4-c1f4-4799-9651-1ebdf9bf6d3e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:07:37.262494 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:07:37.337050 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 156/326] 10.222.0.18 () {58 vars in 1024 bytes} [Tue Sep 13 03:07:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 13 03:07:37.432626 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c15a89ce-f724-4e38-910d-9065d02de2f5 None None] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:07:37.475025 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:07:37.539308 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c15a89ce-f724-4e38-910d-9065d02de2f5 None None] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:07:37.574435 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0226a9a-9941-46bb-b9f7-021defcd14e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:07:37.575155 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0226a9a-9941-46bb-b9f7-021defcd14e7 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:07:37.575430 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0226a9a-9941-46bb-b9f7-021defcd14e7 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:07:37.575700 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0226a9a-9941-46bb-b9f7-021defcd14e7 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:07:37.587115 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0226a9a-9941-46bb-b9f7-021defcd14e7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 03:07:37.592237 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application [None req-d0226a9a-9941-46bb-b9f7-021defcd14e7 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 13 03:07:37.592237 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 03:07:37.592237 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 03:07:37.592237 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 03:07:37.592237 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 03:07:37.592237 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 03:07:37.592237 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 03:07:37.592237 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 03:07:37.592237 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 03:07:37.592237 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 03:07:37.592237 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 03:07:37.592237 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 03:07:37.592237 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 03:07:37.592237 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 03:07:37.592237 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 03:07:37.592237 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 03:07:37.593136 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 03:07:37.593136 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 03:07:37.593136 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 03:07:37.593136 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 03:07:37.593136 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 03:07:37.593136 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 03:07:37.593136 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 03:07:37.593136 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 03:07:37.593136 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 03:07:37.593136 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 03:07:37.593136 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 03:07:37.593136 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application with Lock( Sep 13 03:07:37.593136 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 03:07:37.593136 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._enter() Sep 13 03:07:37.593136 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 03:07:37.593136 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 03:07:37.595515 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 03:07:37.595515 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.creator() Sep 13 03:07:37.595515 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 03:07:37.595515 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application created_value = creator( Sep 13 03:07:37.595515 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 03:07:37.595515 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 03:07:37.595515 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 03:07:37.595515 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 03:07:37.595515 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 03:07:37.595515 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 03:07:37.595515 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 13 03:07:37.595515 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Sep 13 03:07:37.595515 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 171/327] 10.222.0.18 () {56 vars in 1099 bytes} [Tue Sep 13 03:07:37 2022] GET /identity/v3/roles/member => generated 84 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 03:07:37.600159 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aed67a4-c1f4-4799-9651-1ebdf9bf6d3e None None] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:07:37.605713 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0226a9a-9941-46bb-b9f7-021defcd14e7 None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:07:37.725337 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:07:37.729463 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0226a9a-9941-46bb-b9f7-021defcd14e7 None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:07:37.746033 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aed67a4-c1f4-4799-9651-1ebdf9bf6d3e None None] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:07:37.777424 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dca133d1-ccb4-43e1-9694-80c204f45405 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:07:37.780875 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dca133d1-ccb4-43e1-9694-80c204f45405 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:07:37.781352 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dca133d1-ccb4-43e1-9694-80c204f45405 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:07:37.781844 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dca133d1-ccb4-43e1-9694-80c204f45405 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:07:37.792355 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bafd1d2f-5dd1-4534-b7d5-b288abfe274b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:07:37.793009 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bafd1d2f-5dd1-4534-b7d5-b288abfe274b None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:07:37.793351 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bafd1d2f-5dd1-4534-b7d5-b288abfe274b None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:07:37.793691 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bafd1d2f-5dd1-4534-b7d5-b288abfe274b None admin] PATH_INFO: `/v3/roles` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:07:37.794466 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bafd1d2f-5dd1-4534-b7d5-b288abfe274b None admin] RBAC: Adding query filter params (name=member) {{(pid=81399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 03:07:37.794883 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bafd1d2f-5dd1-4534-b7d5-b288abfe274b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:07:37.796323 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bafd1d2f-5dd1-4534-b7d5-b288abfe274b None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:07:37.796528 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dca133d1-ccb4-43e1-9694-80c204f45405 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 03:07:37.803193 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application [None req-dca133d1-ccb4-43e1-9694-80c204f45405 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 13 03:07:37.803193 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 03:07:37.803193 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 03:07:37.803193 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 03:07:37.803193 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 03:07:37.803193 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 03:07:37.803193 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 03:07:37.803193 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 03:07:37.803193 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 03:07:37.803193 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 03:07:37.803193 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 03:07:37.803193 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 03:07:37.803193 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 03:07:37.803193 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 03:07:37.803193 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 03:07:37.803193 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 03:07:37.804097 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 03:07:37.804097 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 03:07:37.804097 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 03:07:37.804097 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 03:07:37.804097 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 03:07:37.804097 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 03:07:37.804097 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 03:07:37.804097 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 03:07:37.804097 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 03:07:37.804097 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 03:07:37.804097 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 03:07:37.804097 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application with Lock( Sep 13 03:07:37.804097 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 03:07:37.804097 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._enter() Sep 13 03:07:37.804097 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 03:07:37.804097 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 03:07:37.804906 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 03:07:37.804906 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.creator() Sep 13 03:07:37.804906 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 03:07:37.804906 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application created_value = creator( Sep 13 03:07:37.804906 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 03:07:37.804906 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 03:07:37.804906 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 03:07:37.804906 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 03:07:37.804906 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 03:07:37.804906 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 03:07:37.804906 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 13 03:07:37.804906 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Sep 13 03:07:37.812183 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 157/329] 10.222.0.18 () {56 vars in 1101 bytes} [Tue Sep 13 03:07:37 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 03:07:37.812470 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 172/329] 10.222.0.18 () {56 vars in 1099 bytes} [Tue Sep 13 03:07:37 2022] GET /identity/v3/roles/reader => generated 84 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 13 03:07:37.815618 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bafd1d2f-5dd1-4534-b7d5-b288abfe274b None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:07:37.816844 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dca133d1-ccb4-43e1-9694-80c204f45405 None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:07:37.819099 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:07:37.938794 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bafd1d2f-5dd1-4534-b7d5-b288abfe274b None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:07:37.947354 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dca133d1-ccb4-43e1-9694-80c204f45405 None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:07:37.975797 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-215c618d-24e8-4bc4-bf7e-f6145e09369d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:07:37.976774 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-215c618d-24e8-4bc4-bf7e-f6145e09369d None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:07:37.977254 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-215c618d-24e8-4bc4-bf7e-f6145e09369d None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:07:37.977726 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-215c618d-24e8-4bc4-bf7e-f6145e09369d None admin] PATH_INFO: `/v3/users/2d48d7cb2b294c6ea181c169ad83c520` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:07:37.979103 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-215c618d-24e8-4bc4-bf7e-f6145e09369d None admin] RBAC: Authorizing `identity:get_user(user_id=2d48d7cb2b294c6ea181c169ad83c520)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:07:37.980735 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-215c618d-24e8-4bc4-bf7e-f6145e09369d None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:07:37.983042 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 158/330] 10.222.0.18 () {56 vars in 1177 bytes} [Tue Sep 13 03:07:37 2022] GET /identity/v3/users/2d48d7cb2b294c6ea181c169ad83c520 => generated 293 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 03:07:37.985292 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d8fd3d0-3043-45f7-ac9f-d894198bdde5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:07:37.985948 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d8fd3d0-3043-45f7-ac9f-d894198bdde5 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:07:37.986225 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d8fd3d0-3043-45f7-ac9f-d894198bdde5 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:07:37.986492 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d8fd3d0-3043-45f7-ac9f-d894198bdde5 None admin] PATH_INFO: `/v3/roles` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:07:37.987209 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d8fd3d0-3043-45f7-ac9f-d894198bdde5 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 03:07:37.987743 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d8fd3d0-3043-45f7-ac9f-d894198bdde5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:07:37.988643 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-215c618d-24e8-4bc4-bf7e-f6145e09369d None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:07:37.989156 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d8fd3d0-3043-45f7-ac9f-d894198bdde5 None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:07:37.998909 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 173/331] 10.222.0.18 () {56 vars in 1101 bytes} [Tue Sep 13 03:07:37 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 03:07:38.005012 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d8fd3d0-3043-45f7-ac9f-d894198bdde5 None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:07:38.032375 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:07:38.080329 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:07:38.126091 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-215c618d-24e8-4bc4-bf7e-f6145e09369d None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:07:38.148489 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d8fd3d0-3043-45f7-ac9f-d894198bdde5 None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:07:38.162716 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6d47dc5-e293-405f-b916-749eade7f81f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:07:38.163648 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d47dc5-e293-405f-b916-749eade7f81f None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:07:38.164115 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d47dc5-e293-405f-b916-749eade7f81f None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:07:38.164574 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d47dc5-e293-405f-b916-749eade7f81f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:07:38.165487 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6d47dc5-e293-405f-b916-749eade7f81f None admin] RBAC: Adding query filter params (role.id=1eb7a5b5203945c29e6d22d653f47847, user.id=2d48d7cb2b294c6ea181c169ad83c520, scope.system=all) {{(pid=81399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 03:07:38.166024 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6d47dc5-e293-405f-b916-749eade7f81f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:07:38.167686 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6d47dc5-e293-405f-b916-749eade7f81f None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:07:38.181690 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 159/332] 10.222.0.18 () {56 vars in 1308 bytes} [Tue Sep 13 03:07:37 2022] GET /identity/v3/role_assignments?user.id=2d48d7cb2b294c6ea181c169ad83c520&scope.system=all&role.id=1eb7a5b5203945c29e6d22d653f47847 => generated 510 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 03:07:38.192519 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9885b314-159f-4b6c-8ce6-3b8450545e2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:07:38.193184 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9885b314-159f-4b6c-8ce6-3b8450545e2a None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:07:38.193466 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9885b314-159f-4b6c-8ce6-3b8450545e2a None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:07:38.193739 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9885b314-159f-4b6c-8ce6-3b8450545e2a None admin] PATH_INFO: `/v3/users/a8487a20e1a74e4b90d755457519f3ee` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:07:38.199207 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9885b314-159f-4b6c-8ce6-3b8450545e2a None admin] RBAC: Authorizing `identity:get_user(user_id=a8487a20e1a74e4b90d755457519f3ee)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:07:38.201218 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9885b314-159f-4b6c-8ce6-3b8450545e2a None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:07:38.203617 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 174/333] 10.222.0.18 () {56 vars in 1177 bytes} [Tue Sep 13 03:07:38 2022] GET /identity/v3/users/a8487a20e1a74e4b90d755457519f3ee => generated 293 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 03:07:38.209782 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6d47dc5-e293-405f-b916-749eade7f81f None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:07:38.326834 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6d47dc5-e293-405f-b916-749eade7f81f None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:07:38.356749 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:07:38.368288 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4de004f4-a2a2-449c-8bb6-aa42b3b0c44e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:07:38.369436 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de004f4-a2a2-449c-8bb6-aa42b3b0c44e None admin] REQUEST_METHOD: `PUT` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:07:38.369925 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de004f4-a2a2-449c-8bb6-aa42b3b0c44e None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:07:38.370418 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de004f4-a2a2-449c-8bb6-aa42b3b0c44e None admin] PATH_INFO: `/v3/system/users/a8487a20e1a74e4b90d755457519f3ee/roles/502e6c11e747432988d7f723cee0af64` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:07:38.372335 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4de004f4-a2a2-449c-8bb6-aa42b3b0c44e None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=a8487a20e1a74e4b90d755457519f3ee, role_id=502e6c11e747432988d7f723cee0af64)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:07:38.373978 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4de004f4-a2a2-449c-8bb6-aa42b3b0c44e None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:07:38.395014 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 160/334] 10.222.0.18 () {58 vars in 1334 bytes} [Tue Sep 13 03:07:38 2022] PUT /identity/v3/system/users/a8487a20e1a74e4b90d755457519f3ee/roles/502e6c11e747432988d7f723cee0af64 => generated 0 bytes in 186 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 03:07:39.944561 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d519533-4d9d-4623-bc2e-8b2ab19ea274 None None] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:07:39.945547 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d519533-4d9d-4623-bc2e-8b2ab19ea274 None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:07:39.946023 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d519533-4d9d-4623-bc2e-8b2ab19ea274 None None] PATH_INFO: `/` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:07:39.947864 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 175/335] 10.222.0.18 () {54 vars in 927 bytes} [Tue Sep 13 03:07:39 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 03:07:39.952987 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e02be0e1-128c-4761-979f-eaf8064cb974 None None] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:07:39.954360 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e02be0e1-128c-4761-979f-eaf8064cb974 None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:07:39.954847 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e02be0e1-128c-4761-979f-eaf8064cb974 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:07:39.977836 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.auth.core [None req-e02be0e1-128c-4761-979f-eaf8064cb974 None None] MFA Rules not processed for user `49f280ab086b4610855c4ffa378517d9`. Rule list: `[]` (Enabled: `True`). {{(pid=81399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 03:07:39.979375 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-e02be0e1-128c-4761-979f-eaf8064cb974 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:07:40.077982 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 161/336] 10.222.0.18 () {58 vars in 1024 bytes} [Tue Sep 13 03:07:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 13 03:07:40.370781 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ff6808-7739-4a2b-af50-3f36956b7d89 None None] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:07:40.371423 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ff6808-7739-4a2b-af50-3f36956b7d89 None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:07:40.371892 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ff6808-7739-4a2b-af50-3f36956b7d89 None None] PATH_INFO: `/` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:07:40.373687 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 176/337] 10.222.0.18 () {54 vars in 927 bytes} [Tue Sep 13 03:07:40 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 03:07:40.379577 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1259d3a-158c-4ce3-9a8b-f76edfdb5b0e None None] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:07:40.380109 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1259d3a-158c-4ce3-9a8b-f76edfdb5b0e None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:07:40.380586 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1259d3a-158c-4ce3-9a8b-f76edfdb5b0e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:07:40.403229 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.auth.core [None req-e1259d3a-158c-4ce3-9a8b-f76edfdb5b0e None None] MFA Rules not processed for user `49f280ab086b4610855c4ffa378517d9`. Rule list: `[]` (Enabled: `True`). {{(pid=81399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 03:07:40.404701 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-e1259d3a-158c-4ce3-9a8b-f76edfdb5b0e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:07:40.488557 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 162/338] 10.222.0.18 () {58 vars in 1024 bytes} [Tue Sep 13 03:07:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 13 03:07:40.508239 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9ff6808-7739-4a2b-af50-3f36956b7d89 None None] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:07:40.617663 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:07:40.632764 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9ff6808-7739-4a2b-af50-3f36956b7d89 None None] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:07:40.673850 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26395c7b-0a52-40ee-940c-5f3dbd666c22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:07:40.674530 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26395c7b-0a52-40ee-940c-5f3dbd666c22 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:07:40.674831 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26395c7b-0a52-40ee-940c-5f3dbd666c22 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:07:40.675097 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26395c7b-0a52-40ee-940c-5f3dbd666c22 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:07:40.686103 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26395c7b-0a52-40ee-940c-5f3dbd666c22 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 03:07:40.691564 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application [None req-26395c7b-0a52-40ee-940c-5f3dbd666c22 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 13 03:07:40.691564 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 03:07:40.691564 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 03:07:40.691564 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 03:07:40.691564 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 03:07:40.691564 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 03:07:40.691564 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 03:07:40.691564 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 03:07:40.691564 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 03:07:40.691564 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 03:07:40.691564 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 03:07:40.691564 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 03:07:40.691564 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 03:07:40.691564 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 03:07:40.691564 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 03:07:40.691564 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 03:07:40.692466 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 03:07:40.692466 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 03:07:40.692466 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 03:07:40.692466 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 03:07:40.692466 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 03:07:40.692466 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 03:07:40.692466 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 03:07:40.692466 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 03:07:40.692466 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 03:07:40.692466 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 03:07:40.692466 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 03:07:40.692466 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application with Lock( Sep 13 03:07:40.692466 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 03:07:40.692466 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._enter() Sep 13 03:07:40.692466 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 03:07:40.692466 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 03:07:40.693267 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 03:07:40.693267 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.creator() Sep 13 03:07:40.693267 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 03:07:40.693267 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application created_value = creator( Sep 13 03:07:40.693267 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 03:07:40.693267 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 03:07:40.693267 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 03:07:40.693267 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 03:07:40.693267 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 03:07:40.693267 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 03:07:40.693267 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 13 03:07:40.693267 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Sep 13 03:07:40.693267 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 177/339] 10.222.0.18 () {56 vars in 1099 bytes} [Tue Sep 13 03:07:40 2022] GET /identity/v3/roles/member => generated 84 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 03:07:40.700321 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1259d3a-158c-4ce3-9a8b-f76edfdb5b0e None None] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:07:40.761114 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:07:40.812133 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1259d3a-158c-4ce3-9a8b-f76edfdb5b0e None None] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:07:40.852597 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06f01c07-07f2-4ca1-b3c0-ebee36462c43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:07:40.853588 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f01c07-07f2-4ca1-b3c0-ebee36462c43 None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:07:40.854040 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f01c07-07f2-4ca1-b3c0-ebee36462c43 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:07:40.854481 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f01c07-07f2-4ca1-b3c0-ebee36462c43 None admin] PATH_INFO: `/v3/roles` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:07:40.855334 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06f01c07-07f2-4ca1-b3c0-ebee36462c43 None admin] RBAC: Adding query filter params (name=member) {{(pid=81399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 03:07:40.855831 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06f01c07-07f2-4ca1-b3c0-ebee36462c43 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:07:40.857393 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06f01c07-07f2-4ca1-b3c0-ebee36462c43 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:07:40.870522 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 163/340] 10.222.0.18 () {56 vars in 1101 bytes} [Tue Sep 13 03:07:40 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 03:07:40.876011 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26395c7b-0a52-40ee-940c-5f3dbd666c22 None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:07:40.900328 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06f01c07-07f2-4ca1-b3c0-ebee36462c43 None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:07:41.037951 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06f01c07-07f2-4ca1-b3c0-ebee36462c43 None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:07:41.046824 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26395c7b-0a52-40ee-940c-5f3dbd666c22 None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:07:41.056547 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:07:41.094322 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68c6aacc-7519-4e7a-937b-5ad326ae64b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:07:41.095353 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c6aacc-7519-4e7a-937b-5ad326ae64b6 None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:07:41.095837 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c6aacc-7519-4e7a-937b-5ad326ae64b6 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:07:41.096281 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c6aacc-7519-4e7a-937b-5ad326ae64b6 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:07:41.108655 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68c6aacc-7519-4e7a-937b-5ad326ae64b6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 03:07:41.115739 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application [None req-68c6aacc-7519-4e7a-937b-5ad326ae64b6 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 13 03:07:41.115739 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 03:07:41.115739 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 03:07:41.115739 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 03:07:41.115739 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 03:07:41.115739 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 03:07:41.115739 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 03:07:41.115739 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 03:07:41.115739 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 03:07:41.115739 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 03:07:41.115739 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 03:07:41.115739 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 03:07:41.115739 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 03:07:41.115739 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 03:07:41.115739 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 03:07:41.115739 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 03:07:41.116666 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 03:07:41.116666 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 03:07:41.116666 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 03:07:41.116666 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 03:07:41.116666 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 03:07:41.116666 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 03:07:41.116666 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 03:07:41.116666 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 03:07:41.116666 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 03:07:41.116666 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 03:07:41.116666 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 03:07:41.116666 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application with Lock( Sep 13 03:07:41.116666 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 03:07:41.116666 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._enter() Sep 13 03:07:41.116666 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 03:07:41.116666 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 03:07:41.117463 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 03:07:41.117463 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.creator() Sep 13 03:07:41.117463 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 03:07:41.117463 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application created_value = creator( Sep 13 03:07:41.117463 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 03:07:41.117463 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 03:07:41.117463 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 03:07:41.117463 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 03:07:41.117463 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 03:07:41.117463 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 03:07:41.117463 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 13 03:07:41.117463 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Sep 13 03:07:41.119562 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 164/341] 10.222.0.18 () {56 vars in 1114 bytes} [Tue Sep 13 03:07:40 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 13 03:07:41.126145 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68c6aacc-7519-4e7a-937b-5ad326ae64b6 None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:07:41.130730 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c284c0e9-d4a4-4a10-8a10-f4b39fde629c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:07:41.131497 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c284c0e9-d4a4-4a10-8a10-f4b39fde629c None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:07:41.132086 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c284c0e9-d4a4-4a10-8a10-f4b39fde629c None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:07:41.133219 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c284c0e9-d4a4-4a10-8a10-f4b39fde629c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:07:41.171077 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:07:41.275687 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c284c0e9-d4a4-4a10-8a10-f4b39fde629c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:07:41.278682 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c284c0e9-d4a4-4a10-8a10-f4b39fde629c None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:07:41.319212 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68c6aacc-7519-4e7a-937b-5ad326ae64b6 None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:07:41.390963 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4b15b9c-f8c7-4f45-828c-17d7ebd1a36e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:07:41.392493 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b15b9c-f8c7-4f45-828c-17d7ebd1a36e None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:07:41.393564 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b15b9c-f8c7-4f45-828c-17d7ebd1a36e None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:07:41.394576 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b15b9c-f8c7-4f45-828c-17d7ebd1a36e None admin] PATH_INFO: `/v3/roles` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:07:41.395805 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4b15b9c-f8c7-4f45-828c-17d7ebd1a36e None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 03:07:41.396415 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4b15b9c-f8c7-4f45-828c-17d7ebd1a36e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:07:41.397995 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4b15b9c-f8c7-4f45-828c-17d7ebd1a36e None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:07:41.414951 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 165/342] 10.222.0.18 () {56 vars in 1111 bytes} [Tue Sep 13 03:07:41 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 290 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 03:07:41.417897 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd310d4-81df-414a-94c8-61790256fde4 None None] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:07:41.418947 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd310d4-81df-414a-94c8-61790256fde4 None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:07:41.419423 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd310d4-81df-414a-94c8-61790256fde4 None None] PATH_INFO: `/` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:07:41.421997 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 166/343] 10.222.0.18 () {54 vars in 927 bytes} [Tue Sep 13 03:07:41 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 03:07:41.424065 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e89f127a-210e-4b16-98d3-ef29d9586974 None None] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:07:41.424547 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e89f127a-210e-4b16-98d3-ef29d9586974 None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:07:41.425012 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e89f127a-210e-4b16-98d3-ef29d9586974 None None] PATH_INFO: `/` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:07:41.426895 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 167/344] 10.222.0.18 () {54 vars in 927 bytes} [Tue Sep 13 03:07:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 03:07:41.427894 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e89f127a-210e-4b16-98d3-ef29d9586974 None None] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:07:41.452539 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 178/345] 10.222.0.18 () {58 vars in 1282 bytes} [Tue Sep 13 03:07:40 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 577 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 13 03:07:41.454135 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c65015d-b621-449a-b1e9-0ba1f2b390e6 None None] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:07:41.454425 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c65015d-b621-449a-b1e9-0ba1f2b390e6 None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:07:41.459638 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c65015d-b621-449a-b1e9-0ba1f2b390e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:07:41.477614 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:07:41.483161 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.auth.core [None req-7c65015d-b621-449a-b1e9-0ba1f2b390e6 None None] MFA Rules not processed for user `49f280ab086b4610855c4ffa378517d9`. Rule list: `[]` (Enabled: `True`). {{(pid=81400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 03:07:41.484610 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-7c65015d-b621-449a-b1e9-0ba1f2b390e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:07:41.541610 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:07:41.563216 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e89f127a-210e-4b16-98d3-ef29d9586974 None None] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:07:41.572216 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 179/346] 10.222.0.18 () {58 vars in 1024 bytes} [Tue Sep 13 03:07:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 13 03:07:41.573829 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d08224f-ba7f-4e0f-94d3-f424f2b1ebeb None None] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:07:41.574336 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d08224f-ba7f-4e0f-94d3-f424f2b1ebeb None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:07:41.574963 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d08224f-ba7f-4e0f-94d3-f424f2b1ebeb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:07:41.599232 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.auth.core [None req-5d08224f-ba7f-4e0f-94d3-f424f2b1ebeb None None] MFA Rules not processed for user `49f280ab086b4610855c4ffa378517d9`. Rule list: `[]` (Enabled: `True`). {{(pid=81400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 03:07:41.600805 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aeeed94-3b04-43bf-9854-ab2ac5c18496 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:07:41.600956 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-5d08224f-ba7f-4e0f-94d3-f424f2b1ebeb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:07:41.601479 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aeeed94-3b04-43bf-9854-ab2ac5c18496 None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:07:41.601760 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aeeed94-3b04-43bf-9854-ab2ac5c18496 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:07:41.602030 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aeeed94-3b04-43bf-9854-ab2ac5c18496 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:07:41.681180 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aeeed94-3b04-43bf-9854-ab2ac5c18496 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:07:41.683846 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aeeed94-3b04-43bf-9854-ab2ac5c18496 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:07:41.685261 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 180/347] 10.222.0.18 () {58 vars in 1024 bytes} [Tue Sep 13 03:07:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 13 03:07:41.686497 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d08224f-ba7f-4e0f-94d3-f424f2b1ebeb None None] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:07:41.754486 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:07:41.862949 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 168/348] 10.222.0.18 () {58 vars in 1282 bytes} [Tue Sep 13 03:07:41 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 435 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 13 03:07:41.870759 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aeeed94-3b04-43bf-9854-ab2ac5c18496 None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:07:41.875582 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d08224f-ba7f-4e0f-94d3-f424f2b1ebeb None None] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:07:41.896289 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:07:41.934140 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3e79081-b461-43a8-9d62-a936e40a2f89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:07:41.934723 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e79081-b461-43a8-9d62-a936e40a2f89 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:07:41.934997 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e79081-b461-43a8-9d62-a936e40a2f89 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:07:41.935268 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e79081-b461-43a8-9d62-a936e40a2f89 None admin] PATH_INFO: `/v3/projects/83aeaa07fb5d40d3b285aeb01d1252b1` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:07:41.936413 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3e79081-b461-43a8-9d62-a936e40a2f89 None admin] RBAC: Authorizing `identity:get_project(project_id=83aeaa07fb5d40d3b285aeb01d1252b1)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:07:41.938036 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3e79081-b461-43a8-9d62-a936e40a2f89 None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:07:41.939883 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 181/349] 10.222.0.18 () {56 vars in 1186 bytes} [Tue Sep 13 03:07:41 2022] GET /identity/v3/projects/83aeaa07fb5d40d3b285aeb01d1252b1 => generated 298 bytes in 253 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 03:07:41.945283 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3e79081-b461-43a8-9d62-a936e40a2f89 None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:07:42.113714 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3e79081-b461-43a8-9d62-a936e40a2f89 None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:07:42.117822 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aeeed94-3b04-43bf-9854-ab2ac5c18496 None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:07:42.168004 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbdb6fe7-6413-42d6-8136-688432af589d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:07:42.168645 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbdb6fe7-6413-42d6-8136-688432af589d None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:07:42.168915 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbdb6fe7-6413-42d6-8136-688432af589d None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:07:42.169179 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbdb6fe7-6413-42d6-8136-688432af589d None admin] PATH_INFO: `/v3/groups/7cea0c7f4a7b4b8f977cd4c1adc36486` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:07:42.170239 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e1bbfd3-0b23-4880-a622-a229ee80f409 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:07:42.171200 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e1bbfd3-0b23-4880-a622-a229ee80f409 None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:07:42.171924 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e1bbfd3-0b23-4880-a622-a229ee80f409 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:07:42.172450 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e1bbfd3-0b23-4880-a622-a229ee80f409 None admin] PATH_INFO: `/v3/projects/83aeaa07fb5d40d3b285aeb01d1252b1` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:07:42.174238 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e1bbfd3-0b23-4880-a622-a229ee80f409 None admin] RBAC: Authorizing `identity:get_project(project_id=83aeaa07fb5d40d3b285aeb01d1252b1)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:07:42.175818 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbdb6fe7-6413-42d6-8136-688432af589d None admin] RBAC: Authorizing `identity:get_group(group_id=7cea0c7f4a7b4b8f977cd4c1adc36486)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:07:42.176437 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e1bbfd3-0b23-4880-a622-a229ee80f409 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:07:42.177302 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbdb6fe7-6413-42d6-8136-688432af589d None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:07:42.178942 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 169/350] 10.222.0.18 () {56 vars in 1186 bytes} [Tue Sep 13 03:07:41 2022] GET /identity/v3/projects/83aeaa07fb5d40d3b285aeb01d1252b1 => generated 298 bytes in 311 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 03:07:42.179193 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 182/351] 10.222.0.18 () {56 vars in 1180 bytes} [Tue Sep 13 03:07:41 2022] GET /identity/v3/groups/7cea0c7f4a7b4b8f977cd4c1adc36486 => generated 226 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 03:07:42.179961 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbdb6fe7-6413-42d6-8136-688432af589d None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:07:42.180994 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b8c964-9be3-4fae-b77f-4bf42370347a None None] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:07:42.181466 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b8c964-9be3-4fae-b77f-4bf42370347a None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:07:42.181937 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b8c964-9be3-4fae-b77f-4bf42370347a None None] PATH_INFO: `/` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:07:42.184791 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 170/352] 10.222.0.18 () {54 vars in 927 bytes} [Tue Sep 13 03:07:42 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 03:07:42.185721 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75b8c964-9be3-4fae-b77f-4bf42370347a None None] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:07:42.192897 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:07:42.237217 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:07:42.316258 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbdb6fe7-6413-42d6-8136-688432af589d None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:07:42.317707 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75b8c964-9be3-4fae-b77f-4bf42370347a None None] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:07:42.361641 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1e3bd85-df4a-4814-9d66-21430be14a9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:07:42.362695 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1e3bd85-df4a-4814-9d66-21430be14a9f None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:07:42.363152 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1e3bd85-df4a-4814-9d66-21430be14a9f None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:07:42.363598 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1e3bd85-df4a-4814-9d66-21430be14a9f None admin] PATH_INFO: `/v3/roles/member` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:07:42.366068 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26356c8e-3808-4751-9231-52de1b26cef2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:07:42.367226 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26356c8e-3808-4751-9231-52de1b26cef2 None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:07:42.367509 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26356c8e-3808-4751-9231-52de1b26cef2 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:07:42.367780 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26356c8e-3808-4751-9231-52de1b26cef2 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:07:42.380038 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1e3bd85-df4a-4814-9d66-21430be14a9f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 03:07:42.385514 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application [None req-b1e3bd85-df4a-4814-9d66-21430be14a9f None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 13 03:07:42.385514 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 03:07:42.385514 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 03:07:42.385514 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 03:07:42.385514 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 03:07:42.385514 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 03:07:42.385514 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 03:07:42.385514 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 03:07:42.385514 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 03:07:42.385514 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 03:07:42.385514 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 03:07:42.385514 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 03:07:42.385514 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 03:07:42.385514 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 03:07:42.385514 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 03:07:42.385514 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 03:07:42.386437 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 03:07:42.386437 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 03:07:42.386437 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 03:07:42.386437 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 03:07:42.386437 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 03:07:42.386437 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 03:07:42.386437 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 03:07:42.386437 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 03:07:42.386437 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 03:07:42.386437 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 03:07:42.386437 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 03:07:42.386437 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application with Lock( Sep 13 03:07:42.386437 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 03:07:42.386437 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._enter() Sep 13 03:07:42.386437 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 03:07:42.386437 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 03:07:42.387292 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 03:07:42.387292 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.creator() Sep 13 03:07:42.387292 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 03:07:42.387292 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application created_value = creator( Sep 13 03:07:42.387292 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 03:07:42.387292 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 03:07:42.387292 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 03:07:42.387292 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 03:07:42.387292 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 03:07:42.387292 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 03:07:42.387292 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 13 03:07:42.387292 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Sep 13 03:07:42.387924 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26356c8e-3808-4751-9231-52de1b26cef2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 03:07:42.388003 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 183/353] 10.222.0.18 () {56 vars in 1099 bytes} [Tue Sep 13 03:07:42 2022] GET /identity/v3/roles/member => generated 84 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 03:07:42.388003 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1e3bd85-df4a-4814-9d66-21430be14a9f None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:07:42.392909 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application [None req-26356c8e-3808-4751-9231-52de1b26cef2 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 13 03:07:42.392909 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 03:07:42.392909 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 03:07:42.392909 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 03:07:42.392909 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 03:07:42.392909 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 03:07:42.392909 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 03:07:42.392909 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 03:07:42.392909 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 03:07:42.392909 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 03:07:42.392909 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 03:07:42.392909 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 03:07:42.392909 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 03:07:42.392909 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 03:07:42.392909 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 03:07:42.392909 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 03:07:42.393866 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 03:07:42.393866 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 03:07:42.393866 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 03:07:42.393866 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 03:07:42.393866 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 03:07:42.393866 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 03:07:42.393866 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 03:07:42.393866 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 03:07:42.393866 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 03:07:42.393866 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 03:07:42.393866 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 03:07:42.393866 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application with Lock( Sep 13 03:07:42.393866 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 03:07:42.393866 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._enter() Sep 13 03:07:42.393866 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 03:07:42.393866 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 03:07:42.394747 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 03:07:42.394747 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.creator() Sep 13 03:07:42.394747 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 03:07:42.394747 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application created_value = creator( Sep 13 03:07:42.394747 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 03:07:42.394747 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 03:07:42.394747 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 03:07:42.394747 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 03:07:42.394747 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 03:07:42.394747 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 03:07:42.394747 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 13 03:07:42.394747 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Sep 13 03:07:42.394747 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 171/354] 10.222.0.18 () {56 vars in 1099 bytes} [Tue Sep 13 03:07:42 2022] GET /identity/v3/roles/reader => generated 84 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 03:07:42.399128 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6941c84c-f10a-4438-ab93-1f1a60ea464b None None] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:07:42.399128 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6941c84c-f10a-4438-ab93-1f1a60ea464b None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:07:42.399128 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6941c84c-f10a-4438-ab93-1f1a60ea464b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:07:42.418036 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.auth.core [None req-6941c84c-f10a-4438-ab93-1f1a60ea464b None None] MFA Rules not processed for user `49f280ab086b4610855c4ffa378517d9`. Rule list: `[]` (Enabled: `True`). {{(pid=81399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 03:07:42.421153 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-6941c84c-f10a-4438-ab93-1f1a60ea464b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:07:42.490696 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:07:42.505340 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 172/355] 10.222.0.18 () {58 vars in 1024 bytes} [Tue Sep 13 03:07:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 13 03:07:42.506272 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6941c84c-f10a-4438-ab93-1f1a60ea464b None None] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:07:42.537825 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:07:42.544307 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1e3bd85-df4a-4814-9d66-21430be14a9f None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:07:42.590261 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57395945-9cdf-45b3-a15b-75b9ec91ded6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:07:42.590820 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57395945-9cdf-45b3-a15b-75b9ec91ded6 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:07:42.591097 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57395945-9cdf-45b3-a15b-75b9ec91ded6 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:07:42.591367 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57395945-9cdf-45b3-a15b-75b9ec91ded6 None admin] PATH_INFO: `/v3/groups/7cea0c7f4a7b4b8f977cd4c1adc36486` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:07:42.592474 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57395945-9cdf-45b3-a15b-75b9ec91ded6 None admin] RBAC: Authorizing `identity:get_group(group_id=7cea0c7f4a7b4b8f977cd4c1adc36486)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:07:42.593969 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57395945-9cdf-45b3-a15b-75b9ec91ded6 None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:07:42.596547 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 184/356] 10.222.0.18 () {56 vars in 1180 bytes} [Tue Sep 13 03:07:42 2022] GET /identity/v3/groups/7cea0c7f4a7b4b8f977cd4c1adc36486 => generated 226 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 03:07:42.596813 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57395945-9cdf-45b3-a15b-75b9ec91ded6 None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:07:42.651535 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6941c84c-f10a-4438-ab93-1f1a60ea464b None None] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:07:42.697813 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6534c487-6138-4c40-af42-201fabcdc556 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:07:42.698426 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6534c487-6138-4c40-af42-201fabcdc556 None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:07:42.698736 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6534c487-6138-4c40-af42-201fabcdc556 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:07:42.699014 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6534c487-6138-4c40-af42-201fabcdc556 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:07:42.699742 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6534c487-6138-4c40-af42-201fabcdc556 None admin] RBAC: Adding query filter params (group.id=7cea0c7f4a7b4b8f977cd4c1adc36486, role.id=1eb7a5b5203945c29e6d22d653f47847, scope.project.id=83aeaa07fb5d40d3b285aeb01d1252b1) {{(pid=81399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 03:07:42.700051 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6534c487-6138-4c40-af42-201fabcdc556 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:07:42.701482 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6534c487-6138-4c40-af42-201fabcdc556 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:07:42.708599 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 173/357] 10.222.0.18 () {56 vars in 1376 bytes} [Tue Sep 13 03:07:42 2022] GET /identity/v3/role_assignments?group.id=7cea0c7f4a7b4b8f977cd4c1adc36486&scope.project.id=83aeaa07fb5d40d3b285aeb01d1252b1&role.id=1eb7a5b5203945c29e6d22d653f47847 => generated 260 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 03:07:42.709241 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6534c487-6138-4c40-af42-201fabcdc556 None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:07:42.720613 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57395945-9cdf-45b3-a15b-75b9ec91ded6 None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:07:42.769279 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-418527ae-fecd-4471-817c-077454943176 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:07:42.770082 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-418527ae-fecd-4471-817c-077454943176 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:07:42.770355 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-418527ae-fecd-4471-817c-077454943176 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:07:42.770639 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-418527ae-fecd-4471-817c-077454943176 None admin] PATH_INFO: `/v3/roles` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:07:42.771566 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-418527ae-fecd-4471-817c-077454943176 None admin] RBAC: Adding query filter params (name=member) {{(pid=81400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 03:07:42.772267 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-418527ae-fecd-4471-817c-077454943176 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:07:42.773660 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-418527ae-fecd-4471-817c-077454943176 None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:07:42.783104 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:07:42.787710 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 185/358] 10.222.0.18 () {56 vars in 1101 bytes} [Tue Sep 13 03:07:42 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 03:07:42.792749 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d44813af-440a-4445-8ce8-c6b82bf1c97c None None] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:07:42.793042 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d44813af-440a-4445-8ce8-c6b82bf1c97c None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:07:42.793288 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d44813af-440a-4445-8ce8-c6b82bf1c97c None None] PATH_INFO: `/` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:07:42.794885 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 186/359] 10.222.0.18 () {54 vars in 927 bytes} [Tue Sep 13 03:07:42 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 03:07:42.795623 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d44813af-440a-4445-8ce8-c6b82bf1c97c None None] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:07:42.846756 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:07:42.847000 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6534c487-6138-4c40-af42-201fabcdc556 None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:07:42.887269 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7097a0be-eb87-46b6-afa0-a25c135ceeb3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:07:42.887898 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7097a0be-eb87-46b6-afa0-a25c135ceeb3 None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:07:42.888176 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7097a0be-eb87-46b6-afa0-a25c135ceeb3 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:07:42.888446 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7097a0be-eb87-46b6-afa0-a25c135ceeb3 None admin] PATH_INFO: `/v3/roles` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:07:42.889116 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7097a0be-eb87-46b6-afa0-a25c135ceeb3 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 03:07:42.889416 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7097a0be-eb87-46b6-afa0-a25c135ceeb3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:07:42.891388 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7097a0be-eb87-46b6-afa0-a25c135ceeb3 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:07:42.901332 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 174/360] 10.222.0.18 () {56 vars in 1101 bytes} [Tue Sep 13 03:07:42 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 03:07:42.902422 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f9edbeb-fa0d-4839-9bc0-e8f76a01264c None None] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:07:42.902738 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f9edbeb-fa0d-4839-9bc0-e8f76a01264c None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:07:42.903014 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f9edbeb-fa0d-4839-9bc0-e8f76a01264c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:07:42.923700 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.auth.core [None req-2f9edbeb-fa0d-4839-9bc0-e8f76a01264c None None] MFA Rules not processed for user `49f280ab086b4610855c4ffa378517d9`. Rule list: `[]` (Enabled: `True`). {{(pid=81399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 03:07:42.924915 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-2f9edbeb-fa0d-4839-9bc0-e8f76a01264c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:07:42.940852 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d44813af-440a-4445-8ce8-c6b82bf1c97c None None] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:07:42.982998 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30f9c543-6e47-4cb4-88bf-bd07abbc93a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:07:42.983604 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30f9c543-6e47-4cb4-88bf-bd07abbc93a1 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:07:42.983877 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30f9c543-6e47-4cb4-88bf-bd07abbc93a1 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:07:42.984141 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30f9c543-6e47-4cb4-88bf-bd07abbc93a1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:07:42.984952 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30f9c543-6e47-4cb4-88bf-bd07abbc93a1 None admin] RBAC: Adding query filter params (group.id=7cea0c7f4a7b4b8f977cd4c1adc36486, role.id=a9aa35ebda4d4aff80e62f750e9104ab, scope.project.id=83aeaa07fb5d40d3b285aeb01d1252b1) {{(pid=81400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 03:07:42.985256 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30f9c543-6e47-4cb4-88bf-bd07abbc93a1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:07:42.988784 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30f9c543-6e47-4cb4-88bf-bd07abbc93a1 None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:07:42.995193 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 187/361] 10.222.0.18 () {56 vars in 1376 bytes} [Tue Sep 13 03:07:42 2022] GET /identity/v3/role_assignments?group.id=7cea0c7f4a7b4b8f977cd4c1adc36486&scope.project.id=83aeaa07fb5d40d3b285aeb01d1252b1&role.id=a9aa35ebda4d4aff80e62f750e9104ab => generated 260 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 03:07:42.996004 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30f9c543-6e47-4cb4-88bf-bd07abbc93a1 None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:07:43.008250 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 175/362] 10.222.0.18 () {58 vars in 1024 bytes} [Tue Sep 13 03:07:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 13 03:07:43.009944 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f9edbeb-fa0d-4839-9bc0-e8f76a01264c None None] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:07:43.094447 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:07:43.152367 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:07:43.160762 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30f9c543-6e47-4cb4-88bf-bd07abbc93a1 None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:07:43.191223 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f9edbeb-fa0d-4839-9bc0-e8f76a01264c None None] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:07:43.201700 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbfc6097-30a6-4f77-acb2-4baee726a8cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:07:43.202714 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbfc6097-30a6-4f77-acb2-4baee726a8cf None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:07:43.203192 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbfc6097-30a6-4f77-acb2-4baee726a8cf None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:07:43.203706 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbfc6097-30a6-4f77-acb2-4baee726a8cf None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:07:43.271354 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9206123-211d-45ef-876a-fc7e8162f2ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:07:43.271967 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9206123-211d-45ef-876a-fc7e8162f2ad None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:07:43.272248 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9206123-211d-45ef-876a-fc7e8162f2ad None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:07:43.272521 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9206123-211d-45ef-876a-fc7e8162f2ad None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:07:43.282043 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbfc6097-30a6-4f77-acb2-4baee726a8cf None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:07:43.283776 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbfc6097-30a6-4f77-acb2-4baee726a8cf None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:07:43.289648 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9206123-211d-45ef-876a-fc7e8162f2ad None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 03:07:43.295234 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application [None req-b9206123-211d-45ef-876a-fc7e8162f2ad None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 13 03:07:43.295234 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 03:07:43.295234 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 03:07:43.295234 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 03:07:43.295234 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 03:07:43.295234 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 03:07:43.295234 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 03:07:43.295234 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 03:07:43.295234 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 03:07:43.295234 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 03:07:43.295234 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 03:07:43.295234 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 03:07:43.295234 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 03:07:43.295234 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 03:07:43.295234 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 03:07:43.295234 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 03:07:43.296156 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 03:07:43.296156 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 03:07:43.296156 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 03:07:43.296156 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 03:07:43.296156 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 03:07:43.296156 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 03:07:43.296156 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 03:07:43.296156 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 03:07:43.296156 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 03:07:43.296156 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 03:07:43.296156 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 03:07:43.296156 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application with Lock( Sep 13 03:07:43.296156 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 03:07:43.296156 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._enter() Sep 13 03:07:43.296156 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 03:07:43.296156 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 03:07:43.297013 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 03:07:43.297013 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.creator() Sep 13 03:07:43.297013 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 03:07:43.297013 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application created_value = creator( Sep 13 03:07:43.297013 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 03:07:43.297013 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 03:07:43.297013 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 03:07:43.297013 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 03:07:43.297013 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 03:07:43.297013 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 03:07:43.297013 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 13 03:07:43.297013 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Sep 13 03:07:43.297013 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 176/363] 10.222.0.18 () {56 vars in 1114 bytes} [Tue Sep 13 03:07:43 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 287 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 03:07:43.301890 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9206123-211d-45ef-876a-fc7e8162f2ad None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:07:43.394999 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:07:43.403389 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 188/364] 10.222.0.18 () {58 vars in 1282 bytes} [Tue Sep 13 03:07:42 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 407 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 13 03:07:43.404901 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbfc6097-30a6-4f77-acb2-4baee726a8cf None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:07:43.440405 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9206123-211d-45ef-876a-fc7e8162f2ad None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:07:43.459702 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:07:43.485412 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f2d302b-7c1e-4776-97f6-e806eb6e5253 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:07:43.487032 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f2d302b-7c1e-4776-97f6-e806eb6e5253 None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:07:43.487296 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f2d302b-7c1e-4776-97f6-e806eb6e5253 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:07:43.487572 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f2d302b-7c1e-4776-97f6-e806eb6e5253 None admin] PATH_INFO: `/v3/users/a8487a20e1a74e4b90d755457519f3ee` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:07:43.491514 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f2d302b-7c1e-4776-97f6-e806eb6e5253 None admin] RBAC: Authorizing `identity:get_user(user_id=a8487a20e1a74e4b90d755457519f3ee)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:07:43.493715 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f2d302b-7c1e-4776-97f6-e806eb6e5253 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:07:43.496780 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 177/365] 10.222.0.18 () {56 vars in 1177 bytes} [Tue Sep 13 03:07:43 2022] GET /identity/v3/users/a8487a20e1a74e4b90d755457519f3ee => generated 293 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 03:07:43.497806 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f2d302b-7c1e-4776-97f6-e806eb6e5253 None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:07:43.528716 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbfc6097-30a6-4f77-acb2-4baee726a8cf None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:07:43.568396 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6602d9d-0c3c-45ee-a398-a2f2a6328840 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:07:43.569290 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6602d9d-0c3c-45ee-a398-a2f2a6328840 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:07:43.569408 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6602d9d-0c3c-45ee-a398-a2f2a6328840 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:07:43.569681 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6602d9d-0c3c-45ee-a398-a2f2a6328840 None admin] PATH_INFO: `/v3/roles` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:07:43.570352 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6602d9d-0c3c-45ee-a398-a2f2a6328840 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 03:07:43.570724 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6602d9d-0c3c-45ee-a398-a2f2a6328840 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:07:43.572154 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6602d9d-0c3c-45ee-a398-a2f2a6328840 None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:07:43.587957 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 189/366] 10.222.0.18 () {56 vars in 1111 bytes} [Tue Sep 13 03:07:43 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 03:07:43.588933 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6602d9d-0c3c-45ee-a398-a2f2a6328840 None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:07:43.637242 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f2d302b-7c1e-4776-97f6-e806eb6e5253 None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:07:43.657357 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:07:43.685040 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e9c3da9-88f4-4c3c-b1b6-cdc26df685c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:07:43.686138 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e9c3da9-88f4-4c3c-b1b6-cdc26df685c5 None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:07:43.686982 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e9c3da9-88f4-4c3c-b1b6-cdc26df685c5 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:07:43.687569 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e9c3da9-88f4-4c3c-b1b6-cdc26df685c5 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:07:43.701579 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e9c3da9-88f4-4c3c-b1b6-cdc26df685c5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 03:07:43.708668 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6602d9d-0c3c-45ee-a398-a2f2a6328840 None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:07:43.711915 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application [None req-3e9c3da9-88f4-4c3c-b1b6-cdc26df685c5 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 13 03:07:43.711915 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 03:07:43.711915 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 03:07:43.711915 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 03:07:43.711915 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 03:07:43.711915 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 03:07:43.711915 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 03:07:43.711915 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 03:07:43.711915 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 03:07:43.711915 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 03:07:43.711915 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 03:07:43.711915 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 03:07:43.711915 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 03:07:43.711915 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 03:07:43.711915 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 03:07:43.711915 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 03:07:43.712849 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 03:07:43.712849 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 03:07:43.712849 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 03:07:43.712849 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 03:07:43.712849 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 03:07:43.712849 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 03:07:43.712849 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 03:07:43.712849 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 03:07:43.712849 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 03:07:43.712849 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 03:07:43.712849 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 03:07:43.712849 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application with Lock( Sep 13 03:07:43.712849 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 03:07:43.712849 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._enter() Sep 13 03:07:43.712849 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 03:07:43.712849 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 03:07:43.713695 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 03:07:43.713695 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.creator() Sep 13 03:07:43.713695 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 03:07:43.713695 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application created_value = creator( Sep 13 03:07:43.713695 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 03:07:43.713695 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 03:07:43.713695 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 03:07:43.713695 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 03:07:43.713695 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 03:07:43.713695 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 03:07:43.713695 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 13 03:07:43.713695 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Sep 13 03:07:43.716818 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 178/367] 10.222.0.18 () {56 vars in 1096 bytes} [Tue Sep 13 03:07:43 2022] GET /identity/v3/roles/admin => generated 83 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 03:07:43.717807 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e9c3da9-88f4-4c3c-b1b6-cdc26df685c5 None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:07:43.751638 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ebdfd01-bdf1-433a-99cc-d4618e7de8dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:07:43.752437 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ebdfd01-bdf1-433a-99cc-d4618e7de8dd None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:07:43.752783 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ebdfd01-bdf1-433a-99cc-d4618e7de8dd None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:07:43.753127 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ebdfd01-bdf1-433a-99cc-d4618e7de8dd None admin] PATH_INFO: `/v3/projects/54999ea237f64c5f972ef79e765facd0` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:07:43.755361 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ebdfd01-bdf1-433a-99cc-d4618e7de8dd None admin] RBAC: Authorizing `identity:get_project(project_id=54999ea237f64c5f972ef79e765facd0)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:07:43.757158 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ebdfd01-bdf1-433a-99cc-d4618e7de8dd None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:07:43.760191 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 190/368] 10.222.0.18 () {56 vars in 1186 bytes} [Tue Sep 13 03:07:43 2022] GET /identity/v3/projects/54999ea237f64c5f972ef79e765facd0 => generated 302 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 03:07:43.761144 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ebdfd01-bdf1-433a-99cc-d4618e7de8dd None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:07:43.770974 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:07:43.863384 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e9c3da9-88f4-4c3c-b1b6-cdc26df685c5 None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:07:43.900243 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ebdfd01-bdf1-433a-99cc-d4618e7de8dd None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:07:43.911414 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d7d6ad7-ab31-4629-a9a7-c8d4a4017998 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:07:43.912312 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d7d6ad7-ab31-4629-a9a7-c8d4a4017998 None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:07:43.913164 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d7d6ad7-ab31-4629-a9a7-c8d4a4017998 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:07:43.913932 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d7d6ad7-ab31-4629-a9a7-c8d4a4017998 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:07:43.914869 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d7d6ad7-ab31-4629-a9a7-c8d4a4017998 None admin] RBAC: Adding query filter params (role.id=502e6c11e747432988d7f723cee0af64, user.id=a8487a20e1a74e4b90d755457519f3ee, scope.system=all) {{(pid=81399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 03:07:43.915622 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d7d6ad7-ab31-4629-a9a7-c8d4a4017998 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:07:43.918113 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d7d6ad7-ab31-4629-a9a7-c8d4a4017998 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:07:43.928380 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:07:43.935670 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 179/369] 10.222.0.18 () {56 vars in 1308 bytes} [Tue Sep 13 03:07:43 2022] GET /identity/v3/role_assignments?user.id=a8487a20e1a74e4b90d755457519f3ee&scope.system=all&role.id=502e6c11e747432988d7f723cee0af64 => generated 510 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 03:07:43.936804 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d7d6ad7-ab31-4629-a9a7-c8d4a4017998 None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:07:43.950605 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d6540f9-ebd4-4c46-88a3-7b5ef5f545f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:07:43.951290 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d6540f9-ebd4-4c46-88a3-7b5ef5f545f8 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:07:43.951623 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d6540f9-ebd4-4c46-88a3-7b5ef5f545f8 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:07:43.951897 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d6540f9-ebd4-4c46-88a3-7b5ef5f545f8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:07:44.059957 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:07:44.069189 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d7d6ad7-ab31-4629-a9a7-c8d4a4017998 None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:07:44.107518 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d6540f9-ebd4-4c46-88a3-7b5ef5f545f8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:07:44.109104 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d6540f9-ebd4-4c46-88a3-7b5ef5f545f8 None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:07:44.110940 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eae4c1ef-e2d2-4c1a-bf24-040e199ff4b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:07:44.112097 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae4c1ef-e2d2-4c1a-bf24-040e199ff4b3 None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:07:44.112593 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae4c1ef-e2d2-4c1a-bf24-040e199ff4b3 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:07:44.113084 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae4c1ef-e2d2-4c1a-bf24-040e199ff4b3 None admin] PATH_INFO: `/v3/roles` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:07:44.113958 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eae4c1ef-e2d2-4c1a-bf24-040e199ff4b3 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 03:07:44.114478 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eae4c1ef-e2d2-4c1a-bf24-040e199ff4b3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:07:44.117252 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eae4c1ef-e2d2-4c1a-bf24-040e199ff4b3 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:07:44.128068 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 180/370] 10.222.0.18 () {56 vars in 1099 bytes} [Tue Sep 13 03:07:43 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 03:07:44.129250 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eae4c1ef-e2d2-4c1a-bf24-040e199ff4b3 None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:07:44.251366 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eae4c1ef-e2d2-4c1a-bf24-040e199ff4b3 None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:07:44.300165 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-228bdf93-4801-4d9a-9240-4cb220308eb5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:07:44.300813 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-228bdf93-4801-4d9a-9240-4cb220308eb5 None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:07:44.301150 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-228bdf93-4801-4d9a-9240-4cb220308eb5 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:07:44.301425 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-228bdf93-4801-4d9a-9240-4cb220308eb5 None admin] PATH_INFO: `/v3/groups/7cea0c7f4a7b4b8f977cd4c1adc36486` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:07:44.303755 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-228bdf93-4801-4d9a-9240-4cb220308eb5 None admin] RBAC: Authorizing `identity:get_group(group_id=7cea0c7f4a7b4b8f977cd4c1adc36486)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:07:44.305831 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-228bdf93-4801-4d9a-9240-4cb220308eb5 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:07:44.307834 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 181/371] 10.222.0.18 () {56 vars in 1180 bytes} [Tue Sep 13 03:07:44 2022] GET /identity/v3/groups/7cea0c7f4a7b4b8f977cd4c1adc36486 => generated 226 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 03:07:44.309099 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-228bdf93-4801-4d9a-9240-4cb220308eb5 None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:07:44.336865 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 191/372] 10.222.0.18 () {58 vars in 1282 bytes} [Tue Sep 13 03:07:43 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 576 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 13 03:07:44.337742 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d6540f9-ebd4-4c46-88a3-7b5ef5f545f8 None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:07:44.355033 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:07:44.381782 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:07:44.449599 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-228bdf93-4801-4d9a-9240-4cb220308eb5 None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:07:44.480210 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d6540f9-ebd4-4c46-88a3-7b5ef5f545f8 None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:07:44.495359 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-930519df-3c35-4f46-9f74-dab6d578f315 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:07:44.496344 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-930519df-3c35-4f46-9f74-dab6d578f315 None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:07:44.496768 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-930519df-3c35-4f46-9f74-dab6d578f315 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:07:44.497210 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-930519df-3c35-4f46-9f74-dab6d578f315 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:07:44.520843 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97f708c0-efcc-4231-8f7f-302e510877b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:07:44.521506 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f708c0-efcc-4231-8f7f-302e510877b8 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:07:44.521883 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f708c0-efcc-4231-8f7f-302e510877b8 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:07:44.522254 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f708c0-efcc-4231-8f7f-302e510877b8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:07:44.523095 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97f708c0-efcc-4231-8f7f-302e510877b8 None admin] RBAC: Adding query filter params (group.id=7cea0c7f4a7b4b8f977cd4c1adc36486, role.id=1eb7a5b5203945c29e6d22d653f47847, scope.project.id=54999ea237f64c5f972ef79e765facd0) {{(pid=81400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 03:07:44.523474 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97f708c0-efcc-4231-8f7f-302e510877b8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:07:44.525037 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97f708c0-efcc-4231-8f7f-302e510877b8 None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:07:44.533716 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 192/373] 10.222.0.18 () {56 vars in 1376 bytes} [Tue Sep 13 03:07:44 2022] GET /identity/v3/role_assignments?group.id=7cea0c7f4a7b4b8f977cd4c1adc36486&scope.project.id=54999ea237f64c5f972ef79e765facd0&role.id=1eb7a5b5203945c29e6d22d653f47847 => generated 260 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 03:07:44.534767 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97f708c0-efcc-4231-8f7f-302e510877b8 None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:07:44.628589 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-930519df-3c35-4f46-9f74-dab6d578f315 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:07:44.630418 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-930519df-3c35-4f46-9f74-dab6d578f315 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:07:44.668766 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:07:44.697965 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:07:44.699547 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97f708c0-efcc-4231-8f7f-302e510877b8 None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:07:44.771842 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-003dcb45-4a32-4199-8410-a70055d68e76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:07:44.772401 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-003dcb45-4a32-4199-8410-a70055d68e76 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:07:44.772680 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-003dcb45-4a32-4199-8410-a70055d68e76 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:07:44.772946 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-003dcb45-4a32-4199-8410-a70055d68e76 None admin] PATH_INFO: `/v3/projects/54999ea237f64c5f972ef79e765facd0` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:07:44.778511 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-003dcb45-4a32-4199-8410-a70055d68e76 None admin] RBAC: Authorizing `identity:get_project(project_id=54999ea237f64c5f972ef79e765facd0)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:07:44.779983 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-003dcb45-4a32-4199-8410-a70055d68e76 None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:07:44.782422 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 193/374] 10.222.0.18 () {56 vars in 1186 bytes} [Tue Sep 13 03:07:44 2022] GET /identity/v3/projects/54999ea237f64c5f972ef79e765facd0 => generated 302 bytes in 248 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 03:07:44.788393 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-003dcb45-4a32-4199-8410-a70055d68e76 None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:07:44.822123 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 182/375] 10.222.0.18 () {58 vars in 1282 bytes} [Tue Sep 13 03:07:44 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 514 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 13 03:07:44.833759 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-930519df-3c35-4f46-9f74-dab6d578f315 None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:07:44.954463 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-003dcb45-4a32-4199-8410-a70055d68e76 None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:07:45.021581 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-930519df-3c35-4f46-9f74-dab6d578f315 None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:07:45.024479 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69093cac-904a-4bd7-9dd4-d39d12128081 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:07:45.025058 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69093cac-904a-4bd7-9dd4-d39d12128081 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:07:45.025330 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69093cac-904a-4bd7-9dd4-d39d12128081 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:07:45.025597 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69093cac-904a-4bd7-9dd4-d39d12128081 None admin] PATH_INFO: `/v3/groups/7cea0c7f4a7b4b8f977cd4c1adc36486` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:07:45.027388 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69093cac-904a-4bd7-9dd4-d39d12128081 None admin] RBAC: Authorizing `identity:get_group(group_id=7cea0c7f4a7b4b8f977cd4c1adc36486)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:07:45.028825 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69093cac-904a-4bd7-9dd4-d39d12128081 None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:07:45.036086 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 194/376] 10.222.0.18 () {56 vars in 1180 bytes} [Tue Sep 13 03:07:44 2022] GET /identity/v3/groups/7cea0c7f4a7b4b8f977cd4c1adc36486 => generated 226 bytes in 248 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 03:07:45.038179 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69093cac-904a-4bd7-9dd4-d39d12128081 None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:07:45.079010 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:07:45.092095 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bb9c518-8348-4b17-b5c0-2ac3b13b1c57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:07:45.092745 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb9c518-8348-4b17-b5c0-2ac3b13b1c57 None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:07:45.093025 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb9c518-8348-4b17-b5c0-2ac3b13b1c57 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:07:45.093300 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb9c518-8348-4b17-b5c0-2ac3b13b1c57 None admin] PATH_INFO: `/v3/projects/76c5181fc7be45279ac00017bb4640b3` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:07:45.094641 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:07:45.096864 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bb9c518-8348-4b17-b5c0-2ac3b13b1c57 None admin] RBAC: Authorizing `identity:get_project(project_id=76c5181fc7be45279ac00017bb4640b3)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:07:45.098340 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bb9c518-8348-4b17-b5c0-2ac3b13b1c57 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:07:45.100136 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 183/377] 10.222.0.18 () {56 vars in 1186 bytes} [Tue Sep 13 03:07:44 2022] GET /identity/v3/projects/76c5181fc7be45279ac00017bb4640b3 => generated 344 bytes in 267 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 03:07:45.109883 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bb9c518-8348-4b17-b5c0-2ac3b13b1c57 None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:07:45.239105 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69093cac-904a-4bd7-9dd4-d39d12128081 None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:07:45.299654 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bb9c518-8348-4b17-b5c0-2ac3b13b1c57 None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:07:45.309508 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb0da224-3e71-4be1-9365-f0e5163908c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:07:45.310121 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb0da224-3e71-4be1-9365-f0e5163908c1 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:07:45.310395 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb0da224-3e71-4be1-9365-f0e5163908c1 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:07:45.310689 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb0da224-3e71-4be1-9365-f0e5163908c1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:07:45.311423 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb0da224-3e71-4be1-9365-f0e5163908c1 None admin] RBAC: Adding query filter params (group.id=7cea0c7f4a7b4b8f977cd4c1adc36486, role.id=a9aa35ebda4d4aff80e62f750e9104ab, scope.project.id=54999ea237f64c5f972ef79e765facd0) {{(pid=81400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 03:07:45.311730 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb0da224-3e71-4be1-9365-f0e5163908c1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:07:45.313662 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb0da224-3e71-4be1-9365-f0e5163908c1 None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:07:45.326003 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 195/378] 10.222.0.18 () {56 vars in 1376 bytes} [Tue Sep 13 03:07:45 2022] GET /identity/v3/role_assignments?group.id=7cea0c7f4a7b4b8f977cd4c1adc36486&scope.project.id=54999ea237f64c5f972ef79e765facd0&role.id=a9aa35ebda4d4aff80e62f750e9104ab => generated 260 bytes in 288 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 03:07:45.352176 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34d13382-6ca2-4a86-af5d-e8f34f77271e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:07:45.352929 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d13382-6ca2-4a86-af5d-e8f34f77271e None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:07:45.353212 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d13382-6ca2-4a86-af5d-e8f34f77271e None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:07:45.353481 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d13382-6ca2-4a86-af5d-e8f34f77271e None admin] PATH_INFO: `/v3/groups/228dcaf0a4474b2b804468d12ab54ded` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:07:45.360592 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34d13382-6ca2-4a86-af5d-e8f34f77271e None admin] RBAC: Authorizing `identity:get_group(group_id=228dcaf0a4474b2b804468d12ab54ded)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:07:45.362140 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34d13382-6ca2-4a86-af5d-e8f34f77271e None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:07:45.364280 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 184/379] 10.222.0.18 () {56 vars in 1180 bytes} [Tue Sep 13 03:07:45 2022] GET /identity/v3/groups/228dcaf0a4474b2b804468d12ab54ded => generated 229 bytes in 255 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 03:07:45.370257 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb0da224-3e71-4be1-9365-f0e5163908c1 None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:07:45.393385 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0447266-9451-43ff-83b0-3f3250bed6b8 None None] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:07:45.393721 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0447266-9451-43ff-83b0-3f3250bed6b8 None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:07:45.393937 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0447266-9451-43ff-83b0-3f3250bed6b8 None None] PATH_INFO: `/` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:07:45.395489 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 185/380] 10.222.0.18 () {54 vars in 927 bytes} [Tue Sep 13 03:07:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 13 03:07:45.400627 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-672a0683-5d52-4dfc-9cc6-fca06abf2365 None None] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:07:45.400912 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-672a0683-5d52-4dfc-9cc6-fca06abf2365 None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:07:45.401189 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-672a0683-5d52-4dfc-9cc6-fca06abf2365 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:07:45.420147 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.auth.core [None req-672a0683-5d52-4dfc-9cc6-fca06abf2365 None None] MFA Rules not processed for user `49f280ab086b4610855c4ffa378517d9`. Rule list: `[]` (Enabled: `True`). {{(pid=81399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 03:07:45.421351 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-672a0683-5d52-4dfc-9cc6-fca06abf2365 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:07:45.484463 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:07:45.498281 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb0da224-3e71-4be1-9365-f0e5163908c1 None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:07:45.540245 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d203be19-6c8d-4d7e-9982-5bb57336b8af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:07:45.540872 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d203be19-6c8d-4d7e-9982-5bb57336b8af None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:07:45.541146 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d203be19-6c8d-4d7e-9982-5bb57336b8af None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:07:45.541416 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d203be19-6c8d-4d7e-9982-5bb57336b8af None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:07:45.542151 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d203be19-6c8d-4d7e-9982-5bb57336b8af None admin] RBAC: Adding query filter params (group.id=228dcaf0a4474b2b804468d12ab54ded, role.id=425d4b481c7846e0bc07e7e805d5b15f, scope.project.id=76c5181fc7be45279ac00017bb4640b3) {{(pid=81400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 03:07:45.542455 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d203be19-6c8d-4d7e-9982-5bb57336b8af None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:07:45.544029 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d203be19-6c8d-4d7e-9982-5bb57336b8af None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:07:45.551476 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 196/381] 10.222.0.18 () {56 vars in 1376 bytes} [Tue Sep 13 03:07:45 2022] GET /identity/v3/role_assignments?group.id=228dcaf0a4474b2b804468d12ab54ded&scope.project.id=76c5181fc7be45279ac00017bb4640b3&role.id=425d4b481c7846e0bc07e7e805d5b15f => generated 260 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 03:07:45.552373 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:07:45.573798 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 186/382] 10.222.0.18 () {58 vars in 1024 bytes} [Tue Sep 13 03:07:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 13 03:07:45.786749 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc973d41-6136-4bbc-8c00-f7de58640ecf None None] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:07:45.787033 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc973d41-6136-4bbc-8c00-f7de58640ecf None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:07:45.787306 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc973d41-6136-4bbc-8c00-f7de58640ecf None None] PATH_INFO: `/` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:07:45.788659 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 197/383] 10.222.0.18 () {54 vars in 927 bytes} [Tue Sep 13 03:07:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 03:07:45.794077 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa6324d7-abfe-43e3-978f-7de20579411e None None] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:07:45.794364 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa6324d7-abfe-43e3-978f-7de20579411e None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:07:45.794662 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa6324d7-abfe-43e3-978f-7de20579411e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:07:45.816910 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.auth.core [None req-aa6324d7-abfe-43e3-978f-7de20579411e None None] MFA Rules not processed for user `49f280ab086b4610855c4ffa378517d9`. Rule list: `[]` (Enabled: `True`). {{(pid=81399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 03:07:45.818036 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-aa6324d7-abfe-43e3-978f-7de20579411e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:07:45.878938 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc973d41-6136-4bbc-8c00-f7de58640ecf None None] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:07:45.900842 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 187/384] 10.222.0.18 () {58 vars in 1024 bytes} [Tue Sep 13 03:07:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 13 03:07:46.013381 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc973d41-6136-4bbc-8c00-f7de58640ecf None None] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:07:46.055881 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09cb4b3f-70c1-44a4-a60e-a89336f46f4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:07:46.056514 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09cb4b3f-70c1-44a4-a60e-a89336f46f4d None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:07:46.056789 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09cb4b3f-70c1-44a4-a60e-a89336f46f4d None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:07:46.057054 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09cb4b3f-70c1-44a4-a60e-a89336f46f4d None admin] PATH_INFO: `/v3/roles/member` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:07:46.075698 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09cb4b3f-70c1-44a4-a60e-a89336f46f4d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 03:07:46.081248 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application [None req-09cb4b3f-70c1-44a4-a60e-a89336f46f4d None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 13 03:07:46.081248 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 03:07:46.081248 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 03:07:46.081248 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 03:07:46.081248 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 03:07:46.081248 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 03:07:46.081248 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 03:07:46.081248 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 03:07:46.081248 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 03:07:46.081248 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 03:07:46.081248 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 03:07:46.081248 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 03:07:46.081248 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 03:07:46.081248 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 03:07:46.081248 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 03:07:46.081248 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 03:07:46.083394 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 03:07:46.083394 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 03:07:46.083394 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 03:07:46.083394 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 03:07:46.083394 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 03:07:46.083394 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 03:07:46.083394 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 03:07:46.083394 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 03:07:46.083394 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 03:07:46.083394 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 03:07:46.083394 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 03:07:46.083394 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application with Lock( Sep 13 03:07:46.083394 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 03:07:46.083394 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._enter() Sep 13 03:07:46.083394 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 03:07:46.083394 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 03:07:46.085434 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 03:07:46.085434 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.creator() Sep 13 03:07:46.085434 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 03:07:46.085434 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application created_value = creator( Sep 13 03:07:46.085434 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 03:07:46.085434 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 03:07:46.085434 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 03:07:46.085434 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 03:07:46.085434 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 03:07:46.085434 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 03:07:46.085434 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 13 03:07:46.085434 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Sep 13 03:07:46.085434 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 198/385] 10.222.0.18 () {56 vars in 1099 bytes} [Tue Sep 13 03:07:45 2022] GET /identity/v3/roles/member => generated 84 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 03:07:46.091866 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa6324d7-abfe-43e3-978f-7de20579411e None None] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:07:46.210125 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa6324d7-abfe-43e3-978f-7de20579411e None None] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:07:46.230341 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:07:46.254237 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9573b9ee-7fa8-4a2d-9d48-f8d987713d59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:07:46.254909 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9573b9ee-7fa8-4a2d-9d48-f8d987713d59 None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:07:46.255198 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9573b9ee-7fa8-4a2d-9d48-f8d987713d59 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:07:46.255472 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9573b9ee-7fa8-4a2d-9d48-f8d987713d59 None admin] PATH_INFO: `/v3/roles` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:07:46.256148 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9573b9ee-7fa8-4a2d-9d48-f8d987713d59 None admin] RBAC: Adding query filter params (name=member) {{(pid=81399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 03:07:46.256450 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9573b9ee-7fa8-4a2d-9d48-f8d987713d59 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:07:46.257882 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9573b9ee-7fa8-4a2d-9d48-f8d987713d59 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:07:46.267598 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 188/386] 10.222.0.18 () {56 vars in 1101 bytes} [Tue Sep 13 03:07:46 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 03:07:46.273130 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09cb4b3f-70c1-44a4-a60e-a89336f46f4d None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:07:46.291876 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:07:46.393713 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09cb4b3f-70c1-44a4-a60e-a89336f46f4d None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:07:46.433706 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-683d68d3-11e7-42ac-85dc-492ff9af01e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:07:46.434384 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-683d68d3-11e7-42ac-85dc-492ff9af01e6 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:07:46.434690 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-683d68d3-11e7-42ac-85dc-492ff9af01e6 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:07:46.434966 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-683d68d3-11e7-42ac-85dc-492ff9af01e6 None admin] PATH_INFO: `/v3/groups/7cea0c7f4a7b4b8f977cd4c1adc36486` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:07:46.436482 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-683d68d3-11e7-42ac-85dc-492ff9af01e6 None admin] RBAC: Authorizing `identity:get_group(group_id=7cea0c7f4a7b4b8f977cd4c1adc36486)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:07:46.438595 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-683d68d3-11e7-42ac-85dc-492ff9af01e6 None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:07:46.441062 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 199/387] 10.222.0.18 () {56 vars in 1180 bytes} [Tue Sep 13 03:07:46 2022] GET /identity/v3/groups/7cea0c7f4a7b4b8f977cd4c1adc36486 => generated 226 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 03:07:46.447676 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9573b9ee-7fa8-4a2d-9d48-f8d987713d59 None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:07:46.554623 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-683d68d3-11e7-42ac-85dc-492ff9af01e6 None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:07:46.576222 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9573b9ee-7fa8-4a2d-9d48-f8d987713d59 None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:07:46.611313 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22e4ef84-e82b-468f-8f09-eee4e079d5cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:07:46.612053 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e4ef84-e82b-468f-8f09-eee4e079d5cf None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:07:46.612333 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e4ef84-e82b-468f-8f09-eee4e079d5cf None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:07:46.612605 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e4ef84-e82b-468f-8f09-eee4e079d5cf None admin] PATH_INFO: `/v3/projects/83aeaa07fb5d40d3b285aeb01d1252b1` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:07:46.615512 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22e4ef84-e82b-468f-8f09-eee4e079d5cf None admin] RBAC: Authorizing `identity:get_project(project_id=83aeaa07fb5d40d3b285aeb01d1252b1)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:07:46.616956 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22e4ef84-e82b-468f-8f09-eee4e079d5cf None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:07:46.618981 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 189/388] 10.222.0.18 () {56 vars in 1186 bytes} [Tue Sep 13 03:07:46 2022] GET /identity/v3/projects/83aeaa07fb5d40d3b285aeb01d1252b1 => generated 298 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 03:07:46.624923 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22e4ef84-e82b-468f-8f09-eee4e079d5cf None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:07:46.725257 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:07:46.727035 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:07:46.760896 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-683d68d3-11e7-42ac-85dc-492ff9af01e6 None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:07:46.789227 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22e4ef84-e82b-468f-8f09-eee4e079d5cf None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:07:46.827000 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cf9b83d-75a6-4ea8-af92-758f48d2d2be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:07:46.827671 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf9b83d-75a6-4ea8-af92-758f48d2d2be None admin] REQUEST_METHOD: `PUT` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:07:46.827938 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf9b83d-75a6-4ea8-af92-758f48d2d2be None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:07:46.828277 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf9b83d-75a6-4ea8-af92-758f48d2d2be None admin] PATH_INFO: `/v3/projects/83aeaa07fb5d40d3b285aeb01d1252b1/groups/7cea0c7f4a7b4b8f977cd4c1adc36486/roles/1eb7a5b5203945c29e6d22d653f47847` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:07:46.831662 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cf9b83d-75a6-4ea8-af92-758f48d2d2be None admin] RBAC: Authorizing `identity:create_grant(project_id=83aeaa07fb5d40d3b285aeb01d1252b1, group_id=7cea0c7f4a7b4b8f977cd4c1adc36486, role_id=1eb7a5b5203945c29e6d22d653f47847)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:07:46.832560 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56554260-0da8-4d95-9438-2f27f786f7ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:07:46.833114 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cf9b83d-75a6-4ea8-af92-758f48d2d2be None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:07:46.833630 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56554260-0da8-4d95-9438-2f27f786f7ca None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:07:46.834103 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56554260-0da8-4d95-9438-2f27f786f7ca None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:07:46.834564 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56554260-0da8-4d95-9438-2f27f786f7ca None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:07:46.849854 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 190/389] 10.222.0.18 () {58 vars in 1442 bytes} [Tue Sep 13 03:07:46 2022] PUT /identity/v3/projects/83aeaa07fb5d40d3b285aeb01d1252b1/groups/7cea0c7f4a7b4b8f977cd4c1adc36486/roles/1eb7a5b5203945c29e6d22d653f47847 => generated 0 bytes in 225 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 13 03:07:46.855206 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56554260-0da8-4d95-9438-2f27f786f7ca None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 03:07:46.861928 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application [None req-56554260-0da8-4d95-9438-2f27f786f7ca None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 13 03:07:46.861928 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 03:07:46.861928 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 03:07:46.861928 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 03:07:46.861928 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 03:07:46.861928 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 03:07:46.861928 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 03:07:46.861928 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 03:07:46.861928 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 03:07:46.861928 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 03:07:46.861928 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 03:07:46.861928 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 03:07:46.861928 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 03:07:46.861928 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 03:07:46.861928 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 03:07:46.861928 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 03:07:46.863052 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 03:07:46.863052 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 03:07:46.863052 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 03:07:46.863052 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 03:07:46.863052 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 03:07:46.863052 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 03:07:46.863052 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 03:07:46.863052 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 03:07:46.863052 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 03:07:46.863052 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 03:07:46.863052 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 03:07:46.863052 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application with Lock( Sep 13 03:07:46.863052 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 03:07:46.863052 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._enter() Sep 13 03:07:46.863052 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 03:07:46.863052 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 03:07:46.864458 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 03:07:46.864458 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.creator() Sep 13 03:07:46.864458 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 03:07:46.864458 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application created_value = creator( Sep 13 03:07:46.864458 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 03:07:46.864458 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 03:07:46.864458 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 03:07:46.864458 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 03:07:46.864458 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 03:07:46.864458 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 03:07:46.864458 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 13 03:07:46.864458 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Sep 13 03:07:46.864458 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 200/390] 10.222.0.18 () {56 vars in 1114 bytes} [Tue Sep 13 03:07:46 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 309 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 13 03:07:46.869164 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cf9b83d-75a6-4ea8-af92-758f48d2d2be None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:07:46.999191 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cf9b83d-75a6-4ea8-af92-758f48d2d2be None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:07:47.023767 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:07:47.037614 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83c8506f-b977-40a8-9e2d-103c255b5783 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:07:47.038281 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83c8506f-b977-40a8-9e2d-103c255b5783 None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:07:47.038562 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83c8506f-b977-40a8-9e2d-103c255b5783 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:07:47.038915 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83c8506f-b977-40a8-9e2d-103c255b5783 None admin] PATH_INFO: `/v3/roles` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:07:47.039655 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83c8506f-b977-40a8-9e2d-103c255b5783 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 03:07:47.043132 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83c8506f-b977-40a8-9e2d-103c255b5783 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:07:47.044575 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83c8506f-b977-40a8-9e2d-103c255b5783 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:07:47.054583 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 191/391] 10.222.0.18 () {56 vars in 1111 bytes} [Tue Sep 13 03:07:46 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 03:07:47.068401 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56554260-0da8-4d95-9438-2f27f786f7ca None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:07:47.208696 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56554260-0da8-4d95-9438-2f27f786f7ca None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:07:47.255797 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e5bdcde-1dd5-4654-b2c1-da6b5d3f7f92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:07:47.256800 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e5bdcde-1dd5-4654-b2c1-da6b5d3f7f92 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:07:47.257284 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e5bdcde-1dd5-4654-b2c1-da6b5d3f7f92 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:07:47.257748 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e5bdcde-1dd5-4654-b2c1-da6b5d3f7f92 None admin] PATH_INFO: `/v3/groups/7cea0c7f4a7b4b8f977cd4c1adc36486` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:07:47.259491 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e5bdcde-1dd5-4654-b2c1-da6b5d3f7f92 None admin] RBAC: Authorizing `identity:get_group(group_id=7cea0c7f4a7b4b8f977cd4c1adc36486)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:07:47.261159 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e5bdcde-1dd5-4654-b2c1-da6b5d3f7f92 None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:07:47.263406 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 201/392] 10.222.0.18 () {56 vars in 1180 bytes} [Tue Sep 13 03:07:47 2022] GET /identity/v3/groups/7cea0c7f4a7b4b8f977cd4c1adc36486 => generated 226 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 03:07:47.269729 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83c8506f-b977-40a8-9e2d-103c255b5783 None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:07:47.362402 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-befb03da-5d68-4187-92c8-785f63342923 None None] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:07:47.363094 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-befb03da-5d68-4187-92c8-785f63342923 None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:07:47.363558 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-befb03da-5d68-4187-92c8-785f63342923 None None] PATH_INFO: `/` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:07:47.365366 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 202/393] 10.222.0.18 () {54 vars in 927 bytes} [Tue Sep 13 03:07:47 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 03:07:47.369973 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-444b7d10-67b7-4e4b-a13e-ce0cc34d16c5 None None] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:07:47.370478 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-444b7d10-67b7-4e4b-a13e-ce0cc34d16c5 None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:07:47.370988 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-444b7d10-67b7-4e4b-a13e-ce0cc34d16c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:07:47.384046 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83c8506f-b977-40a8-9e2d-103c255b5783 None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:07:47.394885 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.auth.core [None req-444b7d10-67b7-4e4b-a13e-ce0cc34d16c5 None None] MFA Rules not processed for user `49f280ab086b4610855c4ffa378517d9`. Rule list: `[]` (Enabled: `True`). {{(pid=81400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 03:07:47.396083 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-444b7d10-67b7-4e4b-a13e-ce0cc34d16c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:07:47.398374 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:07:47.423226 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-645b7b43-715a-49b2-9ead-678fe0522772 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:07:47.423863 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-645b7b43-715a-49b2-9ead-678fe0522772 None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:07:47.424140 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-645b7b43-715a-49b2-9ead-678fe0522772 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:07:47.424410 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-645b7b43-715a-49b2-9ead-678fe0522772 None admin] PATH_INFO: `/v3/projects/83aeaa07fb5d40d3b285aeb01d1252b1` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:07:47.425540 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-645b7b43-715a-49b2-9ead-678fe0522772 None admin] RBAC: Authorizing `identity:get_project(project_id=83aeaa07fb5d40d3b285aeb01d1252b1)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:07:47.427006 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-645b7b43-715a-49b2-9ead-678fe0522772 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:07:47.428952 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 192/394] 10.222.0.18 () {56 vars in 1186 bytes} [Tue Sep 13 03:07:47 2022] GET /identity/v3/projects/83aeaa07fb5d40d3b285aeb01d1252b1 => generated 298 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 03:07:47.434574 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-645b7b43-715a-49b2-9ead-678fe0522772 None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:07:47.509170 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:07:47.537280 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 203/395] 10.222.0.18 () {58 vars in 1024 bytes} [Tue Sep 13 03:07:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 13 03:07:47.572668 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-645b7b43-715a-49b2-9ead-678fe0522772 None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:07:47.607237 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9bd1003-0cba-47b4-8e4b-de6d03452b2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:07:47.607907 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9bd1003-0cba-47b4-8e4b-de6d03452b2f None admin] REQUEST_METHOD: `PUT` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:07:47.608186 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9bd1003-0cba-47b4-8e4b-de6d03452b2f None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:07:47.608458 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9bd1003-0cba-47b4-8e4b-de6d03452b2f None admin] PATH_INFO: `/v3/projects/83aeaa07fb5d40d3b285aeb01d1252b1/groups/7cea0c7f4a7b4b8f977cd4c1adc36486/roles/a9aa35ebda4d4aff80e62f750e9104ab` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:07:47.610718 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9bd1003-0cba-47b4-8e4b-de6d03452b2f None admin] RBAC: Authorizing `identity:create_grant(project_id=83aeaa07fb5d40d3b285aeb01d1252b1, group_id=7cea0c7f4a7b4b8f977cd4c1adc36486, role_id=a9aa35ebda4d4aff80e62f750e9104ab)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:07:47.612156 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9bd1003-0cba-47b4-8e4b-de6d03452b2f None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:07:47.632692 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 193/396] 10.222.0.18 () {58 vars in 1442 bytes} [Tue Sep 13 03:07:47 2022] PUT /identity/v3/projects/83aeaa07fb5d40d3b285aeb01d1252b1/groups/7cea0c7f4a7b4b8f977cd4c1adc36486/roles/a9aa35ebda4d4aff80e62f750e9104ab => generated 0 bytes in 198 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 13 03:07:47.983011 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-444b7d10-67b7-4e4b-a13e-ce0cc34d16c5 None None] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:07:48.026772 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd1c17e6-a22a-4e9e-bd8d-fedf7e4a19b0 None None] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:07:48.027077 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd1c17e6-a22a-4e9e-bd8d-fedf7e4a19b0 None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:07:48.027319 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd1c17e6-a22a-4e9e-bd8d-fedf7e4a19b0 None None] PATH_INFO: `/` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:07:48.028651 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 194/397] 10.222.0.18 () {54 vars in 927 bytes} [Tue Sep 13 03:07:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 03:07:48.033933 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86df195b-11e3-4779-a9ea-35672d9db4da None None] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:07:48.034212 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86df195b-11e3-4779-a9ea-35672d9db4da None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:07:48.034477 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86df195b-11e3-4779-a9ea-35672d9db4da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:07:48.065832 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.auth.core [None req-86df195b-11e3-4779-a9ea-35672d9db4da None None] MFA Rules not processed for user `49f280ab086b4610855c4ffa378517d9`. Rule list: `[]` (Enabled: `True`). {{(pid=81399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 03:07:48.068193 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-86df195b-11e3-4779-a9ea-35672d9db4da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:07:48.147689 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-444b7d10-67b7-4e4b-a13e-ce0cc34d16c5 None None] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:07:48.187748 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f487f9cb-84ff-498f-b167-b5ecae87cacd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:07:48.188380 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f487f9cb-84ff-498f-b167-b5ecae87cacd None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:07:48.188668 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f487f9cb-84ff-498f-b167-b5ecae87cacd None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:07:48.188951 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f487f9cb-84ff-498f-b167-b5ecae87cacd None admin] PATH_INFO: `/v3/roles/member` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:07:48.203356 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:07:48.204243 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 195/398] 10.222.0.18 () {58 vars in 1024 bytes} [Tue Sep 13 03:07:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 13 03:07:48.212788 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f487f9cb-84ff-498f-b167-b5ecae87cacd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 03:07:48.217989 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application [None req-f487f9cb-84ff-498f-b167-b5ecae87cacd None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 13 03:07:48.217989 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 03:07:48.217989 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 03:07:48.217989 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 03:07:48.217989 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 03:07:48.217989 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 03:07:48.217989 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 03:07:48.217989 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 03:07:48.217989 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 03:07:48.217989 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 03:07:48.217989 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 03:07:48.217989 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 03:07:48.217989 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 03:07:48.217989 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 03:07:48.217989 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 03:07:48.217989 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 03:07:48.219459 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 03:07:48.219459 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 03:07:48.219459 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 03:07:48.219459 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 03:07:48.219459 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 03:07:48.219459 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 03:07:48.219459 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 03:07:48.219459 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 03:07:48.219459 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 03:07:48.219459 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 03:07:48.219459 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 03:07:48.219459 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application with Lock( Sep 13 03:07:48.219459 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 03:07:48.219459 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._enter() Sep 13 03:07:48.219459 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 03:07:48.219459 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 03:07:48.220273 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 03:07:48.220273 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.creator() Sep 13 03:07:48.220273 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 03:07:48.220273 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application created_value = creator( Sep 13 03:07:48.220273 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 03:07:48.220273 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 03:07:48.220273 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 03:07:48.220273 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 03:07:48.220273 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 03:07:48.220273 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 03:07:48.220273 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 13 03:07:48.220273 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Sep 13 03:07:48.220273 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 204/399] 10.222.0.18 () {56 vars in 1099 bytes} [Tue Sep 13 03:07:47 2022] GET /identity/v3/roles/member => generated 84 bytes in 237 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 03:07:48.227816 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86df195b-11e3-4779-a9ea-35672d9db4da None None] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:07:48.264250 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:07:48.343593 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86df195b-11e3-4779-a9ea-35672d9db4da None None] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:07:48.378250 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-255c0928-c55c-4dc9-83fd-e8a290345d67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:07:48.378895 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-255c0928-c55c-4dc9-83fd-e8a290345d67 None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:07:48.379172 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-255c0928-c55c-4dc9-83fd-e8a290345d67 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:07:48.379438 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-255c0928-c55c-4dc9-83fd-e8a290345d67 None admin] PATH_INFO: `/v3/roles` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:07:48.380099 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-255c0928-c55c-4dc9-83fd-e8a290345d67 None admin] RBAC: Adding query filter params (name=member) {{(pid=81399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 03:07:48.380398 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-255c0928-c55c-4dc9-83fd-e8a290345d67 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:07:48.381774 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-255c0928-c55c-4dc9-83fd-e8a290345d67 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:07:48.390293 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 196/400] 10.222.0.18 () {56 vars in 1101 bytes} [Tue Sep 13 03:07:48 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 03:07:48.399904 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f487f9cb-84ff-498f-b167-b5ecae87cacd None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:07:48.553833 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f487f9cb-84ff-498f-b167-b5ecae87cacd None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:07:48.589142 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f164757b-ea85-46f4-9641-6d769645e288 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:07:48.589695 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-255c0928-c55c-4dc9-83fd-e8a290345d67 None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:07:48.590150 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f164757b-ea85-46f4-9641-6d769645e288 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:07:48.590150 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f164757b-ea85-46f4-9641-6d769645e288 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:07:48.590377 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f164757b-ea85-46f4-9641-6d769645e288 None admin] PATH_INFO: `/v3/groups/7cea0c7f4a7b4b8f977cd4c1adc36486` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:07:48.591510 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f164757b-ea85-46f4-9641-6d769645e288 None admin] RBAC: Authorizing `identity:get_group(group_id=7cea0c7f4a7b4b8f977cd4c1adc36486)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:07:48.593082 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f164757b-ea85-46f4-9641-6d769645e288 None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:07:48.595134 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 205/401] 10.222.0.18 () {56 vars in 1180 bytes} [Tue Sep 13 03:07:48 2022] GET /identity/v3/groups/7cea0c7f4a7b4b8f977cd4c1adc36486 => generated 226 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 03:07:48.603628 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f164757b-ea85-46f4-9641-6d769645e288 None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:07:48.709668 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:07:48.728043 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:07:48.742840 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-255c0928-c55c-4dc9-83fd-e8a290345d67 None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:07:48.758048 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f164757b-ea85-46f4-9641-6d769645e288 None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:07:48.777935 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c711653a-467f-49b1-8ed7-a5beb22d7f6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:07:48.778574 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c711653a-467f-49b1-8ed7-a5beb22d7f6b None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:07:48.778873 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c711653a-467f-49b1-8ed7-a5beb22d7f6b None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:07:48.779141 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c711653a-467f-49b1-8ed7-a5beb22d7f6b None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:07:48.789920 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c711653a-467f-49b1-8ed7-a5beb22d7f6b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 03:07:48.795444 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application [None req-c711653a-467f-49b1-8ed7-a5beb22d7f6b None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 13 03:07:48.795444 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 03:07:48.795444 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 03:07:48.795444 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 03:07:48.795444 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 03:07:48.795444 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 03:07:48.795444 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 03:07:48.795444 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 03:07:48.795444 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 03:07:48.795444 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 03:07:48.795444 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 03:07:48.795444 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 03:07:48.795444 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 03:07:48.795444 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 03:07:48.795444 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 03:07:48.795444 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 03:07:48.796325 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 03:07:48.796325 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 03:07:48.796325 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 03:07:48.796325 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 03:07:48.796325 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 03:07:48.796325 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 03:07:48.796325 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 03:07:48.796325 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 03:07:48.796325 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 03:07:48.796325 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 03:07:48.796325 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 03:07:48.796325 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application with Lock( Sep 13 03:07:48.796325 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 03:07:48.796325 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._enter() Sep 13 03:07:48.796325 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 03:07:48.796325 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 03:07:48.797165 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 03:07:48.797165 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.creator() Sep 13 03:07:48.797165 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 03:07:48.797165 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application created_value = creator( Sep 13 03:07:48.797165 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 03:07:48.797165 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 03:07:48.797165 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 03:07:48.797165 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 03:07:48.797165 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 03:07:48.797165 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 03:07:48.797165 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 13 03:07:48.797165 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Sep 13 03:07:48.797165 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 197/402] 10.222.0.18 () {56 vars in 1114 bytes} [Tue Sep 13 03:07:48 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 03:07:48.802974 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c02bab7-619e-4284-81a1-340fd1eb4b1a None None] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:07:48.802974 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c02bab7-619e-4284-81a1-340fd1eb4b1a None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:07:48.802974 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c02bab7-619e-4284-81a1-340fd1eb4b1a None None] PATH_INFO: `/` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:07:48.804342 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 198/403] 10.222.0.18 () {54 vars in 927 bytes} [Tue Sep 13 03:07:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 03:07:48.805569 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c02bab7-619e-4284-81a1-340fd1eb4b1a None None] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:07:48.811600 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fe0595a-7830-46c2-adcb-83220777a552 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:07:48.812586 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe0595a-7830-46c2-adcb-83220777a552 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:07:48.813164 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe0595a-7830-46c2-adcb-83220777a552 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:07:48.813646 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe0595a-7830-46c2-adcb-83220777a552 None admin] PATH_INFO: `/v3/projects/54999ea237f64c5f972ef79e765facd0` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:07:48.815853 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fe0595a-7830-46c2-adcb-83220777a552 None admin] RBAC: Authorizing `identity:get_project(project_id=54999ea237f64c5f972ef79e765facd0)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:07:48.817680 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fe0595a-7830-46c2-adcb-83220777a552 None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:07:48.821378 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 206/404] 10.222.0.18 () {56 vars in 1186 bytes} [Tue Sep 13 03:07:48 2022] GET /identity/v3/projects/54999ea237f64c5f972ef79e765facd0 => generated 302 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 03:07:48.823857 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08db22ef-dc94-42fb-ae86-c9ffab156170 None None] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:07:48.824339 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08db22ef-dc94-42fb-ae86-c9ffab156170 None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:07:48.825045 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08db22ef-dc94-42fb-ae86-c9ffab156170 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:07:48.850312 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.auth.core [None req-08db22ef-dc94-42fb-ae86-c9ffab156170 None None] MFA Rules not processed for user `49f280ab086b4610855c4ffa378517d9`. Rule list: `[]` (Enabled: `True`). {{(pid=81400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 03:07:48.852408 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-08db22ef-dc94-42fb-ae86-c9ffab156170 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:07:48.916281 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c02bab7-619e-4284-81a1-340fd1eb4b1a None None] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:07:48.937375 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 207/405] 10.222.0.18 () {58 vars in 1024 bytes} [Tue Sep 13 03:07:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 13 03:07:48.939056 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08db22ef-dc94-42fb-ae86-c9ffab156170 None None] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:07:48.969491 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf063d96-bb0e-4baa-a1d9-6dc80555242d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:07:48.970477 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf063d96-bb0e-4baa-a1d9-6dc80555242d None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:07:48.971042 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf063d96-bb0e-4baa-a1d9-6dc80555242d None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:07:48.972127 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf063d96-bb0e-4baa-a1d9-6dc80555242d None admin] PATH_INFO: `/v3/roles` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:07:48.973032 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf063d96-bb0e-4baa-a1d9-6dc80555242d None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 03:07:48.973546 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf063d96-bb0e-4baa-a1d9-6dc80555242d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:07:48.975355 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf063d96-bb0e-4baa-a1d9-6dc80555242d None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:07:49.000784 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 199/406] 10.222.0.18 () {56 vars in 1111 bytes} [Tue Sep 13 03:07:48 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 03:07:49.010645 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf063d96-bb0e-4baa-a1d9-6dc80555242d None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:07:49.031361 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:07:49.043787 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:07:49.083782 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08db22ef-dc94-42fb-ae86-c9ffab156170 None None] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:07:49.122637 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-420046de-3179-4425-b103-46c2f7a56317 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:07:49.123307 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-420046de-3179-4425-b103-46c2f7a56317 None admin] REQUEST_METHOD: `PUT` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:07:49.123584 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-420046de-3179-4425-b103-46c2f7a56317 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:07:49.124176 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-420046de-3179-4425-b103-46c2f7a56317 None admin] PATH_INFO: `/v3/projects/54999ea237f64c5f972ef79e765facd0/groups/7cea0c7f4a7b4b8f977cd4c1adc36486/roles/1eb7a5b5203945c29e6d22d653f47847` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:07:49.126780 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-420046de-3179-4425-b103-46c2f7a56317 None admin] RBAC: Authorizing `identity:create_grant(project_id=54999ea237f64c5f972ef79e765facd0, group_id=7cea0c7f4a7b4b8f977cd4c1adc36486, role_id=1eb7a5b5203945c29e6d22d653f47847)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:07:49.128232 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-420046de-3179-4425-b103-46c2f7a56317 None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:07:49.145442 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 208/407] 10.222.0.18 () {58 vars in 1442 bytes} [Tue Sep 13 03:07:48 2022] PUT /identity/v3/projects/54999ea237f64c5f972ef79e765facd0/groups/7cea0c7f4a7b4b8f977cd4c1adc36486/roles/1eb7a5b5203945c29e6d22d653f47847 => generated 0 bytes in 207 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 03:07:49.231625 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf063d96-bb0e-4baa-a1d9-6dc80555242d None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:07:49.247750 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-420046de-3179-4425-b103-46c2f7a56317 None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:07:49.299353 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a47c485f-ddaa-495e-bf11-6423c9d4bccc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:07:49.300565 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a47c485f-ddaa-495e-bf11-6423c9d4bccc None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:07:49.300877 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a47c485f-ddaa-495e-bf11-6423c9d4bccc None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:07:49.301137 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a47c485f-ddaa-495e-bf11-6423c9d4bccc None admin] PATH_INFO: `/v3/groups/7cea0c7f4a7b4b8f977cd4c1adc36486` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:07:49.303427 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a47c485f-ddaa-495e-bf11-6423c9d4bccc None admin] RBAC: Authorizing `identity:get_group(group_id=7cea0c7f4a7b4b8f977cd4c1adc36486)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:07:49.305412 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a47c485f-ddaa-495e-bf11-6423c9d4bccc None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:07:49.313809 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 200/408] 10.222.0.18 () {56 vars in 1180 bytes} [Tue Sep 13 03:07:49 2022] GET /identity/v3/groups/7cea0c7f4a7b4b8f977cd4c1adc36486 => generated 226 bytes in 303 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 03:07:49.315128 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a47c485f-ddaa-495e-bf11-6423c9d4bccc None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:07:49.412789 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-420046de-3179-4425-b103-46c2f7a56317 None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:07:49.446577 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:07:49.454276 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:07:49.463473 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6857b668-1159-48f0-a6e1-618d0ae2b06c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:07:49.464861 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6857b668-1159-48f0-a6e1-618d0ae2b06c None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:07:49.465303 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6857b668-1159-48f0-a6e1-618d0ae2b06c None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:07:49.465712 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6857b668-1159-48f0-a6e1-618d0ae2b06c None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:07:49.478046 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a47c485f-ddaa-495e-bf11-6423c9d4bccc None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:07:49.479845 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6857b668-1159-48f0-a6e1-618d0ae2b06c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 03:07:49.487261 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application [None req-6857b668-1159-48f0-a6e1-618d0ae2b06c None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 13 03:07:49.487261 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 03:07:49.487261 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 03:07:49.487261 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 03:07:49.487261 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 03:07:49.487261 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 03:07:49.487261 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 03:07:49.487261 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 03:07:49.487261 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 03:07:49.487261 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 03:07:49.487261 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 03:07:49.487261 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 03:07:49.487261 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 03:07:49.487261 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 03:07:49.487261 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 03:07:49.487261 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 03:07:49.488139 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 03:07:49.488139 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 03:07:49.488139 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 03:07:49.488139 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 03:07:49.488139 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 03:07:49.488139 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 03:07:49.488139 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 03:07:49.488139 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 03:07:49.488139 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 03:07:49.488139 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 03:07:49.488139 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 03:07:49.488139 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application with Lock( Sep 13 03:07:49.488139 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 03:07:49.488139 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._enter() Sep 13 03:07:49.488139 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 03:07:49.488139 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 03:07:49.488946 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 03:07:49.488946 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.creator() Sep 13 03:07:49.488946 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 03:07:49.488946 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application created_value = creator( Sep 13 03:07:49.488946 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 03:07:49.488946 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 03:07:49.488946 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 03:07:49.488946 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 03:07:49.488946 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 03:07:49.488946 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 03:07:49.488946 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 13 03:07:49.488946 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Sep 13 03:07:49.491720 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 209/409] 10.222.0.18 () {56 vars in 1096 bytes} [Tue Sep 13 03:07:49 2022] GET /identity/v3/roles/admin => generated 83 bytes in 245 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 03:07:49.497492 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6857b668-1159-48f0-a6e1-618d0ae2b06c None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:07:49.522330 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ead4aad3-0945-47a4-bf3c-ae8acef14ac3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:07:49.522988 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead4aad3-0945-47a4-bf3c-ae8acef14ac3 None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:07:49.523264 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead4aad3-0945-47a4-bf3c-ae8acef14ac3 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:07:49.523531 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead4aad3-0945-47a4-bf3c-ae8acef14ac3 None admin] PATH_INFO: `/v3/projects/54999ea237f64c5f972ef79e765facd0` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:07:49.524948 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ead4aad3-0945-47a4-bf3c-ae8acef14ac3 None admin] RBAC: Authorizing `identity:get_project(project_id=54999ea237f64c5f972ef79e765facd0)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:07:49.526659 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ead4aad3-0945-47a4-bf3c-ae8acef14ac3 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:07:49.528902 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 201/410] 10.222.0.18 () {56 vars in 1186 bytes} [Tue Sep 13 03:07:49 2022] GET /identity/v3/projects/54999ea237f64c5f972ef79e765facd0 => generated 302 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 03:07:49.535707 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ead4aad3-0945-47a4-bf3c-ae8acef14ac3 None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:07:49.653408 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ead4aad3-0945-47a4-bf3c-ae8acef14ac3 None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:07:49.657466 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6857b668-1159-48f0-a6e1-618d0ae2b06c None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:07:49.691704 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-690ecf7b-211e-4aa4-b17e-1309daed4cd1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:07:49.692419 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-690ecf7b-211e-4aa4-b17e-1309daed4cd1 None admin] REQUEST_METHOD: `PUT` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:07:49.692700 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-690ecf7b-211e-4aa4-b17e-1309daed4cd1 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:07:49.692967 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-690ecf7b-211e-4aa4-b17e-1309daed4cd1 None admin] PATH_INFO: `/v3/projects/54999ea237f64c5f972ef79e765facd0/groups/7cea0c7f4a7b4b8f977cd4c1adc36486/roles/a9aa35ebda4d4aff80e62f750e9104ab` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:07:49.695157 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-690ecf7b-211e-4aa4-b17e-1309daed4cd1 None admin] RBAC: Authorizing `identity:create_grant(project_id=54999ea237f64c5f972ef79e765facd0, group_id=7cea0c7f4a7b4b8f977cd4c1adc36486, role_id=a9aa35ebda4d4aff80e62f750e9104ab)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:07:49.696176 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3bdd9d3-784e-40d7-90c1-d9ea6d105f0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:07:49.696704 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-690ecf7b-211e-4aa4-b17e-1309daed4cd1 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:07:49.696811 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3bdd9d3-784e-40d7-90c1-d9ea6d105f0f None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:07:49.697041 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3bdd9d3-784e-40d7-90c1-d9ea6d105f0f None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:07:49.697311 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3bdd9d3-784e-40d7-90c1-d9ea6d105f0f None admin] PATH_INFO: `/v3/roles` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:07:49.697977 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3bdd9d3-784e-40d7-90c1-d9ea6d105f0f None admin] RBAC: Adding query filter params (name=admin) {{(pid=81400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 03:07:49.698271 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3bdd9d3-784e-40d7-90c1-d9ea6d105f0f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:07:49.700234 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3bdd9d3-784e-40d7-90c1-d9ea6d105f0f None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:07:49.710391 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 210/411] 10.222.0.18 () {56 vars in 1099 bytes} [Tue Sep 13 03:07:49 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 03:07:49.711788 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-192c10d5-2e28-4619-8ec7-2a9b9efb4516 None None] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:07:49.712071 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-192c10d5-2e28-4619-8ec7-2a9b9efb4516 None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:07:49.712199 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 202/412] 10.222.0.18 () {58 vars in 1442 bytes} [Tue Sep 13 03:07:49 2022] PUT /identity/v3/projects/54999ea237f64c5f972ef79e765facd0/groups/7cea0c7f4a7b4b8f977cd4c1adc36486/roles/a9aa35ebda4d4aff80e62f750e9104ab => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 03:07:49.712393 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-192c10d5-2e28-4619-8ec7-2a9b9efb4516 None None] PATH_INFO: `/` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:07:49.713892 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 211/413] 10.222.0.18 () {54 vars in 927 bytes} [Tue Sep 13 03:07:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 03:07:49.716454 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-690ecf7b-211e-4aa4-b17e-1309daed4cd1 None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:07:49.720212 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d069b604-3311-4a75-a058-eaf34b67ef03 None None] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:07:49.720494 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d069b604-3311-4a75-a058-eaf34b67ef03 None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:07:49.720768 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d069b604-3311-4a75-a058-eaf34b67ef03 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:07:49.731867 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:07:49.740853 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.auth.core [None req-d069b604-3311-4a75-a058-eaf34b67ef03 None None] MFA Rules not processed for user `49f280ab086b4610855c4ffa378517d9`. Rule list: `[]` (Enabled: `True`). {{(pid=81400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 03:07:49.742057 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-d069b604-3311-4a75-a058-eaf34b67ef03 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:07:49.810628 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:07:49.868370 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 212/414] 10.222.0.18 () {58 vars in 1024 bytes} [Tue Sep 13 03:07:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 13 03:07:49.885332 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-690ecf7b-211e-4aa4-b17e-1309daed4cd1 None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:07:49.945819 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43a89f3a-3738-46de-b9b8-8759089b3d8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:07:49.946377 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43a89f3a-3738-46de-b9b8-8759089b3d8c None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:07:49.946679 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43a89f3a-3738-46de-b9b8-8759089b3d8c None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:07:49.946956 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43a89f3a-3738-46de-b9b8-8759089b3d8c None admin] PATH_INFO: `/v3/groups/228dcaf0a4474b2b804468d12ab54ded` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:07:49.948064 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43a89f3a-3738-46de-b9b8-8759089b3d8c None admin] RBAC: Authorizing `identity:get_group(group_id=228dcaf0a4474b2b804468d12ab54ded)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:07:49.949490 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43a89f3a-3738-46de-b9b8-8759089b3d8c None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:07:49.951101 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 203/415] 10.222.0.18 () {56 vars in 1180 bytes} [Tue Sep 13 03:07:49 2022] GET /identity/v3/groups/228dcaf0a4474b2b804468d12ab54ded => generated 229 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 03:07:49.956766 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d069b604-3311-4a75-a058-eaf34b67ef03 None None] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:07:50.072889 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d069b604-3311-4a75-a058-eaf34b67ef03 None None] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:07:50.119152 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-483d86af-25de-4e28-a6ed-cfbcebcaa063 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:07:50.119789 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-483d86af-25de-4e28-a6ed-cfbcebcaa063 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:07:50.120063 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-483d86af-25de-4e28-a6ed-cfbcebcaa063 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:07:50.120331 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-483d86af-25de-4e28-a6ed-cfbcebcaa063 None admin] PATH_INFO: `/v3/projects/76c5181fc7be45279ac00017bb4640b3` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:07:50.121459 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-483d86af-25de-4e28-a6ed-cfbcebcaa063 None admin] RBAC: Authorizing `identity:get_project(project_id=76c5181fc7be45279ac00017bb4640b3)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:07:50.122930 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-483d86af-25de-4e28-a6ed-cfbcebcaa063 None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:07:50.124889 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 213/416] 10.222.0.18 () {56 vars in 1186 bytes} [Tue Sep 13 03:07:49 2022] GET /identity/v3/projects/76c5181fc7be45279ac00017bb4640b3 => generated 344 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 03:07:50.130164 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43a89f3a-3738-46de-b9b8-8759089b3d8c None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:07:50.219803 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-483d86af-25de-4e28-a6ed-cfbcebcaa063 None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:07:50.242994 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:07:50.267494 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43a89f3a-3738-46de-b9b8-8759089b3d8c None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:07:50.303718 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a496455e-21fd-49eb-8198-5df9b69fbca3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:07:50.304495 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a496455e-21fd-49eb-8198-5df9b69fbca3 None admin] REQUEST_METHOD: `PUT` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:07:50.304775 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a496455e-21fd-49eb-8198-5df9b69fbca3 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:07:50.305042 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a496455e-21fd-49eb-8198-5df9b69fbca3 None admin] PATH_INFO: `/v3/projects/76c5181fc7be45279ac00017bb4640b3/groups/228dcaf0a4474b2b804468d12ab54ded/roles/425d4b481c7846e0bc07e7e805d5b15f` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:07:50.307331 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a496455e-21fd-49eb-8198-5df9b69fbca3 None admin] RBAC: Authorizing `identity:create_grant(project_id=76c5181fc7be45279ac00017bb4640b3, group_id=228dcaf0a4474b2b804468d12ab54ded, role_id=425d4b481c7846e0bc07e7e805d5b15f)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:07:50.308831 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a496455e-21fd-49eb-8198-5df9b69fbca3 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:07:50.313089 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:07:50.335335 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 204/417] 10.222.0.18 () {58 vars in 1442 bytes} [Tue Sep 13 03:07:50 2022] PUT /identity/v3/projects/76c5181fc7be45279ac00017bb4640b3/groups/228dcaf0a4474b2b804468d12ab54ded/roles/425d4b481c7846e0bc07e7e805d5b15f => generated 0 bytes in 205 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 03:07:50.353840 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6c646e7-3100-44a5-9d74-ed94d3fd417a None None] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:07:50.354188 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6c646e7-3100-44a5-9d74-ed94d3fd417a None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:07:50.354392 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6c646e7-3100-44a5-9d74-ed94d3fd417a None None] PATH_INFO: `/` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:07:50.355942 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 205/418] 10.222.0.18 () {54 vars in 927 bytes} [Tue Sep 13 03:07:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 13 03:07:50.373146 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-337116ed-b375-46da-a9bc-ba94a42af34b None None] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:07:50.373454 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-337116ed-b375-46da-a9bc-ba94a42af34b None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:07:50.373694 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-337116ed-b375-46da-a9bc-ba94a42af34b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:07:50.403538 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.auth.core [None req-337116ed-b375-46da-a9bc-ba94a42af34b None None] MFA Rules not processed for user `49f280ab086b4610855c4ffa378517d9`. Rule list: `[]` (Enabled: `True`). {{(pid=81399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 03:07:50.410894 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-337116ed-b375-46da-a9bc-ba94a42af34b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:07:50.420692 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-483d86af-25de-4e28-a6ed-cfbcebcaa063 None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:07:50.483494 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38b31863-23da-4244-bafa-62b57dc70f56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:07:50.484376 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38b31863-23da-4244-bafa-62b57dc70f56 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:07:50.484655 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38b31863-23da-4244-bafa-62b57dc70f56 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:07:50.484970 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38b31863-23da-4244-bafa-62b57dc70f56 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:07:50.497590 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38b31863-23da-4244-bafa-62b57dc70f56 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 03:07:50.503458 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application [None req-38b31863-23da-4244-bafa-62b57dc70f56 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 13 03:07:50.503458 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 03:07:50.503458 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 03:07:50.503458 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 03:07:50.503458 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 03:07:50.503458 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 03:07:50.503458 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 03:07:50.503458 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 03:07:50.503458 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 03:07:50.503458 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 03:07:50.503458 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 03:07:50.503458 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 03:07:50.503458 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 03:07:50.503458 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 03:07:50.503458 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 03:07:50.503458 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 03:07:50.504397 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 03:07:50.504397 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 03:07:50.504397 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 03:07:50.504397 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 03:07:50.504397 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 03:07:50.504397 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 03:07:50.504397 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 03:07:50.504397 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 03:07:50.504397 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 03:07:50.504397 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 03:07:50.504397 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 03:07:50.504397 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application with Lock( Sep 13 03:07:50.504397 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 03:07:50.504397 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._enter() Sep 13 03:07:50.504397 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 03:07:50.504397 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 03:07:50.505200 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 03:07:50.505200 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.creator() Sep 13 03:07:50.505200 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 03:07:50.505200 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application created_value = creator( Sep 13 03:07:50.505200 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 03:07:50.505200 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 03:07:50.505200 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 03:07:50.505200 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 03:07:50.505200 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 03:07:50.505200 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 03:07:50.505200 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 13 03:07:50.505200 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Sep 13 03:07:50.506349 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 214/419] 10.222.0.18 () {56 vars in 1099 bytes} [Tue Sep 13 03:07:50 2022] GET /identity/v3/roles/member => generated 84 bytes in 287 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 03:07:50.512785 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38b31863-23da-4244-bafa-62b57dc70f56 None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:07:50.574455 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 206/420] 10.222.0.18 () {58 vars in 1024 bytes} [Tue Sep 13 03:07:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 202 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 13 03:07:50.641698 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38b31863-23da-4244-bafa-62b57dc70f56 None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:07:50.644939 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:07:50.678999 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2423b257-4245-47fb-8276-3704cff110c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:07:50.679606 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2423b257-4245-47fb-8276-3704cff110c1 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:07:50.679882 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2423b257-4245-47fb-8276-3704cff110c1 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:07:50.680401 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2423b257-4245-47fb-8276-3704cff110c1 None admin] PATH_INFO: `/v3/roles` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:07:50.681063 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2423b257-4245-47fb-8276-3704cff110c1 None admin] RBAC: Adding query filter params (name=member) {{(pid=81400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 03:07:50.681359 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2423b257-4245-47fb-8276-3704cff110c1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:07:50.682748 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2423b257-4245-47fb-8276-3704cff110c1 None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:07:50.695418 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 215/421] 10.222.0.18 () {56 vars in 1101 bytes} [Tue Sep 13 03:07:50 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 03:07:50.701507 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-337116ed-b375-46da-a9bc-ba94a42af34b None None] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:07:50.806901 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:07:50.866644 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-337116ed-b375-46da-a9bc-ba94a42af34b None None] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:07:50.916448 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2423b257-4245-47fb-8276-3704cff110c1 None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:07:50.936338 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c0974cb-ec8b-4b38-bfdb-2d3bd0a7a4b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:07:50.937302 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c0974cb-ec8b-4b38-bfdb-2d3bd0a7a4b5 None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:07:50.938258 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c0974cb-ec8b-4b38-bfdb-2d3bd0a7a4b5 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:07:50.938674 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c0974cb-ec8b-4b38-bfdb-2d3bd0a7a4b5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:07:51.066317 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c0974cb-ec8b-4b38-bfdb-2d3bd0a7a4b5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:07:51.068874 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c0974cb-ec8b-4b38-bfdb-2d3bd0a7a4b5 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:07:51.141375 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2423b257-4245-47fb-8276-3704cff110c1 None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:07:51.157414 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:07:51.194873 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 207/422] 10.222.0.18 () {58 vars in 1282 bytes} [Tue Sep 13 03:07:50 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 494 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 13 03:07:51.200309 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c0974cb-ec8b-4b38-bfdb-2d3bd0a7a4b5 None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:07:51.259792 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a091939a-834a-4a33-bc35-89ac535c4ce8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:07:51.260984 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a091939a-834a-4a33-bc35-89ac535c4ce8 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:07:51.261470 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a091939a-834a-4a33-bc35-89ac535c4ce8 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:07:51.261955 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a091939a-834a-4a33-bc35-89ac535c4ce8 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:07:51.288802 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a091939a-834a-4a33-bc35-89ac535c4ce8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 03:07:51.302626 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application [None req-a091939a-834a-4a33-bc35-89ac535c4ce8 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 13 03:07:51.302626 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 03:07:51.302626 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 03:07:51.302626 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 03:07:51.302626 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 03:07:51.302626 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 03:07:51.302626 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 03:07:51.302626 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 03:07:51.302626 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 03:07:51.302626 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 03:07:51.302626 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 03:07:51.302626 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 03:07:51.302626 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 03:07:51.302626 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 03:07:51.302626 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 03:07:51.302626 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 03:07:51.303945 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 03:07:51.303945 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 03:07:51.303945 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 03:07:51.303945 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 03:07:51.303945 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 03:07:51.303945 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 03:07:51.303945 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 03:07:51.303945 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 03:07:51.303945 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 03:07:51.303945 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 03:07:51.303945 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 03:07:51.303945 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application with Lock( Sep 13 03:07:51.303945 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 03:07:51.303945 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._enter() Sep 13 03:07:51.303945 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 03:07:51.303945 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 03:07:51.304837 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 03:07:51.304837 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.creator() Sep 13 03:07:51.304837 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 03:07:51.304837 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application created_value = creator( Sep 13 03:07:51.304837 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 03:07:51.304837 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 03:07:51.304837 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 03:07:51.304837 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 03:07:51.304837 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 03:07:51.304837 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 03:07:51.304837 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 13 03:07:51.304837 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Sep 13 03:07:51.306876 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 216/423] 10.222.0.18 () {56 vars in 1114 bytes} [Tue Sep 13 03:07:50 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 388 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 03:07:51.311581 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a091939a-834a-4a33-bc35-89ac535c4ce8 None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:07:51.321262 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c0974cb-ec8b-4b38-bfdb-2d3bd0a7a4b5 None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:07:51.360179 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67206eb5-b2eb-43b8-bd6f-6a8494389707 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:07:51.360826 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67206eb5-b2eb-43b8-bd6f-6a8494389707 None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:07:51.361147 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67206eb5-b2eb-43b8-bd6f-6a8494389707 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:07:51.361430 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67206eb5-b2eb-43b8-bd6f-6a8494389707 None admin] PATH_INFO: `/v3/projects/83aeaa07fb5d40d3b285aeb01d1252b1` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:07:51.362540 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67206eb5-b2eb-43b8-bd6f-6a8494389707 None admin] RBAC: Authorizing `identity:get_project(project_id=83aeaa07fb5d40d3b285aeb01d1252b1)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:07:51.364001 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67206eb5-b2eb-43b8-bd6f-6a8494389707 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:07:51.365954 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 208/424] 10.222.0.18 () {56 vars in 1186 bytes} [Tue Sep 13 03:07:51 2022] GET /identity/v3/projects/83aeaa07fb5d40d3b285aeb01d1252b1 => generated 298 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 03:07:51.373454 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:07:51.374390 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67206eb5-b2eb-43b8-bd6f-6a8494389707 None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:07:51.444003 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:07:51.493793 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a091939a-834a-4a33-bc35-89ac535c4ce8 None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:07:51.557805 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67206eb5-b2eb-43b8-bd6f-6a8494389707 None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:07:51.561844 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d4b9000-1fbd-408f-8a22-5a46f5d7ced3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:07:51.562781 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d4b9000-1fbd-408f-8a22-5a46f5d7ced3 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:07:51.563253 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d4b9000-1fbd-408f-8a22-5a46f5d7ced3 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:07:51.563725 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d4b9000-1fbd-408f-8a22-5a46f5d7ced3 None admin] PATH_INFO: `/v3/roles` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:07:51.564566 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d4b9000-1fbd-408f-8a22-5a46f5d7ced3 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 03:07:51.565070 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d4b9000-1fbd-408f-8a22-5a46f5d7ced3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:07:51.566649 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d4b9000-1fbd-408f-8a22-5a46f5d7ced3 None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:07:51.583057 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 217/425] 10.222.0.18 () {56 vars in 1111 bytes} [Tue Sep 13 03:07:51 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 272 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 03:07:51.592035 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d4b9000-1fbd-408f-8a22-5a46f5d7ced3 None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:07:51.623748 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b515162-d58b-41d6-aa62-1d82609d0fd8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:07:51.624916 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b515162-d58b-41d6-aa62-1d82609d0fd8 None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:07:51.626430 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b515162-d58b-41d6-aa62-1d82609d0fd8 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:07:51.626953 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b515162-d58b-41d6-aa62-1d82609d0fd8 None admin] PATH_INFO: `/v3/groups/7cea0c7f4a7b4b8f977cd4c1adc36486` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:07:51.629419 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b515162-d58b-41d6-aa62-1d82609d0fd8 None admin] RBAC: Authorizing `identity:get_group(group_id=7cea0c7f4a7b4b8f977cd4c1adc36486)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:07:51.631259 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b515162-d58b-41d6-aa62-1d82609d0fd8 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:07:51.635898 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 209/426] 10.222.0.18 () {56 vars in 1180 bytes} [Tue Sep 13 03:07:51 2022] GET /identity/v3/groups/7cea0c7f4a7b4b8f977cd4c1adc36486 => generated 226 bytes in 262 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 03:07:51.641903 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b515162-d58b-41d6-aa62-1d82609d0fd8 None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:07:51.745074 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:07:51.761975 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d4b9000-1fbd-408f-8a22-5a46f5d7ced3 None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:07:51.812419 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b515162-d58b-41d6-aa62-1d82609d0fd8 None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:07:51.814734 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:07:51.825288 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-684f9e7f-4f95-46b9-ac60-58c4fecef08a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:07:51.825914 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-684f9e7f-4f95-46b9-ac60-58c4fecef08a None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:07:51.826188 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-684f9e7f-4f95-46b9-ac60-58c4fecef08a None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:07:51.826456 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-684f9e7f-4f95-46b9-ac60-58c4fecef08a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:07:51.894194 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e45f5e6d-2580-44d9-b3f6-e71ce3cb7f0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:07:51.894754 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e45f5e6d-2580-44d9-b3f6-e71ce3cb7f0a None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:07:51.895035 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e45f5e6d-2580-44d9-b3f6-e71ce3cb7f0a None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:07:51.895314 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e45f5e6d-2580-44d9-b3f6-e71ce3cb7f0a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:07:51.896036 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e45f5e6d-2580-44d9-b3f6-e71ce3cb7f0a None admin] RBAC: Adding query filter params (group.id=7cea0c7f4a7b4b8f977cd4c1adc36486, role.id=1eb7a5b5203945c29e6d22d653f47847, scope.project.id=83aeaa07fb5d40d3b285aeb01d1252b1) {{(pid=81399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 03:07:51.896339 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e45f5e6d-2580-44d9-b3f6-e71ce3cb7f0a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:07:51.897772 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e45f5e6d-2580-44d9-b3f6-e71ce3cb7f0a None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:07:51.913507 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 210/427] 10.222.0.18 () {56 vars in 1376 bytes} [Tue Sep 13 03:07:51 2022] GET /identity/v3/role_assignments?group.id=7cea0c7f4a7b4b8f977cd4c1adc36486&scope.project.id=83aeaa07fb5d40d3b285aeb01d1252b1&role.id=1eb7a5b5203945c29e6d22d653f47847 => generated 611 bytes in 272 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 03:07:51.921571 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-684f9e7f-4f95-46b9-ac60-58c4fecef08a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:07:51.923195 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-684f9e7f-4f95-46b9-ac60-58c4fecef08a None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:07:52.022049 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:07:52.032085 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51a4a103-e910-4ae8-a227-2df4ca820664 None None] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:07:52.032437 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51a4a103-e910-4ae8-a227-2df4ca820664 None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:07:52.032634 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51a4a103-e910-4ae8-a227-2df4ca820664 None None] PATH_INFO: `/` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:07:52.034169 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 211/428] 10.222.0.18 () {54 vars in 927 bytes} [Tue Sep 13 03:07:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 03:07:52.037749 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 218/429] 10.222.0.18 () {58 vars in 1282 bytes} [Tue Sep 13 03:07:51 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 448 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 13 03:07:52.039759 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e54f707a-091a-4140-9a3a-ad4ddfb6f78d None None] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:07:52.040045 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e54f707a-091a-4140-9a3a-ad4ddfb6f78d None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:07:52.040318 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e54f707a-091a-4140-9a3a-ad4ddfb6f78d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:07:52.043344 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-684f9e7f-4f95-46b9-ac60-58c4fecef08a None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:07:52.065237 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.auth.core [None req-e54f707a-091a-4140-9a3a-ad4ddfb6f78d None None] MFA Rules not processed for user `49f280ab086b4610855c4ffa378517d9`. Rule list: `[]` (Enabled: `True`). {{(pid=81399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 03:07:52.069323 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-e54f707a-091a-4140-9a3a-ad4ddfb6f78d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:07:52.165978 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 212/430] 10.222.0.18 () {58 vars in 1024 bytes} [Tue Sep 13 03:07:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 13 03:07:52.170570 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-684f9e7f-4f95-46b9-ac60-58c4fecef08a None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:07:52.206491 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5315be26-803b-4ffa-afdf-f0977e0f87ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:07:52.207148 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5315be26-803b-4ffa-afdf-f0977e0f87ac None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:07:52.207423 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5315be26-803b-4ffa-afdf-f0977e0f87ac None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:07:52.207694 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5315be26-803b-4ffa-afdf-f0977e0f87ac None admin] PATH_INFO: `/v3/projects/83aeaa07fb5d40d3b285aeb01d1252b1` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:07:52.208807 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5315be26-803b-4ffa-afdf-f0977e0f87ac None admin] RBAC: Authorizing `identity:get_project(project_id=83aeaa07fb5d40d3b285aeb01d1252b1)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:07:52.210251 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5315be26-803b-4ffa-afdf-f0977e0f87ac None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:07:52.212194 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 219/431] 10.222.0.18 () {56 vars in 1186 bytes} [Tue Sep 13 03:07:52 2022] GET /identity/v3/projects/83aeaa07fb5d40d3b285aeb01d1252b1 => generated 298 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 03:07:52.217657 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e54f707a-091a-4140-9a3a-ad4ddfb6f78d None None] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:07:52.320511 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:07:52.327204 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e54f707a-091a-4140-9a3a-ad4ddfb6f78d None None] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:07:52.365858 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2173903-ec3c-4e8f-8cf2-e90438c5deac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:07:52.366860 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2173903-ec3c-4e8f-8cf2-e90438c5deac None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:07:52.367357 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2173903-ec3c-4e8f-8cf2-e90438c5deac None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:07:52.367817 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2173903-ec3c-4e8f-8cf2-e90438c5deac None admin] PATH_INFO: `/v3/groups/7cea0c7f4a7b4b8f977cd4c1adc36486` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:07:52.369138 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2173903-ec3c-4e8f-8cf2-e90438c5deac None admin] RBAC: Authorizing `identity:get_group(group_id=7cea0c7f4a7b4b8f977cd4c1adc36486)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:07:52.370865 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2173903-ec3c-4e8f-8cf2-e90438c5deac None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:07:52.373128 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 213/432] 10.222.0.18 () {56 vars in 1180 bytes} [Tue Sep 13 03:07:52 2022] GET /identity/v3/groups/7cea0c7f4a7b4b8f977cd4c1adc36486 => generated 226 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 03:07:52.379293 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5315be26-803b-4ffa-afdf-f0977e0f87ac None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:07:52.448406 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:07:52.456178 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2173903-ec3c-4e8f-8cf2-e90438c5deac None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:07:52.511270 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5315be26-803b-4ffa-afdf-f0977e0f87ac None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:07:52.560779 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca99ebbc-9511-4080-93ea-c788c649833c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:07:52.561385 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca99ebbc-9511-4080-93ea-c788c649833c None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:07:52.561660 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca99ebbc-9511-4080-93ea-c788c649833c None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:07:52.561926 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca99ebbc-9511-4080-93ea-c788c649833c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:07:52.565305 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca99ebbc-9511-4080-93ea-c788c649833c None admin] RBAC: Adding query filter params (group.id=7cea0c7f4a7b4b8f977cd4c1adc36486, role.id=a9aa35ebda4d4aff80e62f750e9104ab, scope.project.id=83aeaa07fb5d40d3b285aeb01d1252b1) {{(pid=81400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 03:07:52.565305 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca99ebbc-9511-4080-93ea-c788c649833c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:07:52.565305 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca99ebbc-9511-4080-93ea-c788c649833c None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:07:52.572751 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 220/433] 10.222.0.18 () {56 vars in 1376 bytes} [Tue Sep 13 03:07:52 2022] GET /identity/v3/role_assignments?group.id=7cea0c7f4a7b4b8f977cd4c1adc36486&scope.project.id=83aeaa07fb5d40d3b285aeb01d1252b1&role.id=a9aa35ebda4d4aff80e62f750e9104ab => generated 611 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 03:07:52.577590 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dda36de-873f-46b4-bb7e-7763989a5205 None None] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:07:52.577863 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dda36de-873f-46b4-bb7e-7763989a5205 None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:07:52.578175 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dda36de-873f-46b4-bb7e-7763989a5205 None None] PATH_INFO: `/` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:07:52.579830 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 221/434] 10.222.0.18 () {54 vars in 927 bytes} [Tue Sep 13 03:07:52 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 03:07:52.585715 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4174d4bc-1f50-402f-b65d-9d378d1a3940 None None] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:07:52.586045 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4174d4bc-1f50-402f-b65d-9d378d1a3940 None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:07:52.586254 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4174d4bc-1f50-402f-b65d-9d378d1a3940 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:07:52.602770 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2173903-ec3c-4e8f-8cf2-e90438c5deac None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:07:52.611605 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.auth.core [None req-4174d4bc-1f50-402f-b65d-9d378d1a3940 None None] MFA Rules not processed for user `49f280ab086b4610855c4ffa378517d9`. Rule list: `[]` (Enabled: `True`). {{(pid=81400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 03:07:52.613665 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-4174d4bc-1f50-402f-b65d-9d378d1a3940 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:07:52.654528 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba9b6e92-6921-4a6d-b0bd-349d4c570cff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:07:52.655261 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba9b6e92-6921-4a6d-b0bd-349d4c570cff None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:07:52.655541 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba9b6e92-6921-4a6d-b0bd-349d4c570cff None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:07:52.655814 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba9b6e92-6921-4a6d-b0bd-349d4c570cff None admin] PATH_INFO: `/v3/roles/member` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:07:52.672271 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba9b6e92-6921-4a6d-b0bd-349d4c570cff None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 03:07:52.678745 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application [None req-ba9b6e92-6921-4a6d-b0bd-349d4c570cff None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 13 03:07:52.678745 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 03:07:52.678745 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 03:07:52.678745 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 03:07:52.678745 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 03:07:52.678745 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 03:07:52.678745 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 03:07:52.678745 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 03:07:52.678745 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 03:07:52.678745 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 03:07:52.678745 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 03:07:52.678745 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 03:07:52.678745 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 03:07:52.678745 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 03:07:52.678745 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 03:07:52.678745 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 03:07:52.679778 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 03:07:52.679778 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 03:07:52.679778 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 03:07:52.679778 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 03:07:52.679778 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 03:07:52.679778 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 03:07:52.679778 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 03:07:52.679778 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 03:07:52.679778 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 03:07:52.679778 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 03:07:52.679778 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 03:07:52.679778 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application with Lock( Sep 13 03:07:52.679778 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 03:07:52.679778 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._enter() Sep 13 03:07:52.679778 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 03:07:52.679778 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 03:07:52.680854 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 03:07:52.680854 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.creator() Sep 13 03:07:52.680854 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 03:07:52.680854 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application created_value = creator( Sep 13 03:07:52.680854 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 03:07:52.680854 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 03:07:52.680854 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 03:07:52.680854 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 03:07:52.680854 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 03:07:52.680854 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 03:07:52.680854 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 13 03:07:52.680854 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Sep 13 03:07:52.680854 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 214/435] 10.222.0.18 () {56 vars in 1099 bytes} [Tue Sep 13 03:07:52 2022] GET /identity/v3/roles/member => generated 84 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 13 03:07:52.688882 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba9b6e92-6921-4a6d-b0bd-349d4c570cff None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:07:52.712524 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 222/436] 10.222.0.18 () {58 vars in 1024 bytes} [Tue Sep 13 03:07:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 13 03:07:52.720705 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:07:52.808765 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba9b6e92-6921-4a6d-b0bd-349d4c570cff None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:07:52.848700 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fafccbfb-0bcb-4bff-afae-805ea2e8a43e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:07:52.849424 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fafccbfb-0bcb-4bff-afae-805ea2e8a43e None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:07:52.849634 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fafccbfb-0bcb-4bff-afae-805ea2e8a43e None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:07:52.849902 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fafccbfb-0bcb-4bff-afae-805ea2e8a43e None admin] PATH_INFO: `/v3/roles` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:07:52.850604 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fafccbfb-0bcb-4bff-afae-805ea2e8a43e None admin] RBAC: Adding query filter params (name=member) {{(pid=81399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 03:07:52.850987 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fafccbfb-0bcb-4bff-afae-805ea2e8a43e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:07:52.852351 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fafccbfb-0bcb-4bff-afae-805ea2e8a43e None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:07:52.862306 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 215/437] 10.222.0.18 () {56 vars in 1101 bytes} [Tue Sep 13 03:07:52 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 03:07:52.865187 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9877f3a3-0f1f-4d3f-b92b-1a110e13622c None None] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:07:52.865465 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9877f3a3-0f1f-4d3f-b92b-1a110e13622c None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:07:52.865730 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9877f3a3-0f1f-4d3f-b92b-1a110e13622c None None] PATH_INFO: `/` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:07:52.867902 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 223/438] 10.222.0.18 () {54 vars in 927 bytes} [Tue Sep 13 03:07:52 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 13 03:07:52.868233 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fafccbfb-0bcb-4bff-afae-805ea2e8a43e None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:07:52.873784 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ea08205-33a0-4022-b2b7-36b6f6e5596e None None] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:07:52.873784 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ea08205-33a0-4022-b2b7-36b6f6e5596e None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:07:52.873784 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ea08205-33a0-4022-b2b7-36b6f6e5596e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:07:52.897236 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.auth.core [None req-8ea08205-33a0-4022-b2b7-36b6f6e5596e None None] MFA Rules not processed for user `49f280ab086b4610855c4ffa378517d9`. Rule list: `[]` (Enabled: `True`). {{(pid=81400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 03:07:52.898420 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-8ea08205-33a0-4022-b2b7-36b6f6e5596e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:07:52.958194 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:07:52.978393 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:07:52.992178 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 224/439] 10.222.0.18 () {58 vars in 1024 bytes} [Tue Sep 13 03:07:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 13 03:07:52.999004 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fafccbfb-0bcb-4bff-afae-805ea2e8a43e None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:07:53.006053 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ea08205-33a0-4022-b2b7-36b6f6e5596e None None] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:07:53.035221 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aaf6f91-b9ac-42f5-a4b2-ffea0a511fd2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:07:53.035838 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aaf6f91-b9ac-42f5-a4b2-ffea0a511fd2 None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:07:53.036112 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aaf6f91-b9ac-42f5-a4b2-ffea0a511fd2 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:07:53.036378 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aaf6f91-b9ac-42f5-a4b2-ffea0a511fd2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:07:53.129806 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aaf6f91-b9ac-42f5-a4b2-ffea0a511fd2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:07:53.132080 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aaf6f91-b9ac-42f5-a4b2-ffea0a511fd2 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:07:53.148992 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ea08205-33a0-4022-b2b7-36b6f6e5596e None None] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:07:53.191370 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8432eca7-6ceb-4131-8d19-d48ad2b4e5ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:07:53.191987 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8432eca7-6ceb-4131-8d19-d48ad2b4e5ed None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:07:53.192257 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8432eca7-6ceb-4131-8d19-d48ad2b4e5ed None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:07:53.192521 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8432eca7-6ceb-4131-8d19-d48ad2b4e5ed None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:07:53.202758 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8432eca7-6ceb-4131-8d19-d48ad2b4e5ed None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 03:07:53.208114 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application [None req-8432eca7-6ceb-4131-8d19-d48ad2b4e5ed None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 13 03:07:53.208114 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 03:07:53.208114 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 03:07:53.208114 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 03:07:53.208114 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 03:07:53.208114 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 03:07:53.208114 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 03:07:53.208114 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 03:07:53.208114 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 03:07:53.208114 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 03:07:53.208114 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 03:07:53.208114 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 03:07:53.208114 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 03:07:53.208114 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 03:07:53.208114 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 03:07:53.208114 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 03:07:53.209007 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 03:07:53.209007 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 03:07:53.209007 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 03:07:53.209007 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 03:07:53.209007 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 03:07:53.209007 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 03:07:53.209007 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 03:07:53.209007 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 03:07:53.209007 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 03:07:53.209007 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 03:07:53.209007 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 03:07:53.209007 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application with Lock( Sep 13 03:07:53.209007 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 03:07:53.209007 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._enter() Sep 13 03:07:53.209007 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 03:07:53.209007 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 03:07:53.209948 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 03:07:53.209948 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.creator() Sep 13 03:07:53.209948 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 03:07:53.209948 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application created_value = creator( Sep 13 03:07:53.209948 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 03:07:53.209948 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 03:07:53.209948 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 03:07:53.209948 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 03:07:53.209948 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 03:07:53.209948 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 03:07:53.209948 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 13 03:07:53.209948 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Sep 13 03:07:53.209948 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 225/440] 10.222.0.18 () {56 vars in 1114 bytes} [Tue Sep 13 03:07:53 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 03:07:53.215348 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8432eca7-6ceb-4131-8d19-d48ad2b4e5ed None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:07:53.237554 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:07:53.253929 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 216/441] 10.222.0.18 () {58 vars in 1282 bytes} [Tue Sep 13 03:07:52 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 386 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Sep 13 03:07:53.259376 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:07:53.259516 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aaf6f91-b9ac-42f5-a4b2-ffea0a511fd2 None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:07:53.317318 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8432eca7-6ceb-4131-8d19-d48ad2b4e5ed None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:07:53.351930 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9d3c11d-e006-458e-9fb3-6ddc507bcda7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:07:53.352517 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9d3c11d-e006-458e-9fb3-6ddc507bcda7 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:07:53.352793 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9d3c11d-e006-458e-9fb3-6ddc507bcda7 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:07:53.353053 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9d3c11d-e006-458e-9fb3-6ddc507bcda7 None admin] PATH_INFO: `/v3/roles` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:07:53.353699 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9d3c11d-e006-458e-9fb3-6ddc507bcda7 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 03:07:53.353988 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9d3c11d-e006-458e-9fb3-6ddc507bcda7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:07:53.355375 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9d3c11d-e006-458e-9fb3-6ddc507bcda7 None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:07:53.362181 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aaf6f91-b9ac-42f5-a4b2-ffea0a511fd2 None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:07:53.362999 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 226/442] 10.222.0.18 () {56 vars in 1111 bytes} [Tue Sep 13 03:07:53 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 03:07:53.363754 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9d3c11d-e006-458e-9fb3-6ddc507bcda7 None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:07:53.394852 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab756c60-afab-4a70-8a8f-da035e43ce67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:07:53.395478 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab756c60-afab-4a70-8a8f-da035e43ce67 None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:07:53.395755 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab756c60-afab-4a70-8a8f-da035e43ce67 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:07:53.396021 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab756c60-afab-4a70-8a8f-da035e43ce67 None admin] PATH_INFO: `/v3/projects/54999ea237f64c5f972ef79e765facd0` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:07:53.397127 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab756c60-afab-4a70-8a8f-da035e43ce67 None admin] RBAC: Authorizing `identity:get_project(project_id=54999ea237f64c5f972ef79e765facd0)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:07:53.398546 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab756c60-afab-4a70-8a8f-da035e43ce67 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:07:53.400322 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 217/443] 10.222.0.18 () {56 vars in 1186 bytes} [Tue Sep 13 03:07:53 2022] GET /identity/v3/projects/54999ea237f64c5f972ef79e765facd0 => generated 302 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 03:07:53.400936 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab756c60-afab-4a70-8a8f-da035e43ce67 None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:07:53.454876 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:07:53.465231 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9d3c11d-e006-458e-9fb3-6ddc507bcda7 None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:07:53.466749 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:07:53.498511 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82adbbb7-7399-429c-b007-3aa8f5571d4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:07:53.498886 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab756c60-afab-4a70-8a8f-da035e43ce67 None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:07:53.499092 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82adbbb7-7399-429c-b007-3aa8f5571d4a None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:07:53.499368 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82adbbb7-7399-429c-b007-3aa8f5571d4a None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:07:53.499637 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82adbbb7-7399-429c-b007-3aa8f5571d4a None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:07:53.509937 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82adbbb7-7399-429c-b007-3aa8f5571d4a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 03:07:53.515205 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application [None req-82adbbb7-7399-429c-b007-3aa8f5571d4a None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 13 03:07:53.515205 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 03:07:53.515205 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 03:07:53.515205 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 03:07:53.515205 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 03:07:53.515205 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 03:07:53.515205 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 03:07:53.515205 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 03:07:53.515205 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 03:07:53.515205 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 03:07:53.515205 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 03:07:53.515205 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 03:07:53.515205 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 03:07:53.515205 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 03:07:53.515205 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 03:07:53.515205 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 03:07:53.517212 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 03:07:53.517212 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 03:07:53.517212 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 03:07:53.517212 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 03:07:53.517212 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 03:07:53.517212 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 03:07:53.517212 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 03:07:53.517212 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 03:07:53.517212 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 03:07:53.517212 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 03:07:53.517212 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 03:07:53.517212 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application with Lock( Sep 13 03:07:53.517212 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 03:07:53.517212 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._enter() Sep 13 03:07:53.517212 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 03:07:53.517212 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 03:07:53.518025 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 03:07:53.518025 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.creator() Sep 13 03:07:53.518025 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 03:07:53.518025 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application created_value = creator( Sep 13 03:07:53.518025 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 03:07:53.518025 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 03:07:53.518025 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 03:07:53.518025 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 03:07:53.518025 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 03:07:53.518025 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 03:07:53.518025 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 13 03:07:53.518025 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Sep 13 03:07:53.518025 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 227/444] 10.222.0.18 () {56 vars in 1096 bytes} [Tue Sep 13 03:07:53 2022] GET /identity/v3/roles/admin => generated 83 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 03:07:53.518025 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82adbbb7-7399-429c-b007-3aa8f5571d4a None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:07:53.535840 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6eade89-a97a-44c2-b0a2-60e199588e6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:07:53.536454 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6eade89-a97a-44c2-b0a2-60e199588e6c None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:07:53.536732 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6eade89-a97a-44c2-b0a2-60e199588e6c None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:07:53.536999 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6eade89-a97a-44c2-b0a2-60e199588e6c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:07:53.601958 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6eade89-a97a-44c2-b0a2-60e199588e6c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:07:53.603436 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6eade89-a97a-44c2-b0a2-60e199588e6c None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:07:53.614586 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82adbbb7-7399-429c-b007-3aa8f5571d4a None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:07:53.647666 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b98532c-50ad-4e65-a9b7-be9c1f2ace52 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:07:53.648269 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b98532c-50ad-4e65-a9b7-be9c1f2ace52 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:07:53.648538 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b98532c-50ad-4e65-a9b7-be9c1f2ace52 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:07:53.648804 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b98532c-50ad-4e65-a9b7-be9c1f2ace52 None admin] PATH_INFO: `/v3/groups/7cea0c7f4a7b4b8f977cd4c1adc36486` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:07:53.649876 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b98532c-50ad-4e65-a9b7-be9c1f2ace52 None admin] RBAC: Authorizing `identity:get_group(group_id=7cea0c7f4a7b4b8f977cd4c1adc36486)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:07:53.651327 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b98532c-50ad-4e65-a9b7-be9c1f2ace52 None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:07:53.652865 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 228/445] 10.222.0.18 () {56 vars in 1180 bytes} [Tue Sep 13 03:07:53 2022] GET /identity/v3/groups/7cea0c7f4a7b4b8f977cd4c1adc36486 => generated 226 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 03:07:53.653595 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b98532c-50ad-4e65-a9b7-be9c1f2ace52 None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:07:53.654882 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:07:53.682629 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:07:53.702217 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 218/446] 10.222.0.18 () {58 vars in 1282 bytes} [Tue Sep 13 03:07:53 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 301 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 13 03:07:53.703620 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6eade89-a97a-44c2-b0a2-60e199588e6c None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:07:53.756496 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b98532c-50ad-4e65-a9b7-be9c1f2ace52 None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:07:53.788404 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82c13818-ba02-4659-ba31-45c7d65095ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:07:53.788996 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c13818-ba02-4659-ba31-45c7d65095ab None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:07:53.789266 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c13818-ba02-4659-ba31-45c7d65095ab None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:07:53.789532 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c13818-ba02-4659-ba31-45c7d65095ab None admin] PATH_INFO: `/v3/roles` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:07:53.790180 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82c13818-ba02-4659-ba31-45c7d65095ab None admin] RBAC: Adding query filter params (name=admin) {{(pid=81400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 03:07:53.790472 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82c13818-ba02-4659-ba31-45c7d65095ab None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:07:53.791857 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82c13818-ba02-4659-ba31-45c7d65095ab None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:07:53.798637 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6eade89-a97a-44c2-b0a2-60e199588e6c None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:07:53.800269 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 229/447] 10.222.0.18 () {56 vars in 1099 bytes} [Tue Sep 13 03:07:53 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 03:07:53.801020 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82c13818-ba02-4659-ba31-45c7d65095ab None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:07:53.836742 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-900c05f3-012a-46aa-aa9a-2a3c20631fa0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:07:53.837338 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-900c05f3-012a-46aa-aa9a-2a3c20631fa0 None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:07:53.837613 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-900c05f3-012a-46aa-aa9a-2a3c20631fa0 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:07:53.837877 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-900c05f3-012a-46aa-aa9a-2a3c20631fa0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:07:53.838580 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-900c05f3-012a-46aa-aa9a-2a3c20631fa0 None admin] RBAC: Adding query filter params (group.id=7cea0c7f4a7b4b8f977cd4c1adc36486, role.id=1eb7a5b5203945c29e6d22d653f47847, scope.project.id=54999ea237f64c5f972ef79e765facd0) {{(pid=81399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 03:07:53.838917 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-900c05f3-012a-46aa-aa9a-2a3c20631fa0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:07:53.840325 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-900c05f3-012a-46aa-aa9a-2a3c20631fa0 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:07:53.846088 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 219/448] 10.222.0.18 () {56 vars in 1376 bytes} [Tue Sep 13 03:07:53 2022] GET /identity/v3/role_assignments?group.id=7cea0c7f4a7b4b8f977cd4c1adc36486&scope.project.id=54999ea237f64c5f972ef79e765facd0&role.id=1eb7a5b5203945c29e6d22d653f47847 => generated 611 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 03:07:53.846807 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-900c05f3-012a-46aa-aa9a-2a3c20631fa0 None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:07:53.880050 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:07:53.896284 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:07:53.909047 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82c13818-ba02-4659-ba31-45c7d65095ab None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:07:53.946850 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d174a34-66d8-40fb-93b4-8625159d8cd3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:07:53.947770 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d174a34-66d8-40fb-93b4-8625159d8cd3 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:07:53.948043 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d174a34-66d8-40fb-93b4-8625159d8cd3 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:07:53.948308 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d174a34-66d8-40fb-93b4-8625159d8cd3 None admin] PATH_INFO: `/v3/projects/54999ea237f64c5f972ef79e765facd0` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:07:53.949462 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d174a34-66d8-40fb-93b4-8625159d8cd3 None admin] RBAC: Authorizing `identity:get_project(project_id=54999ea237f64c5f972ef79e765facd0)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:07:53.950950 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d174a34-66d8-40fb-93b4-8625159d8cd3 None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:07:53.952625 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 230/449] 10.222.0.18 () {56 vars in 1186 bytes} [Tue Sep 13 03:07:53 2022] GET /identity/v3/projects/54999ea237f64c5f972ef79e765facd0 => generated 302 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 03:07:53.958470 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d174a34-66d8-40fb-93b4-8625159d8cd3 None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:07:53.974481 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-900c05f3-012a-46aa-aa9a-2a3c20631fa0 None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:07:54.013447 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ef2a986-308d-4953-b9fa-117c478eacd9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:07:54.014019 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ef2a986-308d-4953-b9fa-117c478eacd9 None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:07:54.014261 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ef2a986-308d-4953-b9fa-117c478eacd9 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:07:54.014531 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ef2a986-308d-4953-b9fa-117c478eacd9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:07:54.070772 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d174a34-66d8-40fb-93b4-8625159d8cd3 None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:07:54.105191 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ef2a986-308d-4953-b9fa-117c478eacd9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:07:54.106842 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ef2a986-308d-4953-b9fa-117c478eacd9 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:07:54.110010 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a589e32-69e2-48a1-b107-307cec38b91d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:07:54.110658 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a589e32-69e2-48a1-b107-307cec38b91d None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:07:54.110940 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a589e32-69e2-48a1-b107-307cec38b91d None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:07:54.111206 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a589e32-69e2-48a1-b107-307cec38b91d None admin] PATH_INFO: `/v3/groups/7cea0c7f4a7b4b8f977cd4c1adc36486` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:07:54.112301 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a589e32-69e2-48a1-b107-307cec38b91d None admin] RBAC: Authorizing `identity:get_group(group_id=7cea0c7f4a7b4b8f977cd4c1adc36486)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:07:54.113735 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a589e32-69e2-48a1-b107-307cec38b91d None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:07:54.115338 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 231/450] 10.222.0.18 () {56 vars in 1180 bytes} [Tue Sep 13 03:07:53 2022] GET /identity/v3/groups/7cea0c7f4a7b4b8f977cd4c1adc36486 => generated 226 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 03:07:54.121135 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a589e32-69e2-48a1-b107-307cec38b91d None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:07:54.130788 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:07:54.159962 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:07:54.230629 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 220/451] 10.222.0.18 () {58 vars in 1282 bytes} [Tue Sep 13 03:07:53 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 384 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 13 03:07:54.236915 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ef2a986-308d-4953-b9fa-117c478eacd9 None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:07:54.244447 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a589e32-69e2-48a1-b107-307cec38b91d None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:07:54.277154 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b886b902-6205-40c8-afa5-112b478b14d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:07:54.277680 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b886b902-6205-40c8-afa5-112b478b14d5 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:07:54.277953 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b886b902-6205-40c8-afa5-112b478b14d5 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:07:54.278226 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b886b902-6205-40c8-afa5-112b478b14d5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:07:54.278962 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b886b902-6205-40c8-afa5-112b478b14d5 None admin] RBAC: Adding query filter params (group.id=7cea0c7f4a7b4b8f977cd4c1adc36486, role.id=a9aa35ebda4d4aff80e62f750e9104ab, scope.project.id=54999ea237f64c5f972ef79e765facd0) {{(pid=81400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 03:07:54.279265 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b886b902-6205-40c8-afa5-112b478b14d5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:07:54.280687 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b886b902-6205-40c8-afa5-112b478b14d5 None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:07:54.286804 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 232/452] 10.222.0.18 () {56 vars in 1376 bytes} [Tue Sep 13 03:07:54 2022] GET /identity/v3/role_assignments?group.id=7cea0c7f4a7b4b8f977cd4c1adc36486&scope.project.id=54999ea237f64c5f972ef79e765facd0&role.id=a9aa35ebda4d4aff80e62f750e9104ab => generated 611 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 03:07:54.336086 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ef2a986-308d-4953-b9fa-117c478eacd9 None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:07:54.355395 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:07:54.373425 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95ae58c9-db4e-4467-9b93-33b5a7953bda None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:07:54.374066 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ae58c9-db4e-4467-9b93-33b5a7953bda None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:07:54.374338 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ae58c9-db4e-4467-9b93-33b5a7953bda None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:07:54.374604 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ae58c9-db4e-4467-9b93-33b5a7953bda None admin] PATH_INFO: `/v3/projects/76c5181fc7be45279ac00017bb4640b3` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:07:54.375853 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95ae58c9-db4e-4467-9b93-33b5a7953bda None admin] RBAC: Authorizing `identity:get_project(project_id=76c5181fc7be45279ac00017bb4640b3)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:07:54.377323 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95ae58c9-db4e-4467-9b93-33b5a7953bda None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:07:54.379033 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 221/453] 10.222.0.18 () {56 vars in 1186 bytes} [Tue Sep 13 03:07:54 2022] GET /identity/v3/projects/76c5181fc7be45279ac00017bb4640b3 => generated 344 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 03:07:54.384638 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b886b902-6205-40c8-afa5-112b478b14d5 None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:07:54.478409 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:07:54.488405 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b886b902-6205-40c8-afa5-112b478b14d5 None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:07:54.522018 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ce6fa7b-f217-471c-a68f-c4b33e3455a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:07:54.522708 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce6fa7b-f217-471c-a68f-c4b33e3455a6 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:07:54.522991 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce6fa7b-f217-471c-a68f-c4b33e3455a6 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:07:54.523261 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce6fa7b-f217-471c-a68f-c4b33e3455a6 None admin] PATH_INFO: `/v3/groups/228dcaf0a4474b2b804468d12ab54ded` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:07:54.524375 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ce6fa7b-f217-471c-a68f-c4b33e3455a6 None admin] RBAC: Authorizing `identity:get_group(group_id=228dcaf0a4474b2b804468d12ab54ded)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:07:54.525799 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ce6fa7b-f217-471c-a68f-c4b33e3455a6 None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:07:54.527437 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 233/454] 10.222.0.18 () {56 vars in 1180 bytes} [Tue Sep 13 03:07:54 2022] GET /identity/v3/groups/228dcaf0a4474b2b804468d12ab54ded => generated 229 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 03:07:54.534174 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95ae58c9-db4e-4467-9b93-33b5a7953bda None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:07:54.633934 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95ae58c9-db4e-4467-9b93-33b5a7953bda None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:07:54.667078 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9d57067-6622-45a0-b044-4a49b507bb15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:07:54.667677 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9d57067-6622-45a0-b044-4a49b507bb15 None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:07:54.667949 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9d57067-6622-45a0-b044-4a49b507bb15 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:07:54.668214 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9d57067-6622-45a0-b044-4a49b507bb15 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:07:54.668923 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9d57067-6622-45a0-b044-4a49b507bb15 None admin] RBAC: Adding query filter params (group.id=228dcaf0a4474b2b804468d12ab54ded, role.id=425d4b481c7846e0bc07e7e805d5b15f, scope.project.id=76c5181fc7be45279ac00017bb4640b3) {{(pid=81399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 03:07:54.669222 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9d57067-6622-45a0-b044-4a49b507bb15 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:07:54.670671 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9d57067-6622-45a0-b044-4a49b507bb15 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:07:54.676151 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 222/455] 10.222.0.18 () {56 vars in 1376 bytes} [Tue Sep 13 03:07:54 2022] GET /identity/v3/role_assignments?group.id=228dcaf0a4474b2b804468d12ab54ded&scope.project.id=76c5181fc7be45279ac00017bb4640b3&role.id=425d4b481c7846e0bc07e7e805d5b15f => generated 611 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 03:07:57.332044 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3b3f2d8-dad5-4833-a7cd-080c70239db3 None None] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:07:57.332538 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3b3f2d8-dad5-4833-a7cd-080c70239db3 None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:07:57.332538 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3b3f2d8-dad5-4833-a7cd-080c70239db3 None None] PATH_INFO: `/` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:07:57.334252 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 234/456] 10.222.0.18 () {54 vars in 927 bytes} [Tue Sep 13 03:07:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 03:07:57.339232 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f46db28-424c-4d1f-91e9-604f67570f7e None None] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:07:57.340042 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f46db28-424c-4d1f-91e9-604f67570f7e None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:07:57.340508 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f46db28-424c-4d1f-91e9-604f67570f7e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:07:57.364653 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.auth.core [None req-8f46db28-424c-4d1f-91e9-604f67570f7e None None] MFA Rules not processed for user `49f280ab086b4610855c4ffa378517d9`. Rule list: `[]` (Enabled: `True`). {{(pid=81399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 03:07:57.366135 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-8f46db28-424c-4d1f-91e9-604f67570f7e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:07:57.414349 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:07:57.445038 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 223/457] 10.222.0.18 () {58 vars in 1024 bytes} [Tue Sep 13 03:07:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 13 03:07:57.595859 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03bcf583-7cdd-455f-9c92-e1687e930515 None None] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:07:57.596452 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03bcf583-7cdd-455f-9c92-e1687e930515 None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:07:57.596917 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03bcf583-7cdd-455f-9c92-e1687e930515 None None] PATH_INFO: `/` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:07:57.598711 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 235/458] 10.222.0.18 () {54 vars in 927 bytes} [Tue Sep 13 03:07:57 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 03:07:57.603762 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ecd3416-703d-4012-a7b6-696bf2928985 None None] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:07:57.604279 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ecd3416-703d-4012-a7b6-696bf2928985 None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:07:57.604745 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ecd3416-703d-4012-a7b6-696bf2928985 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:07:57.626530 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.auth.core [None req-8ecd3416-703d-4012-a7b6-696bf2928985 None None] MFA Rules not processed for user `49f280ab086b4610855c4ffa378517d9`. Rule list: `[]` (Enabled: `True`). {{(pid=81399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 03:07:57.628020 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-8ecd3416-703d-4012-a7b6-696bf2928985 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:07:57.714489 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 224/459] 10.222.0.18 () {58 vars in 1024 bytes} [Tue Sep 13 03:07:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 13 03:07:57.722147 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54bb3595-8823-419e-802c-0917bdcc917f None None] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:07:57.722687 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54bb3595-8823-419e-802c-0917bdcc917f None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:07:57.723173 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54bb3595-8823-419e-802c-0917bdcc917f None None] PATH_INFO: `/` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:07:57.724684 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 236/460] 10.222.0.18 () {54 vars in 927 bytes} [Tue Sep 13 03:07:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 03:07:57.731623 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e57cc30-9a20-45ba-afaf-df8de3ad85b5 None None] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:07:57.732147 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e57cc30-9a20-45ba-afaf-df8de3ad85b5 None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:07:57.732617 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e57cc30-9a20-45ba-afaf-df8de3ad85b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:07:57.760378 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.auth.core [None req-6e57cc30-9a20-45ba-afaf-df8de3ad85b5 None None] MFA Rules not processed for user `49f280ab086b4610855c4ffa378517d9`. Rule list: `[]` (Enabled: `True`). {{(pid=81399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 03:07:57.761851 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-6e57cc30-9a20-45ba-afaf-df8de3ad85b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:07:57.790511 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54bb3595-8823-419e-802c-0917bdcc917f None None] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:07:57.868496 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 225/461] 10.222.0.18 () {58 vars in 1024 bytes} [Tue Sep 13 03:07:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 13 03:07:57.963360 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54bb3595-8823-419e-802c-0917bdcc917f None None] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:07:58.005245 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad144468-1f9c-4aef-bc20-44c2e78b6ee8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:07:58.005881 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad144468-1f9c-4aef-bc20-44c2e78b6ee8 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:07:58.006159 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad144468-1f9c-4aef-bc20-44c2e78b6ee8 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:07:58.006431 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad144468-1f9c-4aef-bc20-44c2e78b6ee8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:07:58.016596 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e57cc30-9a20-45ba-afaf-df8de3ad85b5 None None] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:07:58.017956 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad144468-1f9c-4aef-bc20-44c2e78b6ee8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:07:58.022640 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad144468-1f9c-4aef-bc20-44c2e78b6ee8 None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:07:58.036297 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application [None req-ad144468-1f9c-4aef-bc20-44c2e78b6ee8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 03:07:58.036297 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 03:07:58.036297 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 03:07:58.036297 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 03:07:58.036297 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 03:07:58.036297 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 03:07:58.036297 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 03:07:58.036297 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 03:07:58.036297 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 03:07:58.036297 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 03:07:58.036297 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 03:07:58.036297 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 03:07:58.036297 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 13 03:07:58.036297 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 13 03:07:58.036297 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 13 03:07:58.036297 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 13 03:07:58.037255 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 03:07:58.037255 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 03:07:58.037255 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 03:07:58.037255 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 03:07:58.037255 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 03:07:58.037255 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 03:07:58.037255 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 03:07:58.037255 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application with Lock( Sep 13 03:07:58.037255 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 03:07:58.037255 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._enter() Sep 13 03:07:58.037255 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 03:07:58.037255 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 03:07:58.037255 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 03:07:58.037255 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.creator() Sep 13 03:07:58.037255 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 03:07:58.037255 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application created_value = creator( Sep 13 03:07:58.037255 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 13 03:07:58.038141 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 13 03:07:58.038141 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 03:07:58.038141 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Sep 13 03:07:58.038141 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 237/462] 10.222.0.18 () {56 vars in 1108 bytes} [Tue Sep 13 03:07:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 247 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 03:07:58.043483 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad144468-1f9c-4aef-bc20-44c2e78b6ee8 None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:07:58.060387 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:07:58.109183 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:07:58.161727 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad144468-1f9c-4aef-bc20-44c2e78b6ee8 None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:07:58.173291 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e57cc30-9a20-45ba-afaf-df8de3ad85b5 None None] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:07:58.196078 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6953216a-7bb3-4b93-be64-ff3e2473d9b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:07:58.196669 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6953216a-7bb3-4b93-be64-ff3e2473d9b4 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:07:58.196941 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6953216a-7bb3-4b93-be64-ff3e2473d9b4 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:07:58.197205 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6953216a-7bb3-4b93-be64-ff3e2473d9b4 None admin] PATH_INFO: `/v3/domains` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:07:58.197830 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6953216a-7bb3-4b93-be64-ff3e2473d9b4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 03:07:58.198128 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6953216a-7bb3-4b93-be64-ff3e2473d9b4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:07:58.199507 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6953216a-7bb3-4b93-be64-ff3e2473d9b4 None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:07:58.210765 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e9799f8-b4b9-4795-81b8-e30678a4d444 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:07:58.211569 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e9799f8-b4b9-4795-81b8-e30678a4d444 None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:07:58.216098 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 238/463] 10.222.0.18 () {56 vars in 1109 bytes} [Tue Sep 13 03:07:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 03:07:58.216098 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2764a419-f738-4fed-a6f8-73c02e81aecd None None] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:07:58.216098 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2764a419-f738-4fed-a6f8-73c02e81aecd None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:07:58.216098 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2764a419-f738-4fed-a6f8-73c02e81aecd None None] PATH_INFO: `/` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:07:58.216098 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 239/464] 10.222.0.18 () {54 vars in 927 bytes} [Tue Sep 13 03:07:58 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 03:07:58.216481 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e9799f8-b4b9-4795-81b8-e30678a4d444 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:07:58.216481 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e9799f8-b4b9-4795-81b8-e30678a4d444 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:07:58.219504 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2764a419-f738-4fed-a6f8-73c02e81aecd None None] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:07:58.227291 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e9799f8-b4b9-4795-81b8-e30678a4d444 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:07:58.229173 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e9799f8-b4b9-4795-81b8-e30678a4d444 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:07:58.245169 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application [None req-4e9799f8-b4b9-4795-81b8-e30678a4d444 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 03:07:58.245169 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 03:07:58.245169 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 03:07:58.245169 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 03:07:58.245169 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 03:07:58.245169 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 03:07:58.245169 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 03:07:58.245169 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 03:07:58.245169 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 03:07:58.245169 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 03:07:58.245169 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 03:07:58.245169 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 03:07:58.245169 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 13 03:07:58.245169 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 13 03:07:58.245169 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 13 03:07:58.245169 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 13 03:07:58.246072 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 03:07:58.246072 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 03:07:58.246072 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 03:07:58.246072 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 03:07:58.246072 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 03:07:58.246072 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 03:07:58.246072 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 03:07:58.246072 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application with Lock( Sep 13 03:07:58.246072 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 03:07:58.246072 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._enter() Sep 13 03:07:58.246072 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 03:07:58.246072 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 03:07:58.246072 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 03:07:58.246072 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.creator() Sep 13 03:07:58.246072 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 03:07:58.246072 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application created_value = creator( Sep 13 03:07:58.246072 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 13 03:07:58.246974 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 13 03:07:58.246974 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 03:07:58.246974 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Sep 13 03:07:58.246974 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 226/465] 10.222.0.18 () {56 vars in 1108 bytes} [Tue Sep 13 03:07:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 231 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 03:07:58.247191 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e9799f8-b4b9-4795-81b8-e30678a4d444 None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:07:58.299603 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:07:58.332977 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2764a419-f738-4fed-a6f8-73c02e81aecd None None] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:07:58.348597 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:07:58.359590 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e9799f8-b4b9-4795-81b8-e30678a4d444 None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:07:58.369104 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-528ee9b9-d406-45e2-a628-b7ad3386d493 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:07:58.369726 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-528ee9b9-d406-45e2-a628-b7ad3386d493 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:07:58.370024 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-528ee9b9-d406-45e2-a628-b7ad3386d493 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:07:58.370293 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-528ee9b9-d406-45e2-a628-b7ad3386d493 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:07:58.380786 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-528ee9b9-d406-45e2-a628-b7ad3386d493 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:07:58.382220 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-528ee9b9-d406-45e2-a628-b7ad3386d493 None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:07:58.395352 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d9aabec-f505-462b-8166-10f803bfa9f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:07:58.395959 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d9aabec-f505-462b-8166-10f803bfa9f7 None admin] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:07:58.396233 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d9aabec-f505-462b-8166-10f803bfa9f7 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:07:58.396501 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d9aabec-f505-462b-8166-10f803bfa9f7 None admin] PATH_INFO: `/v3/users` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:07:58.397130 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d9aabec-f505-462b-8166-10f803bfa9f7 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:07:58.397475 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application [None req-528ee9b9-d406-45e2-a628-b7ad3386d493 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 03:07:58.397475 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 03:07:58.397475 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 03:07:58.397475 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 03:07:58.397475 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 03:07:58.397475 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 03:07:58.397475 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 03:07:58.397475 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 03:07:58.397475 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 03:07:58.397475 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 03:07:58.397475 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 03:07:58.397475 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 03:07:58.397475 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 13 03:07:58.397475 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 13 03:07:58.397475 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 13 03:07:58.397475 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 13 03:07:58.398308 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 03:07:58.398308 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 03:07:58.398308 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 03:07:58.398308 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 03:07:58.398308 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 03:07:58.398308 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 03:07:58.398308 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 03:07:58.398308 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application with Lock( Sep 13 03:07:58.398308 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 03:07:58.398308 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._enter() Sep 13 03:07:58.398308 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 03:07:58.398308 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 03:07:58.398308 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 03:07:58.398308 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.creator() Sep 13 03:07:58.398308 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 03:07:58.398308 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application created_value = creator( Sep 13 03:07:58.398308 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 13 03:07:58.399753 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 13 03:07:58.399753 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 03:07:58.399753 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Sep 13 03:07:58.399918 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d9aabec-f505-462b-8166-10f803bfa9f7 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:07:58.399986 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 240/466] 10.222.0.18 () {56 vars in 1108 bytes} [Tue Sep 13 03:07:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 03:07:58.400153 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e7b31b-de30-4185-9013-1c567646cfe4 None None] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:07:58.400426 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e7b31b-de30-4185-9013-1c567646cfe4 None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:07:58.400714 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e7b31b-de30-4185-9013-1c567646cfe4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:07:58.420199 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.auth.core [None req-82e7b31b-de30-4185-9013-1c567646cfe4 None None] MFA Rules not processed for user `49f280ab086b4610855c4ffa378517d9`. Rule list: `[]` (Enabled: `True`). {{(pid=81400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 03:07:58.422285 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-82e7b31b-de30-4185-9013-1c567646cfe4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:07:58.424295 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 227/467] 10.222.0.18 () {60 vars in 1132 bytes} [Tue Sep 13 03:07:58 2022] POST /identity/v3/users => generated 246 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 03:07:58.425153 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d9aabec-f505-462b-8166-10f803bfa9f7 None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:07:58.518642 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 241/468] 10.222.0.18 () {58 vars in 1024 bytes} [Tue Sep 13 03:07:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 13 03:07:58.518642 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82e7b31b-de30-4185-9013-1c567646cfe4 None None] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:07:58.585634 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:07:58.620762 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d9aabec-f505-462b-8166-10f803bfa9f7 None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:07:58.638321 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82e7b31b-de30-4185-9013-1c567646cfe4 None None] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:07:58.680217 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ac8bd62-9c63-496f-975a-9fcd34f6db98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:07:58.681042 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac8bd62-9c63-496f-975a-9fcd34f6db98 None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:07:58.681318 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac8bd62-9c63-496f-975a-9fcd34f6db98 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:07:58.681585 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac8bd62-9c63-496f-975a-9fcd34f6db98 None admin] PATH_INFO: `/v3/domains` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:07:58.682171 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02e917a1-2a6a-483e-8832-7d717d1b1dfe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:07:58.682298 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ac8bd62-9c63-496f-975a-9fcd34f6db98 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 03:07:58.682514 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ac8bd62-9c63-496f-975a-9fcd34f6db98 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:07:58.682808 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02e917a1-2a6a-483e-8832-7d717d1b1dfe None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:07:58.683089 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02e917a1-2a6a-483e-8832-7d717d1b1dfe None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:07:58.683364 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02e917a1-2a6a-483e-8832-7d717d1b1dfe None admin] PATH_INFO: `/v3/domains` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:07:58.683928 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ac8bd62-9c63-496f-975a-9fcd34f6db98 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:07:58.684027 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02e917a1-2a6a-483e-8832-7d717d1b1dfe None admin] RBAC: Adding query filter params (name=Default) {{(pid=81400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 03:07:58.684300 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02e917a1-2a6a-483e-8832-7d717d1b1dfe None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:07:58.685661 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02e917a1-2a6a-483e-8832-7d717d1b1dfe None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:07:58.699659 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 228/469] 10.222.0.18 () {56 vars in 1109 bytes} [Tue Sep 13 03:07:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 275 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 03:07:58.700697 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 242/470] 10.222.0.18 () {56 vars in 1109 bytes} [Tue Sep 13 03:07:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 03:07:58.705070 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ac8bd62-9c63-496f-975a-9fcd34f6db98 None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:07:58.706591 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02e917a1-2a6a-483e-8832-7d717d1b1dfe None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:07:58.719683 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:07:58.844508 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ac8bd62-9c63-496f-975a-9fcd34f6db98 None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:07:58.852155 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02e917a1-2a6a-483e-8832-7d717d1b1dfe None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:07:58.866887 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:07:58.888879 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e62c4859-a472-4a23-b259-4c8c2c7fd969 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:07:58.889629 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e62c4859-a472-4a23-b259-4c8c2c7fd969 None admin] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:07:58.889754 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7485267-c029-481a-8fa2-12f05809792a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:07:58.889953 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e62c4859-a472-4a23-b259-4c8c2c7fd969 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:07:58.890168 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e62c4859-a472-4a23-b259-4c8c2c7fd969 None admin] PATH_INFO: `/v3/users` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:07:58.890411 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7485267-c029-481a-8fa2-12f05809792a None admin] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:07:58.890729 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7485267-c029-481a-8fa2-12f05809792a None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:07:58.890890 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e62c4859-a472-4a23-b259-4c8c2c7fd969 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:07:58.891035 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7485267-c029-481a-8fa2-12f05809792a None admin] PATH_INFO: `/v3/users` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:07:58.891660 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7485267-c029-481a-8fa2-12f05809792a None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:07:58.892322 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e62c4859-a472-4a23-b259-4c8c2c7fd969 None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:07:58.893091 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7485267-c029-481a-8fa2-12f05809792a None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:07:58.917686 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 243/471] 10.222.0.18 () {60 vars in 1132 bytes} [Tue Sep 13 03:07:58 2022] POST /identity/v3/users => generated 248 bytes in 212 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 13 03:07:58.918551 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e62c4859-a472-4a23-b259-4c8c2c7fd969 None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:07:58.920110 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 229/472] 10.222.0.18 () {60 vars in 1132 bytes} [Tue Sep 13 03:07:58 2022] POST /identity/v3/users => generated 248 bytes in 215 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 13 03:07:58.938491 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fefb2984-f5ad-4f7c-8729-05104cf6e302 None None] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:07:58.938806 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fefb2984-f5ad-4f7c-8729-05104cf6e302 None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:07:58.938959 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fefb2984-f5ad-4f7c-8729-05104cf6e302 None None] PATH_INFO: `/` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:07:58.946280 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 230/473] 10.222.0.18 () {54 vars in 927 bytes} [Tue Sep 13 03:07:58 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 03:07:58.948619 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a7cbd5e-8ad6-4f8a-b513-854dbbb6c408 None None] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:07:58.948930 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a7cbd5e-8ad6-4f8a-b513-854dbbb6c408 None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:07:58.949215 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a7cbd5e-8ad6-4f8a-b513-854dbbb6c408 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:07:58.975495 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.auth.core [None req-1a7cbd5e-8ad6-4f8a-b513-854dbbb6c408 None None] MFA Rules not processed for user `49f280ab086b4610855c4ffa378517d9`. Rule list: `[]` (Enabled: `True`). {{(pid=81399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 03:07:58.976743 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-1a7cbd5e-8ad6-4f8a-b513-854dbbb6c408 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:07:59.097028 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:07:59.106340 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 231/474] 10.222.0.18 () {58 vars in 1024 bytes} [Tue Sep 13 03:07:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 13 03:07:59.130435 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e62c4859-a472-4a23-b259-4c8c2c7fd969 None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:07:59.170264 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87c35b49-24a3-4084-9d7a-7db00c853a38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:07:59.171185 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87c35b49-24a3-4084-9d7a-7db00c853a38 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:07:59.171694 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87c35b49-24a3-4084-9d7a-7db00c853a38 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:07:59.172098 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87c35b49-24a3-4084-9d7a-7db00c853a38 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:07:59.184850 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87c35b49-24a3-4084-9d7a-7db00c853a38 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:07:59.186539 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87c35b49-24a3-4084-9d7a-7db00c853a38 None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:07:59.203409 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application [None req-87c35b49-24a3-4084-9d7a-7db00c853a38 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 03:07:59.203409 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 03:07:59.203409 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 03:07:59.203409 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 03:07:59.203409 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 03:07:59.203409 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 03:07:59.203409 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 03:07:59.203409 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 03:07:59.203409 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 03:07:59.203409 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 03:07:59.203409 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 03:07:59.203409 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 03:07:59.203409 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 13 03:07:59.203409 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 13 03:07:59.203409 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 13 03:07:59.203409 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 13 03:07:59.204442 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 03:07:59.204442 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 03:07:59.204442 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 03:07:59.204442 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 03:07:59.204442 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 03:07:59.204442 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 03:07:59.204442 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 03:07:59.204442 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application with Lock( Sep 13 03:07:59.204442 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 03:07:59.204442 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._enter() Sep 13 03:07:59.204442 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 03:07:59.204442 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 03:07:59.204442 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 03:07:59.204442 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.creator() Sep 13 03:07:59.204442 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 03:07:59.204442 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application created_value = creator( Sep 13 03:07:59.204442 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 13 03:07:59.205495 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 13 03:07:59.205495 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 03:07:59.205495 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Sep 13 03:07:59.206902 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 244/475] 10.222.0.18 () {56 vars in 1108 bytes} [Tue Sep 13 03:07:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 289 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 03:07:59.212823 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a7cbd5e-8ad6-4f8a-b513-854dbbb6c408 None None] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:07:59.324708 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a7cbd5e-8ad6-4f8a-b513-854dbbb6c408 None None] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:07:59.363801 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d5f3950-1316-44b3-baea-bec69a2f7f95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:07:59.365126 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d5f3950-1316-44b3-baea-bec69a2f7f95 None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:07:59.365572 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d5f3950-1316-44b3-baea-bec69a2f7f95 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:07:59.366023 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d5f3950-1316-44b3-baea-bec69a2f7f95 None admin] PATH_INFO: `/v3/domains` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:07:59.366860 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d5f3950-1316-44b3-baea-bec69a2f7f95 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 03:07:59.367352 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d5f3950-1316-44b3-baea-bec69a2f7f95 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:07:59.368885 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d5f3950-1316-44b3-baea-bec69a2f7f95 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:07:59.381512 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 232/476] 10.222.0.18 () {56 vars in 1109 bytes} [Tue Sep 13 03:07:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 03:07:59.386946 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87c35b49-24a3-4084-9d7a-7db00c853a38 None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:07:59.413505 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:07:59.419990 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d5f3950-1316-44b3-baea-bec69a2f7f95 None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:07:59.523072 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:07:59.536519 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87c35b49-24a3-4084-9d7a-7db00c853a38 None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:07:59.572252 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c23ab797-4036-4e70-ae59-61cd682f65c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:07:59.572854 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c23ab797-4036-4e70-ae59-61cd682f65c2 None admin] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:07:59.573124 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c23ab797-4036-4e70-ae59-61cd682f65c2 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:07:59.573390 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c23ab797-4036-4e70-ae59-61cd682f65c2 None admin] PATH_INFO: `/v3/users` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:07:59.574111 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c23ab797-4036-4e70-ae59-61cd682f65c2 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:07:59.575696 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c23ab797-4036-4e70-ae59-61cd682f65c2 None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:07:59.601927 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 245/477] 10.222.0.18 () {60 vars in 1132 bytes} [Tue Sep 13 03:07:59 2022] POST /identity/v3/users => generated 249 bytes in 215 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 13 03:07:59.614468 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d5f3950-1316-44b3-baea-bec69a2f7f95 None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:07:59.652799 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80765dbf-8ede-45ed-9071-4d29ddd1e305 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:07:59.653423 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80765dbf-8ede-45ed-9071-4d29ddd1e305 None admin] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:07:59.653697 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80765dbf-8ede-45ed-9071-4d29ddd1e305 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:07:59.653982 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80765dbf-8ede-45ed-9071-4d29ddd1e305 None admin] PATH_INFO: `/v3/roles` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:07:59.654811 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80765dbf-8ede-45ed-9071-4d29ddd1e305 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:07:59.656883 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80765dbf-8ede-45ed-9071-4d29ddd1e305 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:07:59.667966 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.sql.core [None req-80765dbf-8ede-45ed-9071-4d29ddd1e305 None admin] Conflict role: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Sep 13 03:07:59.667966 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Sep 13 03:07:59.667966 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [parameters: {'id': '1b38775e9e2145ab991d5c5be545c06b', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Sep 13 03:07:59.667966 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=81399) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Sep 13 03:07:59.686327 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application [None req-80765dbf-8ede-45ed-9071-4d29ddd1e305 None admin] Conflict occurred attempting to store role - Duplicate entry found with name anotherrole.: keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Sep 13 03:07:59.686327 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 03:07:59.686327 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Sep 13 03:07:59.686327 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application self.dialect.do_execute( Sep 13 03:07:59.686327 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Sep 13 03:07:59.686327 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Sep 13 03:07:59.686327 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Sep 13 03:07:59.686327 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application result = self._query(query) Sep 13 03:07:59.686327 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Sep 13 03:07:59.686327 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application conn.query(q) Sep 13 03:07:59.686327 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Sep 13 03:07:59.686327 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Sep 13 03:07:59.686327 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Sep 13 03:07:59.686327 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application result.read() Sep 13 03:07:59.686327 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Sep 13 03:07:59.687328 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Sep 13 03:07:59.687328 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Sep 13 03:07:59.687328 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application packet.raise_for_error() Sep 13 03:07:59.687328 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Sep 13 03:07:59.687328 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Sep 13 03:07:59.687328 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Sep 13 03:07:59.687328 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Sep 13 03:07:59.687328 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Sep 13 03:07:59.687328 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Sep 13 03:07:59.687328 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Sep 13 03:07:59.687328 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Sep 13 03:07:59.687328 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 03:07:59.687328 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Sep 13 03:07:59.687328 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return method(*args, **kwargs) Sep 13 03:07:59.687328 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 33, in create_role Sep 13 03:07:59.687328 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return ref.to_dict() Sep 13 03:07:59.687328 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Sep 13 03:07:59.687328 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application next(self.gen) Sep 13 03:07:59.688234 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Sep 13 03:07:59.688234 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application yield resource Sep 13 03:07:59.688234 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Sep 13 03:07:59.688234 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application next(self.gen) Sep 13 03:07:59.688234 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Sep 13 03:07:59.688234 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application self.session.rollback() Sep 13 03:07:59.688234 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Sep 13 03:07:59.688234 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application self.force_reraise() Sep 13 03:07:59.688234 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Sep 13 03:07:59.688234 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application raise self.value Sep 13 03:07:59.688234 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Sep 13 03:07:59.688234 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Sep 13 03:07:59.688234 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Sep 13 03:07:59.688234 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application session.commit() Sep 13 03:07:59.688234 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Sep 13 03:07:59.688234 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Sep 13 03:07:59.688234 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Sep 13 03:07:59.689115 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application self._prepare_impl() Sep 13 03:07:59.689115 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Sep 13 03:07:59.689115 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application self.session.flush() Sep 13 03:07:59.689115 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Sep 13 03:07:59.689115 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application self._flush(objects) Sep 13 03:07:59.689115 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Sep 13 03:07:59.689115 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Sep 13 03:07:59.689115 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Sep 13 03:07:59.689115 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application compat.raise_( Sep 13 03:07:59.689115 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Sep 13 03:07:59.689115 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application raise exception Sep 13 03:07:59.689115 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Sep 13 03:07:59.689115 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application flush_context.execute() Sep 13 03:07:59.689115 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Sep 13 03:07:59.689115 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application rec.execute(self) Sep 13 03:07:59.689115 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Sep 13 03:07:59.689115 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Sep 13 03:07:59.689988 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Sep 13 03:07:59.689988 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application _emit_insert_statements( Sep 13 03:07:59.689988 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Sep 13 03:07:59.689988 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application c = connection._execute_20( Sep 13 03:07:59.689988 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Sep 13 03:07:59.689988 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Sep 13 03:07:59.689988 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Sep 13 03:07:59.689988 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Sep 13 03:07:59.689988 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Sep 13 03:07:59.689988 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application ret = self._execute_context( Sep 13 03:07:59.689988 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Sep 13 03:07:59.689988 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Sep 13 03:07:59.689988 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Sep 13 03:07:59.689988 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Sep 13 03:07:59.689988 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Sep 13 03:07:59.689988 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application raise exception Sep 13 03:07:59.691930 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Sep 13 03:07:59.691930 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application self.dialect.do_execute( Sep 13 03:07:59.691930 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Sep 13 03:07:59.691930 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Sep 13 03:07:59.691930 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Sep 13 03:07:59.691930 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application result = self._query(query) Sep 13 03:07:59.691930 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Sep 13 03:07:59.691930 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application conn.query(q) Sep 13 03:07:59.691930 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Sep 13 03:07:59.691930 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Sep 13 03:07:59.691930 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Sep 13 03:07:59.691930 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application result.read() Sep 13 03:07:59.691930 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Sep 13 03:07:59.691930 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Sep 13 03:07:59.691930 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Sep 13 03:07:59.691930 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application packet.raise_for_error() Sep 13 03:07:59.693875 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Sep 13 03:07:59.693875 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Sep 13 03:07:59.693875 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Sep 13 03:07:59.693875 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Sep 13 03:07:59.693875 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Sep 13 03:07:59.693875 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Sep 13 03:07:59.693875 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application [parameters: {'id': '1b38775e9e2145ab991d5c5be545c06b', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Sep 13 03:07:59.693875 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Sep 13 03:07:59.693875 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Sep 13 03:07:59.693875 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 13 03:07:59.693875 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Sep 13 03:07:59.693875 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 03:07:59.693875 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 03:07:59.693875 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 03:07:59.693875 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 03:07:59.693875 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 03:07:59.695836 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 03:07:59.695836 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 03:07:59.695836 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 03:07:59.695836 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 03:07:59.695836 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 03:07:59.695836 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 03:07:59.695836 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 115, in post Sep 13 03:07:59.695836 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application ref = PROVIDERS.role_api.create_role( Sep 13 03:07:59.695836 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 03:07:59.695836 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 03:07:59.695836 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1298, in create_role Sep 13 03:07:59.695836 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application ret = self.driver.create_role(role_id, role) Sep 13 03:07:59.695836 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Sep 13 03:07:59.695836 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Sep 13 03:07:59.695836 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Sep 13 03:07:59.695836 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Sep 13 03:07:59.704445 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 233/478] 10.222.0.18 () {60 vars in 1131 bytes} [Tue Sep 13 03:07:59 2022] POST /identity/v3/roles => generated 144 bytes in 268 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Sep 13 03:07:59.704574 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c23ab797-4036-4e70-ae59-61cd682f65c2 None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:07:59.791362 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:07:59.816021 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c23ab797-4036-4e70-ae59-61cd682f65c2 None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:07:59.851183 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c8f770b-5f33-4d46-83c8-f45d788b8394 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:07:59.851807 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c8f770b-5f33-4d46-83c8-f45d788b8394 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:07:59.852080 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c8f770b-5f33-4d46-83c8-f45d788b8394 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:07:59.852343 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c8f770b-5f33-4d46-83c8-f45d788b8394 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:07:59.862593 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c8f770b-5f33-4d46-83c8-f45d788b8394 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 03:07:59.868052 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application [None req-4c8f770b-5f33-4d46-83c8-f45d788b8394 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 13 03:07:59.868052 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 03:07:59.868052 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 03:07:59.868052 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 03:07:59.868052 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 03:07:59.868052 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 03:07:59.868052 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 03:07:59.868052 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 03:07:59.868052 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 03:07:59.868052 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 03:07:59.868052 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 03:07:59.868052 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 03:07:59.868052 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 03:07:59.868052 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 03:07:59.868052 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 03:07:59.868052 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 03:07:59.870100 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 03:07:59.870100 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 03:07:59.870100 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 03:07:59.870100 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 03:07:59.870100 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 03:07:59.870100 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 03:07:59.870100 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 03:07:59.870100 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 03:07:59.870100 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 03:07:59.870100 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 03:07:59.870100 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 03:07:59.870100 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application with Lock( Sep 13 03:07:59.870100 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 03:07:59.870100 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._enter() Sep 13 03:07:59.870100 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 03:07:59.870100 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 03:07:59.872068 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 03:07:59.872068 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.creator() Sep 13 03:07:59.872068 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 03:07:59.872068 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application created_value = creator( Sep 13 03:07:59.872068 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 03:07:59.872068 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 03:07:59.872068 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 03:07:59.872068 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 03:07:59.872068 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 03:07:59.872068 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 03:07:59.872068 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 13 03:07:59.872068 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Sep 13 03:07:59.872068 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 246/479] 10.222.0.18 () {56 vars in 1114 bytes} [Tue Sep 13 03:07:59 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 03:07:59.877523 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80765dbf-8ede-45ed-9071-4d29ddd1e305 None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:07:59.990961 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80765dbf-8ede-45ed-9071-4d29ddd1e305 None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:08:00.008229 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:08:00.029282 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc99c5d6-cf77-4607-9966-83bed4bc0828 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:08:00.029867 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc99c5d6-cf77-4607-9966-83bed4bc0828 None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:08:00.030141 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc99c5d6-cf77-4607-9966-83bed4bc0828 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:08:00.030405 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc99c5d6-cf77-4607-9966-83bed4bc0828 None admin] PATH_INFO: `/v3/roles` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:08:00.031674 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc99c5d6-cf77-4607-9966-83bed4bc0828 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 03:08:00.032055 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc99c5d6-cf77-4607-9966-83bed4bc0828 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:08:00.033476 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc99c5d6-cf77-4607-9966-83bed4bc0828 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:08:00.042255 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 234/480] 10.222.0.18 () {56 vars in 1111 bytes} [Tue Sep 13 03:07:59 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 03:08:01.207550 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3fea285-2367-469c-96fd-ab96600ad024 None None] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:08:01.208236 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3fea285-2367-469c-96fd-ab96600ad024 None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:08:01.208236 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3fea285-2367-469c-96fd-ab96600ad024 None None] PATH_INFO: `/` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:08:01.209687 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 247/481] 10.222.0.18 () {54 vars in 927 bytes} [Tue Sep 13 03:08:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 03:08:01.214759 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e4e0e8-6f59-4547-a554-780cd0988f3f None None] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:08:01.215243 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e4e0e8-6f59-4547-a554-780cd0988f3f None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:08:01.215394 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e4e0e8-6f59-4547-a554-780cd0988f3f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:08:01.236873 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.auth.core [None req-75e4e0e8-6f59-4547-a554-780cd0988f3f None None] MFA Rules not processed for user `49f280ab086b4610855c4ffa378517d9`. Rule list: `[]` (Enabled: `True`). {{(pid=81399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 03:08:01.238057 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-75e4e0e8-6f59-4547-a554-780cd0988f3f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:08:01.322240 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 235/482] 10.222.0.18 () {58 vars in 1024 bytes} [Tue Sep 13 03:08:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 13 03:08:01.642803 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3fea285-2367-469c-96fd-ab96600ad024 None None] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:08:01.663077 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-417f3980-bd24-4702-a1c2-4a5cafa07584 None None] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:08:01.663424 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-417f3980-bd24-4702-a1c2-4a5cafa07584 None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:08:01.663616 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-417f3980-bd24-4702-a1c2-4a5cafa07584 None None] PATH_INFO: `/` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:08:01.665233 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 236/483] 10.222.0.18 () {54 vars in 927 bytes} [Tue Sep 13 03:08:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 13 03:08:01.682583 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0180a6ef-3470-452d-9178-950c9c01d6b8 None None] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:08:01.682890 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0180a6ef-3470-452d-9178-950c9c01d6b8 None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:08:01.683167 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0180a6ef-3470-452d-9178-950c9c01d6b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:08:01.714252 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.auth.core [None req-0180a6ef-3470-452d-9178-950c9c01d6b8 None None] MFA Rules not processed for user `49f280ab086b4610855c4ffa378517d9`. Rule list: `[]` (Enabled: `True`). {{(pid=81399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 03:08:01.714252 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-0180a6ef-3470-452d-9178-950c9c01d6b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:08:01.799695 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 237/484] 10.222.0.18 () {58 vars in 1024 bytes} [Tue Sep 13 03:08:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 13 03:08:01.803917 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc2bb3fe-5db0-496d-aea6-b9e70b990b93 None None] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:08:01.804205 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc2bb3fe-5db0-496d-aea6-b9e70b990b93 None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:08:01.804479 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc2bb3fe-5db0-496d-aea6-b9e70b990b93 None None] PATH_INFO: `/` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:08:01.806227 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 238/485] 10.222.0.18 () {54 vars in 927 bytes} [Tue Sep 13 03:08:01 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 03:08:01.812163 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27bf0cc0-1ae8-4d99-bb37-66a26c1dd4d8 None None] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:08:01.812445 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27bf0cc0-1ae8-4d99-bb37-66a26c1dd4d8 None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:08:01.812717 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27bf0cc0-1ae8-4d99-bb37-66a26c1dd4d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:08:01.816922 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3fea285-2367-469c-96fd-ab96600ad024 None None] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:08:01.832673 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.auth.core [None req-27bf0cc0-1ae8-4d99-bb37-66a26c1dd4d8 None None] MFA Rules not processed for user `49f280ab086b4610855c4ffa378517d9`. Rule list: `[]` (Enabled: `True`). {{(pid=81399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 03:08:01.833849 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-27bf0cc0-1ae8-4d99-bb37-66a26c1dd4d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:08:01.879403 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:08:01.896485 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31563f7c-4950-42ee-a51f-3bb896136e70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:08:01.897039 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31563f7c-4950-42ee-a51f-3bb896136e70 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:08:01.897313 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31563f7c-4950-42ee-a51f-3bb896136e70 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:08:01.897584 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31563f7c-4950-42ee-a51f-3bb896136e70 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:08:01.912226 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:08:01.917998 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31563f7c-4950-42ee-a51f-3bb896136e70 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 03:08:01.925025 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application [None req-31563f7c-4950-42ee-a51f-3bb896136e70 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 13 03:08:01.925025 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 03:08:01.925025 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 03:08:01.925025 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 03:08:01.925025 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 03:08:01.925025 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 03:08:01.925025 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 03:08:01.925025 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 03:08:01.925025 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 03:08:01.925025 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 03:08:01.925025 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 03:08:01.925025 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 03:08:01.925025 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 03:08:01.925025 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 03:08:01.925025 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 03:08:01.925025 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 03:08:01.925944 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 03:08:01.925944 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 03:08:01.925944 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 03:08:01.925944 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 03:08:01.925944 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 03:08:01.925944 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 03:08:01.925944 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 03:08:01.925944 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 03:08:01.925944 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 03:08:01.925944 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 03:08:01.925944 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 03:08:01.925944 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application with Lock( Sep 13 03:08:01.925944 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 03:08:01.925944 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._enter() Sep 13 03:08:01.925944 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 03:08:01.925944 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 03:08:01.926779 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 03:08:01.926779 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.creator() Sep 13 03:08:01.926779 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 03:08:01.926779 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application created_value = creator( Sep 13 03:08:01.926779 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 03:08:01.926779 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 03:08:01.926779 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 03:08:01.926779 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 03:08:01.926779 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 03:08:01.926779 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 03:08:01.926779 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 13 03:08:01.926779 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Sep 13 03:08:01.927396 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 248/486] 10.222.0.18 () {56 vars in 1102 bytes} [Tue Sep 13 03:08:01 2022] GET /identity/v3/roles/service => generated 85 bytes in 285 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 03:08:01.932799 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31563f7c-4950-42ee-a51f-3bb896136e70 None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:08:01.956839 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 239/487] 10.222.0.18 () {58 vars in 1024 bytes} [Tue Sep 13 03:08:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 13 03:08:02.046450 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31563f7c-4950-42ee-a51f-3bb896136e70 None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:08:02.089600 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d201761-e387-43c9-b2f5-d14e6d0f2bd0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:08:02.090202 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d201761-e387-43c9-b2f5-d14e6d0f2bd0 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:08:02.090473 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d201761-e387-43c9-b2f5-d14e6d0f2bd0 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:08:02.090810 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d201761-e387-43c9-b2f5-d14e6d0f2bd0 None admin] PATH_INFO: `/v3/roles` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:08:02.091468 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d201761-e387-43c9-b2f5-d14e6d0f2bd0 None admin] RBAC: Adding query filter params (name=service) {{(pid=81400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 03:08:02.091765 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d201761-e387-43c9-b2f5-d14e6d0f2bd0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:08:02.093140 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d201761-e387-43c9-b2f5-d14e6d0f2bd0 None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:08:02.102212 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 249/488] 10.222.0.18 () {56 vars in 1103 bytes} [Tue Sep 13 03:08:01 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 03:08:02.107640 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27bf0cc0-1ae8-4d99-bb37-66a26c1dd4d8 None None] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:08:02.164262 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d201761-e387-43c9-b2f5-d14e6d0f2bd0 None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:08:02.226431 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:08:02.227926 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27bf0cc0-1ae8-4d99-bb37-66a26c1dd4d8 None None] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:08:02.266735 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-243b196d-1041-4f54-b548-941ec6191d2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:08:02.267504 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-243b196d-1041-4f54-b548-941ec6191d2d None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:08:02.267785 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-243b196d-1041-4f54-b548-941ec6191d2d None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:08:02.268052 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-243b196d-1041-4f54-b548-941ec6191d2d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:08:02.272600 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d201761-e387-43c9-b2f5-d14e6d0f2bd0 None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:08:02.279183 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-243b196d-1041-4f54-b548-941ec6191d2d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:08:02.280643 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-243b196d-1041-4f54-b548-941ec6191d2d None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:08:02.297596 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application [None req-243b196d-1041-4f54-b548-941ec6191d2d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 03:08:02.297596 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 03:08:02.297596 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 03:08:02.297596 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 03:08:02.297596 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 03:08:02.297596 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 03:08:02.297596 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 03:08:02.297596 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 03:08:02.297596 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 03:08:02.297596 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 03:08:02.297596 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 03:08:02.297596 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 03:08:02.297596 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 13 03:08:02.297596 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 13 03:08:02.297596 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 13 03:08:02.297596 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 13 03:08:02.298545 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 03:08:02.298545 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 03:08:02.298545 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 03:08:02.298545 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 03:08:02.298545 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 03:08:02.298545 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 03:08:02.298545 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 03:08:02.298545 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application with Lock( Sep 13 03:08:02.298545 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 03:08:02.298545 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._enter() Sep 13 03:08:02.298545 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 03:08:02.298545 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 03:08:02.298545 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 03:08:02.298545 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.creator() Sep 13 03:08:02.298545 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 03:08:02.298545 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application created_value = creator( Sep 13 03:08:02.298545 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 13 03:08:02.299498 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 13 03:08:02.299498 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 03:08:02.299498 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Sep 13 03:08:02.299498 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 240/489] 10.222.0.18 () {56 vars in 1108 bytes} [Tue Sep 13 03:08:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 03:08:02.308324 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-243b196d-1041-4f54-b548-941ec6191d2d None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:08:02.326883 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a041ecd2-c789-4c28-a373-cdc67b281544 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:08:02.327508 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a041ecd2-c789-4c28-a373-cdc67b281544 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:08:02.328353 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a041ecd2-c789-4c28-a373-cdc67b281544 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:08:02.328617 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a041ecd2-c789-4c28-a373-cdc67b281544 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:08:02.349466 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a041ecd2-c789-4c28-a373-cdc67b281544 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 03:08:02.360647 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application [None req-a041ecd2-c789-4c28-a373-cdc67b281544 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 13 03:08:02.360647 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 03:08:02.360647 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 03:08:02.360647 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 03:08:02.360647 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 03:08:02.360647 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 03:08:02.360647 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 03:08:02.360647 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 03:08:02.360647 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 03:08:02.360647 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 03:08:02.360647 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 03:08:02.360647 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 03:08:02.360647 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 03:08:02.360647 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 03:08:02.360647 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 03:08:02.360647 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 03:08:02.361597 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 03:08:02.361597 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 03:08:02.361597 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 03:08:02.361597 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 03:08:02.361597 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 03:08:02.361597 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 03:08:02.361597 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 03:08:02.361597 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 03:08:02.361597 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 03:08:02.361597 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 03:08:02.361597 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 03:08:02.361597 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application with Lock( Sep 13 03:08:02.361597 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 03:08:02.361597 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._enter() Sep 13 03:08:02.361597 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 03:08:02.361597 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 03:08:02.362391 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 03:08:02.362391 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.creator() Sep 13 03:08:02.362391 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 03:08:02.362391 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application created_value = creator( Sep 13 03:08:02.362391 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 03:08:02.362391 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 03:08:02.362391 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 03:08:02.362391 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 03:08:02.362391 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 03:08:02.362391 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 03:08:02.362391 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 13 03:08:02.362391 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Sep 13 03:08:02.363543 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:08:02.365479 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 250/490] 10.222.0.18 () {56 vars in 1102 bytes} [Tue Sep 13 03:08:02 2022] GET /identity/v3/roles/service => generated 85 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 13 03:08:02.369874 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a041ecd2-c789-4c28-a373-cdc67b281544 None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:08:02.478489 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-243b196d-1041-4f54-b548-941ec6191d2d None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:08:02.510891 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a041ecd2-c789-4c28-a373-cdc67b281544 None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:08:02.522470 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3af17b0a-1dd3-403b-887e-f5d7fffd03b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:08:02.523391 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3af17b0a-1dd3-403b-887e-f5d7fffd03b0 None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:08:02.523674 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3af17b0a-1dd3-403b-887e-f5d7fffd03b0 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:08:02.523950 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3af17b0a-1dd3-403b-887e-f5d7fffd03b0 None admin] PATH_INFO: `/v3/domains` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:08:02.524834 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3af17b0a-1dd3-403b-887e-f5d7fffd03b0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 03:08:02.525141 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3af17b0a-1dd3-403b-887e-f5d7fffd03b0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:08:02.527313 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3af17b0a-1dd3-403b-887e-f5d7fffd03b0 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:08:02.533117 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:08:02.547051 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 241/491] 10.222.0.18 () {56 vars in 1109 bytes} [Tue Sep 13 03:08:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 03:08:02.547922 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3af17b0a-1dd3-403b-887e-f5d7fffd03b0 None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:08:02.555764 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75c2976a-1d03-4cbd-a80c-dd7f93ef57b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:08:02.556373 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c2976a-1d03-4cbd-a80c-dd7f93ef57b9 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:08:02.556651 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c2976a-1d03-4cbd-a80c-dd7f93ef57b9 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:08:02.557063 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c2976a-1d03-4cbd-a80c-dd7f93ef57b9 None admin] PATH_INFO: `/v3/roles` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:08:02.557724 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75c2976a-1d03-4cbd-a80c-dd7f93ef57b9 None admin] RBAC: Adding query filter params (name=service) {{(pid=81400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 03:08:02.558020 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75c2976a-1d03-4cbd-a80c-dd7f93ef57b9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:08:02.559436 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75c2976a-1d03-4cbd-a80c-dd7f93ef57b9 None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:08:02.568844 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 251/492] 10.222.0.18 () {56 vars in 1103 bytes} [Tue Sep 13 03:08:02 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 03:08:02.569629 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75c2976a-1d03-4cbd-a80c-dd7f93ef57b9 None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:08:02.644566 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:08:02.665019 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3af17b0a-1dd3-403b-887e-f5d7fffd03b0 None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:08:02.668275 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75c2976a-1d03-4cbd-a80c-dd7f93ef57b9 None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:08:02.699229 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2de3eeeb-144c-478a-b3ae-09891533ec73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:08:02.699856 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2de3eeeb-144c-478a-b3ae-09891533ec73 None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:08:02.700123 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2de3eeeb-144c-478a-b3ae-09891533ec73 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:08:02.700391 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2de3eeeb-144c-478a-b3ae-09891533ec73 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:08:02.701977 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9547a7e9-15aa-4130-9399-402a4eacb6a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:08:02.702639 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9547a7e9-15aa-4130-9399-402a4eacb6a6 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:08:02.702903 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9547a7e9-15aa-4130-9399-402a4eacb6a6 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:08:02.703173 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9547a7e9-15aa-4130-9399-402a4eacb6a6 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:08:02.711020 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9547a7e9-15aa-4130-9399-402a4eacb6a6 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:08:02.711438 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2de3eeeb-144c-478a-b3ae-09891533ec73 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 03:08:02.712607 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9547a7e9-15aa-4130-9399-402a4eacb6a6 None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:08:02.716420 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application [None req-2de3eeeb-144c-478a-b3ae-09891533ec73 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 13 03:08:02.716420 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 03:08:02.716420 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 03:08:02.716420 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 03:08:02.716420 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 03:08:02.716420 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 03:08:02.716420 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 03:08:02.716420 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 03:08:02.716420 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 03:08:02.716420 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 03:08:02.716420 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 03:08:02.716420 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 03:08:02.716420 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 03:08:02.716420 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 03:08:02.716420 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 03:08:02.716420 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 03:08:02.717366 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 03:08:02.717366 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 03:08:02.717366 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 03:08:02.717366 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 03:08:02.717366 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 03:08:02.717366 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 03:08:02.717366 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 03:08:02.717366 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 03:08:02.717366 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 03:08:02.717366 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 03:08:02.717366 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 03:08:02.717366 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application with Lock( Sep 13 03:08:02.717366 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 03:08:02.717366 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._enter() Sep 13 03:08:02.717366 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 03:08:02.717366 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 03:08:02.718719 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 03:08:02.718719 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.creator() Sep 13 03:08:02.718719 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 03:08:02.718719 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application created_value = creator( Sep 13 03:08:02.718719 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 03:08:02.718719 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 03:08:02.718719 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 03:08:02.718719 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 03:08:02.718719 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 03:08:02.718719 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 03:08:02.718719 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 13 03:08:02.718719 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Sep 13 03:08:02.718719 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 242/493] 10.222.0.18 () {56 vars in 1102 bytes} [Tue Sep 13 03:08:02 2022] GET /identity/v3/roles/service => generated 85 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 03:08:02.719949 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ff2a08-c275-4ce8-95a9-9f8d78156562 None None] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:08:02.719949 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ff2a08-c275-4ce8-95a9-9f8d78156562 None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:08:02.720099 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ff2a08-c275-4ce8-95a9-9f8d78156562 None None] PATH_INFO: `/` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:08:02.721420 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 243/494] 10.222.0.18 () {54 vars in 927 bytes} [Tue Sep 13 03:08:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 03:08:02.721959 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32ff2a08-c275-4ce8-95a9-9f8d78156562 None None] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:08:02.732370 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application [None req-9547a7e9-15aa-4130-9399-402a4eacb6a6 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 13 03:08:02.732370 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 03:08:02.732370 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 03:08:02.732370 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 03:08:02.732370 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 03:08:02.732370 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 03:08:02.732370 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 03:08:02.732370 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 03:08:02.732370 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 03:08:02.732370 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 03:08:02.732370 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 03:08:02.732370 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 03:08:02.732370 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 13 03:08:02.732370 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 13 03:08:02.732370 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 13 03:08:02.732370 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 13 03:08:02.733301 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 03:08:02.733301 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 03:08:02.733301 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 13 03:08:02.733301 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 13 03:08:02.733301 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 13 03:08:02.733301 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 13 03:08:02.733301 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 03:08:02.733301 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 03:08:02.733301 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 03:08:02.733301 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 03:08:02.733301 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 03:08:02.733301 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application with Lock( Sep 13 03:08:02.733301 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 03:08:02.733301 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._enter() Sep 13 03:08:02.733301 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 03:08:02.733301 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 03:08:02.733301 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 03:08:02.733301 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.creator() Sep 13 03:08:02.735496 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 03:08:02.735496 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application created_value = creator( Sep 13 03:08:02.735496 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 13 03:08:02.735496 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 13 03:08:02.735496 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 13 03:08:02.735496 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 13 03:08:02.735496 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 13 03:08:02.735496 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 13 03:08:02.735496 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 13 03:08:02.735496 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Sep 13 03:08:02.735496 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 252/495] 10.222.0.18 () {56 vars in 1093 bytes} [Tue Sep 13 03:08:02 2022] GET /identity/v3/users/nova => generated 82 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 03:08:02.735496 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9547a7e9-15aa-4130-9399-402a4eacb6a6 None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:08:02.783872 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:08:02.840221 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32ff2a08-c275-4ce8-95a9-9f8d78156562 None None] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:08:02.844616 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9547a7e9-15aa-4130-9399-402a4eacb6a6 None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:08:02.876963 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa10b122-ca06-4980-81b1-8889be0c5c9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:08:02.877573 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa10b122-ca06-4980-81b1-8889be0c5c9c None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:08:02.877677 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5faecf84-10ba-4878-8038-c76e3c66b264 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:08:02.877856 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa10b122-ca06-4980-81b1-8889be0c5c9c None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:08:02.878115 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa10b122-ca06-4980-81b1-8889be0c5c9c None admin] PATH_INFO: `/v3/roles` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:08:02.878286 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5faecf84-10ba-4878-8038-c76e3c66b264 None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:08:02.878531 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5faecf84-10ba-4878-8038-c76e3c66b264 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:08:02.878807 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa10b122-ca06-4980-81b1-8889be0c5c9c None admin] RBAC: Adding query filter params (name=service) {{(pid=81400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 03:08:02.878899 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5faecf84-10ba-4878-8038-c76e3c66b264 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:08:02.879111 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa10b122-ca06-4980-81b1-8889be0c5c9c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:08:02.880528 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa10b122-ca06-4980-81b1-8889be0c5c9c None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:08:02.889879 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 253/496] 10.222.0.18 () {56 vars in 1103 bytes} [Tue Sep 13 03:08:02 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 03:08:02.891770 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-388d39b5-d116-4085-83cc-534c74af42a2 None None] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:08:02.892054 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-388d39b5-d116-4085-83cc-534c74af42a2 None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:08:02.892323 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-388d39b5-d116-4085-83cc-534c74af42a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:08:02.896059 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5faecf84-10ba-4878-8038-c76e3c66b264 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:08:02.897744 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5faecf84-10ba-4878-8038-c76e3c66b264 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:08:02.900078 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:08:02.913235 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.auth.core [None req-388d39b5-d116-4085-83cc-534c74af42a2 None None] MFA Rules not processed for user `49f280ab086b4610855c4ffa378517d9`. Rule list: `[]` (Enabled: `True`). {{(pid=81400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 03:08:02.914531 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-388d39b5-d116-4085-83cc-534c74af42a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:08:02.918050 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application [None req-5faecf84-10ba-4878-8038-c76e3c66b264 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 03:08:02.918050 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 03:08:02.918050 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 03:08:02.918050 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 03:08:02.918050 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 03:08:02.918050 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 03:08:02.918050 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 03:08:02.918050 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 03:08:02.918050 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 03:08:02.918050 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 03:08:02.918050 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 03:08:02.918050 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 03:08:02.918050 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 13 03:08:02.918050 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 13 03:08:02.918050 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 13 03:08:02.918050 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 13 03:08:02.919036 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 03:08:02.919036 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 03:08:02.919036 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 03:08:02.919036 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 03:08:02.919036 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 03:08:02.919036 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 03:08:02.919036 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 03:08:02.919036 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application with Lock( Sep 13 03:08:02.919036 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 03:08:02.919036 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._enter() Sep 13 03:08:02.919036 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 03:08:02.919036 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 03:08:02.919036 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 03:08:02.919036 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.creator() Sep 13 03:08:02.919036 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 03:08:02.919036 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application created_value = creator( Sep 13 03:08:02.919036 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 13 03:08:02.920196 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 13 03:08:02.920196 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 03:08:02.920196 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Sep 13 03:08:02.923919 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 244/497] 10.222.0.18 () {56 vars in 1108 bytes} [Tue Sep 13 03:08:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 03:08:02.924885 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5faecf84-10ba-4878-8038-c76e3c66b264 None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:08:02.994720 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 254/498] 10.222.0.18 () {58 vars in 1024 bytes} [Tue Sep 13 03:08:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 13 03:08:02.994720 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-388d39b5-d116-4085-83cc-534c74af42a2 None None] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:08:03.026728 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:08:03.039062 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5faecf84-10ba-4878-8038-c76e3c66b264 None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:08:03.083962 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c222832-b8d8-4d64-9a92-1470a24527dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:08:03.084581 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c222832-b8d8-4d64-9a92-1470a24527dd None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:08:03.084858 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c222832-b8d8-4d64-9a92-1470a24527dd None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:08:03.085129 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c222832-b8d8-4d64-9a92-1470a24527dd None admin] PATH_INFO: `/v3/users` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:08:03.085847 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c222832-b8d8-4d64-9a92-1470a24527dd None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 03:08:03.086148 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c222832-b8d8-4d64-9a92-1470a24527dd None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:08:03.087599 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c222832-b8d8-4d64-9a92-1470a24527dd None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:08:03.095734 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-388d39b5-d116-4085-83cc-534c74af42a2 None None] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:08:03.124223 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 245/499] 10.222.0.18 () {56 vars in 1133 bytes} [Tue Sep 13 03:08:02 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 03:08:03.125067 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c222832-b8d8-4d64-9a92-1470a24527dd None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:08:03.133132 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f60d806c-2467-42f8-a23b-c61fe9bdc724 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:08:03.133758 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f60d806c-2467-42f8-a23b-c61fe9bdc724 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:08:03.134031 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f60d806c-2467-42f8-a23b-c61fe9bdc724 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:08:03.134298 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f60d806c-2467-42f8-a23b-c61fe9bdc724 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:08:03.145294 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f60d806c-2467-42f8-a23b-c61fe9bdc724 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:08:03.146763 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f60d806c-2467-42f8-a23b-c61fe9bdc724 None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:08:03.161484 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application [None req-f60d806c-2467-42f8-a23b-c61fe9bdc724 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 03:08:03.161484 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 03:08:03.161484 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 03:08:03.161484 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 03:08:03.161484 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 03:08:03.161484 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 03:08:03.161484 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 03:08:03.161484 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 03:08:03.161484 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 03:08:03.161484 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 03:08:03.161484 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 03:08:03.161484 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 03:08:03.161484 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 13 03:08:03.161484 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 13 03:08:03.161484 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 13 03:08:03.161484 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 13 03:08:03.162384 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 03:08:03.162384 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 03:08:03.162384 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 03:08:03.162384 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 03:08:03.162384 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 03:08:03.162384 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 03:08:03.162384 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 03:08:03.162384 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application with Lock( Sep 13 03:08:03.162384 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 03:08:03.162384 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._enter() Sep 13 03:08:03.162384 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 03:08:03.162384 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 03:08:03.162384 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 03:08:03.162384 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.creator() Sep 13 03:08:03.162384 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 03:08:03.162384 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application created_value = creator( Sep 13 03:08:03.162384 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 13 03:08:03.163274 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 13 03:08:03.163274 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 03:08:03.163274 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Sep 13 03:08:03.163274 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 255/500] 10.222.0.18 () {56 vars in 1108 bytes} [Tue Sep 13 03:08:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 03:08:03.163761 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9e2b3a4-28cd-4ef2-81e9-2c6ac10625ed None None] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:08:03.163967 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9e2b3a4-28cd-4ef2-81e9-2c6ac10625ed None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:08:03.164233 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9e2b3a4-28cd-4ef2-81e9-2c6ac10625ed None None] PATH_INFO: `/` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:08:03.169377 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 256/501] 10.222.0.18 () {54 vars in 927 bytes} [Tue Sep 13 03:08:03 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 03:08:03.169377 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9e2b3a4-28cd-4ef2-81e9-2c6ac10625ed None None] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:08:03.190411 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:08:03.243874 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c222832-b8d8-4d64-9a92-1470a24527dd None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:08:03.256357 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:08:03.273185 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9e2b3a4-28cd-4ef2-81e9-2c6ac10625ed None None] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:08:03.280346 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d422d95-db9b-4d71-9bce-e78e970ffc9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:08:03.280984 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d422d95-db9b-4d71-9bce-e78e970ffc9d None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:08:03.281262 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d422d95-db9b-4d71-9bce-e78e970ffc9d None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:08:03.281533 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d422d95-db9b-4d71-9bce-e78e970ffc9d None admin] PATH_INFO: `/v3/domains` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:08:03.282172 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d422d95-db9b-4d71-9bce-e78e970ffc9d None admin] RBAC: Adding query filter params (name=Default) {{(pid=81399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 03:08:03.282476 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d422d95-db9b-4d71-9bce-e78e970ffc9d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:08:03.283902 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d422d95-db9b-4d71-9bce-e78e970ffc9d None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:08:03.294851 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 246/502] 10.222.0.18 () {56 vars in 1109 bytes} [Tue Sep 13 03:08:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 03:08:03.295655 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d422d95-db9b-4d71-9bce-e78e970ffc9d None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:08:03.307893 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec1da3fe-1c73-4c32-a324-6ba3216153e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:08:03.308489 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec1da3fe-1c73-4c32-a324-6ba3216153e9 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:08:03.308796 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec1da3fe-1c73-4c32-a324-6ba3216153e9 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:08:03.309067 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec1da3fe-1c73-4c32-a324-6ba3216153e9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:08:03.374328 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec1da3fe-1c73-4c32-a324-6ba3216153e9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:08:03.376337 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec1da3fe-1c73-4c32-a324-6ba3216153e9 None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:08:03.394081 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d422d95-db9b-4d71-9bce-e78e970ffc9d None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:08:03.411132 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:08:03.432010 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29bdfabb-6a58-4dad-8051-0d34cec07279 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:08:03.432616 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29bdfabb-6a58-4dad-8051-0d34cec07279 None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:08:03.432892 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29bdfabb-6a58-4dad-8051-0d34cec07279 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:08:03.433163 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29bdfabb-6a58-4dad-8051-0d34cec07279 None admin] PATH_INFO: `/v3/domains` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:08:03.433791 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29bdfabb-6a58-4dad-8051-0d34cec07279 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 03:08:03.434085 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29bdfabb-6a58-4dad-8051-0d34cec07279 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:08:03.435490 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29bdfabb-6a58-4dad-8051-0d34cec07279 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:08:03.447649 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 247/503] 10.222.0.18 () {56 vars in 1109 bytes} [Tue Sep 13 03:08:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 03:08:03.448772 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4431b550-3a40-4e8b-94c4-e79681354fd5 None None] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:08:03.449056 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4431b550-3a40-4e8b-94c4-e79681354fd5 None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:08:03.449325 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4431b550-3a40-4e8b-94c4-e79681354fd5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:08:03.459907 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:08:03.467151 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.auth.core [None req-4431b550-3a40-4e8b-94c4-e79681354fd5 None None] MFA Rules not processed for user `49f280ab086b4610855c4ffa378517d9`. Rule list: `[]` (Enabled: `True`). {{(pid=81399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 03:08:03.468363 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-4431b550-3a40-4e8b-94c4-e79681354fd5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:08:03.476456 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 257/504] 10.222.0.18 () {58 vars in 1282 bytes} [Tue Sep 13 03:08:03 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 13 03:08:03.477264 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec1da3fe-1c73-4c32-a324-6ba3216153e9 None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:08:03.546883 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 248/505] 10.222.0.18 () {58 vars in 1024 bytes} [Tue Sep 13 03:08:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 13 03:08:03.546883 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4431b550-3a40-4e8b-94c4-e79681354fd5 None None] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:08:03.593314 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec1da3fe-1c73-4c32-a324-6ba3216153e9 None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:08:03.629013 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ac33b4d-21d9-4038-841a-6379c0c14c78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:08:03.629633 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac33b4d-21d9-4038-841a-6379c0c14c78 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:08:03.630039 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac33b4d-21d9-4038-841a-6379c0c14c78 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:08:03.630286 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac33b4d-21d9-4038-841a-6379c0c14c78 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:08:03.633518 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4431b550-3a40-4e8b-94c4-e79681354fd5 None None] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:08:03.635030 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:08:03.643012 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ac33b4d-21d9-4038-841a-6379c0c14c78 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 03:08:03.648731 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application [None req-7ac33b4d-21d9-4038-841a-6379c0c14c78 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 13 03:08:03.648731 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 03:08:03.648731 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 03:08:03.648731 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 03:08:03.648731 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 03:08:03.648731 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 03:08:03.648731 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 03:08:03.648731 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 03:08:03.648731 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 03:08:03.648731 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 03:08:03.648731 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 03:08:03.648731 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 03:08:03.648731 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 03:08:03.648731 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 03:08:03.648731 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 03:08:03.648731 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 03:08:03.649613 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 03:08:03.649613 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 03:08:03.649613 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 03:08:03.649613 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 03:08:03.649613 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 03:08:03.649613 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 03:08:03.649613 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 03:08:03.649613 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 03:08:03.649613 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 03:08:03.649613 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 03:08:03.649613 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 03:08:03.649613 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application with Lock( Sep 13 03:08:03.649613 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 03:08:03.649613 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._enter() Sep 13 03:08:03.649613 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 03:08:03.649613 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 03:08:03.650428 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 03:08:03.650428 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.creator() Sep 13 03:08:03.650428 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 03:08:03.650428 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application created_value = creator( Sep 13 03:08:03.650428 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 03:08:03.650428 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 03:08:03.650428 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 03:08:03.650428 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 03:08:03.650428 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 03:08:03.650428 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 03:08:03.650428 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 13 03:08:03.650428 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Sep 13 03:08:03.650428 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 258/506] 10.222.0.18 () {56 vars in 1102 bytes} [Tue Sep 13 03:08:03 2022] GET /identity/v3/roles/service => generated 85 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 03:08:03.651680 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ac33b4d-21d9-4038-841a-6379c0c14c78 None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:08:03.674865 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3983db4-cc16-49d6-8767-1bb913d5fce3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:08:03.675803 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3983db4-cc16-49d6-8767-1bb913d5fce3 None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:08:03.676287 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3983db4-cc16-49d6-8767-1bb913d5fce3 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:08:03.676744 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3983db4-cc16-49d6-8767-1bb913d5fce3 None admin] PATH_INFO: `/v3/users/glance` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:08:03.685612 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3983db4-cc16-49d6-8767-1bb913d5fce3 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:08:03.688377 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3983db4-cc16-49d6-8767-1bb913d5fce3 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:08:03.701872 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:08:03.704291 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application [None req-c3983db4-cc16-49d6-8767-1bb913d5fce3 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Sep 13 03:08:03.704291 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 03:08:03.704291 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 03:08:03.704291 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 03:08:03.704291 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 03:08:03.704291 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 03:08:03.704291 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 03:08:03.704291 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 03:08:03.704291 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 03:08:03.704291 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 03:08:03.704291 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 03:08:03.704291 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 03:08:03.704291 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 13 03:08:03.704291 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 13 03:08:03.704291 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 13 03:08:03.704291 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 13 03:08:03.705223 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 03:08:03.705223 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 03:08:03.705223 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 13 03:08:03.705223 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 13 03:08:03.705223 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 13 03:08:03.705223 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 13 03:08:03.705223 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 03:08:03.705223 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 03:08:03.705223 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 03:08:03.705223 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 03:08:03.705223 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 03:08:03.705223 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application with Lock( Sep 13 03:08:03.705223 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 03:08:03.705223 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._enter() Sep 13 03:08:03.705223 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 03:08:03.705223 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 03:08:03.705223 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 03:08:03.705223 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.creator() Sep 13 03:08:03.706132 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 03:08:03.706132 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application created_value = creator( Sep 13 03:08:03.706132 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 13 03:08:03.706132 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 13 03:08:03.706132 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 13 03:08:03.706132 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 13 03:08:03.706132 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 13 03:08:03.706132 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 13 03:08:03.706132 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Sep 13 03:08:03.706132 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Sep 13 03:08:03.708738 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 249/507] 10.222.0.18 () {56 vars in 1099 bytes} [Tue Sep 13 03:08:03 2022] GET /identity/v3/users/glance => generated 84 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 03:08:03.709335 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3983db4-cc16-49d6-8767-1bb913d5fce3 None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:08:03.765147 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ac33b4d-21d9-4038-841a-6379c0c14c78 None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:08:03.802259 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa29901e-92a8-4b65-a7c7-db902b396cf5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:08:03.802939 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa29901e-92a8-4b65-a7c7-db902b396cf5 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:08:03.803206 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa29901e-92a8-4b65-a7c7-db902b396cf5 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:08:03.803480 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa29901e-92a8-4b65-a7c7-db902b396cf5 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:08:03.810726 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa29901e-92a8-4b65-a7c7-db902b396cf5 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:08:03.812816 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa29901e-92a8-4b65-a7c7-db902b396cf5 None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:08:03.818031 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3983db4-cc16-49d6-8767-1bb913d5fce3 None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:08:03.824463 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application [None req-aa29901e-92a8-4b65-a7c7-db902b396cf5 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Sep 13 03:08:03.824463 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 03:08:03.824463 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 03:08:03.824463 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 03:08:03.824463 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 03:08:03.824463 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 03:08:03.824463 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 03:08:03.824463 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 03:08:03.824463 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 03:08:03.824463 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 03:08:03.824463 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 03:08:03.824463 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 03:08:03.824463 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 13 03:08:03.824463 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 13 03:08:03.824463 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 13 03:08:03.824463 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 13 03:08:03.825353 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 03:08:03.825353 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 03:08:03.825353 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 13 03:08:03.825353 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 13 03:08:03.825353 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 13 03:08:03.825353 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 13 03:08:03.825353 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 03:08:03.825353 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 03:08:03.825353 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 03:08:03.825353 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 03:08:03.825353 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 03:08:03.825353 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application with Lock( Sep 13 03:08:03.825353 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 03:08:03.825353 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._enter() Sep 13 03:08:03.825353 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 03:08:03.825353 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 03:08:03.825353 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 03:08:03.825353 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.creator() Sep 13 03:08:03.826267 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 03:08:03.826267 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application created_value = creator( Sep 13 03:08:03.826267 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 13 03:08:03.826267 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 13 03:08:03.826267 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 13 03:08:03.826267 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 13 03:08:03.826267 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 13 03:08:03.826267 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 13 03:08:03.826267 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Sep 13 03:08:03.826267 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Sep 13 03:08:03.826267 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 259/508] 10.222.0.18 () {56 vars in 1099 bytes} [Tue Sep 13 03:08:03 2022] GET /identity/v3/users/cinder => generated 84 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 03:08:03.826882 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa29901e-92a8-4b65-a7c7-db902b396cf5 None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:08:03.854228 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2d8363e-dba0-4040-a8a0-45b35a1a6ff5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:08:03.854882 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d8363e-dba0-4040-a8a0-45b35a1a6ff5 None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:08:03.855162 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d8363e-dba0-4040-a8a0-45b35a1a6ff5 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:08:03.855433 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d8363e-dba0-4040-a8a0-45b35a1a6ff5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:08:03.866129 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2d8363e-dba0-4040-a8a0-45b35a1a6ff5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:08:03.867778 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2d8363e-dba0-4040-a8a0-45b35a1a6ff5 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:08:03.882433 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application [None req-a2d8363e-dba0-4040-a8a0-45b35a1a6ff5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 03:08:03.882433 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 03:08:03.882433 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 03:08:03.882433 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 03:08:03.882433 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 03:08:03.882433 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 03:08:03.882433 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 03:08:03.882433 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 03:08:03.882433 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 03:08:03.882433 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 03:08:03.882433 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 03:08:03.882433 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 03:08:03.882433 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 13 03:08:03.882433 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 13 03:08:03.882433 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 13 03:08:03.882433 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 13 03:08:03.883375 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 03:08:03.883375 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 03:08:03.883375 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 03:08:03.883375 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 03:08:03.883375 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 03:08:03.883375 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 03:08:03.883375 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 03:08:03.883375 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application with Lock( Sep 13 03:08:03.883375 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 03:08:03.883375 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._enter() Sep 13 03:08:03.883375 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 03:08:03.883375 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 03:08:03.883375 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 03:08:03.883375 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.creator() Sep 13 03:08:03.883375 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 03:08:03.883375 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application created_value = creator( Sep 13 03:08:03.883375 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 13 03:08:03.884236 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 13 03:08:03.884236 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 03:08:03.884236 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Sep 13 03:08:03.884236 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 250/509] 10.222.0.18 () {56 vars in 1108 bytes} [Tue Sep 13 03:08:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 03:08:03.884472 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2d8363e-dba0-4040-a8a0-45b35a1a6ff5 None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:08:03.899513 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:08:03.927081 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa29901e-92a8-4b65-a7c7-db902b396cf5 None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:08:03.933972 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:08:03.960865 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42349b76-e0aa-4891-8f2f-67806cd8ca4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:08:03.961464 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42349b76-e0aa-4891-8f2f-67806cd8ca4d None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:08:03.961740 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42349b76-e0aa-4891-8f2f-67806cd8ca4d None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:08:03.962008 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42349b76-e0aa-4891-8f2f-67806cd8ca4d None admin] PATH_INFO: `/v3/roles` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:08:03.962691 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42349b76-e0aa-4891-8f2f-67806cd8ca4d None admin] RBAC: Adding query filter params (name=service) {{(pid=81400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 03:08:03.962994 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42349b76-e0aa-4891-8f2f-67806cd8ca4d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:08:03.964386 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42349b76-e0aa-4891-8f2f-67806cd8ca4d None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:08:03.972778 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 260/510] 10.222.0.18 () {56 vars in 1103 bytes} [Tue Sep 13 03:08:03 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 03:08:03.973836 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42349b76-e0aa-4891-8f2f-67806cd8ca4d None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:08:03.990708 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2d8363e-dba0-4040-a8a0-45b35a1a6ff5 None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:08:04.023423 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ddf0e94-64ee-4312-9570-cc8a9f70fe33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:08:04.024100 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ddf0e94-64ee-4312-9570-cc8a9f70fe33 None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:08:04.024448 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ddf0e94-64ee-4312-9570-cc8a9f70fe33 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:08:04.024786 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ddf0e94-64ee-4312-9570-cc8a9f70fe33 None admin] PATH_INFO: `/v3/users` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:08:04.025572 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ddf0e94-64ee-4312-9570-cc8a9f70fe33 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=81399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 03:08:04.025946 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ddf0e94-64ee-4312-9570-cc8a9f70fe33 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:08:04.027407 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ddf0e94-64ee-4312-9570-cc8a9f70fe33 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:08:04.045595 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 251/511] 10.222.0.18 () {56 vars in 1137 bytes} [Tue Sep 13 03:08:03 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 03:08:04.046942 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ddf0e94-64ee-4312-9570-cc8a9f70fe33 None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:08:04.075668 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42349b76-e0aa-4891-8f2f-67806cd8ca4d None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:08:04.108375 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3848bce4-4570-4b90-9a7a-31fde993fafd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:08:04.109608 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3848bce4-4570-4b90-9a7a-31fde993fafd None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:08:04.109884 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3848bce4-4570-4b90-9a7a-31fde993fafd None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:08:04.110151 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3848bce4-4570-4b90-9a7a-31fde993fafd None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:08:04.117320 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:08:04.120632 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3848bce4-4570-4b90-9a7a-31fde993fafd None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:08:04.122344 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3848bce4-4570-4b90-9a7a-31fde993fafd None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:08:04.135910 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application [None req-3848bce4-4570-4b90-9a7a-31fde993fafd None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 03:08:04.135910 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 03:08:04.135910 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 03:08:04.135910 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 03:08:04.135910 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 03:08:04.135910 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 03:08:04.135910 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 03:08:04.135910 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 03:08:04.135910 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 03:08:04.135910 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 03:08:04.135910 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 03:08:04.135910 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 03:08:04.135910 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 13 03:08:04.135910 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 13 03:08:04.135910 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 13 03:08:04.135910 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 13 03:08:04.136880 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 03:08:04.136880 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 03:08:04.136880 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 03:08:04.136880 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 03:08:04.136880 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 03:08:04.136880 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 03:08:04.136880 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 03:08:04.136880 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application with Lock( Sep 13 03:08:04.136880 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 03:08:04.136880 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._enter() Sep 13 03:08:04.136880 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 03:08:04.136880 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 03:08:04.136880 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 03:08:04.136880 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.creator() Sep 13 03:08:04.136880 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 03:08:04.136880 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application created_value = creator( Sep 13 03:08:04.136880 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 13 03:08:04.137729 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 13 03:08:04.137729 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 03:08:04.137729 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Sep 13 03:08:04.137729 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 261/512] 10.222.0.18 () {56 vars in 1108 bytes} [Tue Sep 13 03:08:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 03:08:04.137729 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3848bce4-4570-4b90-9a7a-31fde993fafd None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:08:04.150128 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ddf0e94-64ee-4312-9570-cc8a9f70fe33 None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:08:04.159230 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:08:04.184634 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcf7ea7a-12e4-4dea-9267-5f41017b29aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:08:04.185241 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcf7ea7a-12e4-4dea-9267-5f41017b29aa None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:08:04.185517 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcf7ea7a-12e4-4dea-9267-5f41017b29aa None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:08:04.185786 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcf7ea7a-12e4-4dea-9267-5f41017b29aa None admin] PATH_INFO: `/v3/users` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:08:04.186509 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcf7ea7a-12e4-4dea-9267-5f41017b29aa None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=81399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 03:08:04.186839 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcf7ea7a-12e4-4dea-9267-5f41017b29aa None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:08:04.188302 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcf7ea7a-12e4-4dea-9267-5f41017b29aa None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:08:04.205310 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 252/513] 10.222.0.18 () {56 vars in 1137 bytes} [Tue Sep 13 03:08:04 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 03:08:04.206087 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcf7ea7a-12e4-4dea-9267-5f41017b29aa None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:08:04.236325 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3848bce4-4570-4b90-9a7a-31fde993fafd None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:08:04.268864 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9291a721-89a8-4307-919f-c7d9d743cd43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:08:04.269483 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9291a721-89a8-4307-919f-c7d9d743cd43 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:08:04.269727 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9291a721-89a8-4307-919f-c7d9d743cd43 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:08:04.270021 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9291a721-89a8-4307-919f-c7d9d743cd43 None admin] PATH_INFO: `/v3/domains` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:08:04.270671 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9291a721-89a8-4307-919f-c7d9d743cd43 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 03:08:04.270967 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9291a721-89a8-4307-919f-c7d9d743cd43 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:08:04.272335 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9291a721-89a8-4307-919f-c7d9d743cd43 None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:08:04.283867 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 262/514] 10.222.0.18 () {56 vars in 1109 bytes} [Tue Sep 13 03:08:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 03:08:04.284730 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9291a721-89a8-4307-919f-c7d9d743cd43 None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:08:04.303274 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcf7ea7a-12e4-4dea-9267-5f41017b29aa None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:08:04.331729 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:08:04.337649 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f435dd56-d88c-4952-a459-5b3fe22096dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:08:04.338274 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f435dd56-d88c-4952-a459-5b3fe22096dc None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:08:04.338548 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f435dd56-d88c-4952-a459-5b3fe22096dc None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:08:04.338846 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f435dd56-d88c-4952-a459-5b3fe22096dc None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:08:04.348601 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f435dd56-d88c-4952-a459-5b3fe22096dc None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:08:04.350050 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f435dd56-d88c-4952-a459-5b3fe22096dc None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:08:04.364072 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:08:04.365234 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application [None req-f435dd56-d88c-4952-a459-5b3fe22096dc None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 03:08:04.365234 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 03:08:04.365234 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 03:08:04.365234 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 03:08:04.365234 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 03:08:04.365234 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 03:08:04.365234 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 03:08:04.365234 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 03:08:04.365234 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 03:08:04.365234 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 03:08:04.365234 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 03:08:04.365234 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 03:08:04.365234 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 13 03:08:04.365234 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 13 03:08:04.365234 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 13 03:08:04.365234 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 13 03:08:04.366200 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 03:08:04.366200 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 03:08:04.366200 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 03:08:04.366200 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 03:08:04.366200 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 03:08:04.366200 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 03:08:04.366200 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 03:08:04.366200 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application with Lock( Sep 13 03:08:04.366200 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 03:08:04.366200 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._enter() Sep 13 03:08:04.366200 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 03:08:04.366200 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 03:08:04.366200 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 03:08:04.366200 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.creator() Sep 13 03:08:04.366200 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 03:08:04.366200 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application created_value = creator( Sep 13 03:08:04.366200 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 13 03:08:04.367411 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 13 03:08:04.367411 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 03:08:04.367411 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Sep 13 03:08:04.367411 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 253/515] 10.222.0.18 () {56 vars in 1108 bytes} [Tue Sep 13 03:08:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 03:08:04.367741 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f435dd56-d88c-4952-a459-5b3fe22096dc None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:08:04.389022 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9291a721-89a8-4307-919f-c7d9d743cd43 None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:08:04.422297 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ded9eb7-3a8c-4c49-a1f9-0b84eab717e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:08:04.422911 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ded9eb7-3a8c-4c49-a1f9-0b84eab717e5 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:08:04.423187 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ded9eb7-3a8c-4c49-a1f9-0b84eab717e5 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:08:04.423516 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ded9eb7-3a8c-4c49-a1f9-0b84eab717e5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:08:04.464478 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f435dd56-d88c-4952-a459-5b3fe22096dc None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:08:04.489140 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ded9eb7-3a8c-4c49-a1f9-0b84eab717e5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:08:04.490578 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ded9eb7-3a8c-4c49-a1f9-0b84eab717e5 None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:08:04.496737 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bb98bca-d9b0-4e46-bf60-a7c4bc3d089e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:08:04.497328 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb98bca-d9b0-4e46-bf60-a7c4bc3d089e None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:08:04.497607 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb98bca-d9b0-4e46-bf60-a7c4bc3d089e None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:08:04.497880 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb98bca-d9b0-4e46-bf60-a7c4bc3d089e None admin] PATH_INFO: `/v3/domains` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:08:04.498511 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bb98bca-d9b0-4e46-bf60-a7c4bc3d089e None admin] RBAC: Adding query filter params (name=Default) {{(pid=81399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 03:08:04.498841 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bb98bca-d9b0-4e46-bf60-a7c4bc3d089e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:08:04.500744 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bb98bca-d9b0-4e46-bf60-a7c4bc3d089e None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:08:04.516807 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 254/516] 10.222.0.18 () {56 vars in 1109 bytes} [Tue Sep 13 03:08:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 03:08:04.516807 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bb98bca-d9b0-4e46-bf60-a7c4bc3d089e None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:08:04.553493 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:08:04.573594 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:08:04.593477 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 263/517] 10.222.0.18 () {58 vars in 1282 bytes} [Tue Sep 13 03:08:04 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 13 03:08:04.594141 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ded9eb7-3a8c-4c49-a1f9-0b84eab717e5 None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:08:04.614375 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bb98bca-d9b0-4e46-bf60-a7c4bc3d089e None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:08:04.646923 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98228622-888a-4ba3-85bc-da17f73479db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:08:04.647524 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98228622-888a-4ba3-85bc-da17f73479db None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:08:04.647804 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98228622-888a-4ba3-85bc-da17f73479db None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:08:04.648077 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98228622-888a-4ba3-85bc-da17f73479db None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:08:04.689632 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ded9eb7-3a8c-4c49-a1f9-0b84eab717e5 None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:08:04.713967 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98228622-888a-4ba3-85bc-da17f73479db None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:08:04.715443 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98228622-888a-4ba3-85bc-da17f73479db None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:08:04.723151 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0023753e-bfe2-4d23-9685-1a5259f7c098 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:08:04.723774 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0023753e-bfe2-4d23-9685-1a5259f7c098 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:08:04.724047 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0023753e-bfe2-4d23-9685-1a5259f7c098 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:08:04.724341 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0023753e-bfe2-4d23-9685-1a5259f7c098 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:08:04.730702 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0023753e-bfe2-4d23-9685-1a5259f7c098 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:08:04.732138 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0023753e-bfe2-4d23-9685-1a5259f7c098 None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:08:04.744896 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application [None req-0023753e-bfe2-4d23-9685-1a5259f7c098 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 13 03:08:04.744896 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 03:08:04.744896 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 03:08:04.744896 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 03:08:04.744896 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 03:08:04.744896 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 03:08:04.744896 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 03:08:04.744896 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 03:08:04.744896 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 03:08:04.744896 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 03:08:04.744896 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 03:08:04.744896 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 03:08:04.744896 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 13 03:08:04.744896 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 13 03:08:04.744896 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 13 03:08:04.745895 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 13 03:08:04.745895 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 03:08:04.745895 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 03:08:04.745895 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 03:08:04.745895 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 03:08:04.745895 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 03:08:04.745895 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 03:08:04.745895 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 03:08:04.745895 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application with Lock( Sep 13 03:08:04.745895 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 03:08:04.745895 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._enter() Sep 13 03:08:04.745895 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 03:08:04.745895 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 03:08:04.745895 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 03:08:04.745895 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.creator() Sep 13 03:08:04.745895 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 03:08:04.745895 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application created_value = creator( Sep 13 03:08:04.747000 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 13 03:08:04.747000 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 13 03:08:04.747000 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 13 03:08:04.747000 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 13 03:08:04.747000 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 13 03:08:04.747000 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 13 03:08:04.747000 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 13 03:08:04.747000 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Sep 13 03:08:04.747000 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 264/518] 10.222.0.18 () {56 vars in 1111 bytes} [Tue Sep 13 03:08:04 2022] GET /identity/v3/projects/service => generated 88 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 03:08:04.747000 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0023753e-bfe2-4d23-9685-1a5259f7c098 None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:08:04.761046 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:08:04.787386 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:08:04.818845 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 255/519] 10.222.0.18 () {58 vars in 1282 bytes} [Tue Sep 13 03:08:04 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 306 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 13 03:08:04.819756 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98228622-888a-4ba3-85bc-da17f73479db None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:08:04.847933 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0023753e-bfe2-4d23-9685-1a5259f7c098 None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:08:04.881247 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6f204a9-5103-4886-aeec-4728e828aca0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:08:04.881967 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f204a9-5103-4886-aeec-4728e828aca0 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:08:04.882242 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f204a9-5103-4886-aeec-4728e828aca0 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:08:04.882511 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f204a9-5103-4886-aeec-4728e828aca0 None admin] PATH_INFO: `/v3/domains` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:08:04.883174 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6f204a9-5103-4886-aeec-4728e828aca0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 03:08:04.883472 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6f204a9-5103-4886-aeec-4728e828aca0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:08:04.884835 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6f204a9-5103-4886-aeec-4728e828aca0 None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:08:04.895651 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 265/520] 10.222.0.18 () {56 vars in 1109 bytes} [Tue Sep 13 03:08:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 03:08:04.896421 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6f204a9-5103-4886-aeec-4728e828aca0 None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:08:04.916490 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98228622-888a-4ba3-85bc-da17f73479db None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:08:04.949711 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c908e69-93fd-4b51-aa04-05a26ca50e06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:08:04.950310 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c908e69-93fd-4b51-aa04-05a26ca50e06 None admin] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:08:04.950585 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c908e69-93fd-4b51-aa04-05a26ca50e06 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:08:04.950885 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c908e69-93fd-4b51-aa04-05a26ca50e06 None admin] PATH_INFO: `/v3/users` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:08:04.951523 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c908e69-93fd-4b51-aa04-05a26ca50e06 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:08:04.952968 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c908e69-93fd-4b51-aa04-05a26ca50e06 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:08:04.968878 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:08:04.985275 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 256/521] 10.222.0.18 () {60 vars in 1132 bytes} [Tue Sep 13 03:08:04 2022] POST /identity/v3/users => generated 247 bytes in 156 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 03:08:04.985275 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c908e69-93fd-4b51-aa04-05a26ca50e06 None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:08:04.998367 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:08:05.003089 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6f204a9-5103-4886-aeec-4728e828aca0 None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:08:05.043189 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83c52343-af9f-47f2-a988-341ee708ccac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:08:05.043826 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83c52343-af9f-47f2-a988-341ee708ccac None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:08:05.044096 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83c52343-af9f-47f2-a988-341ee708ccac None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:08:05.044362 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83c52343-af9f-47f2-a988-341ee708ccac None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:08:05.056592 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83c52343-af9f-47f2-a988-341ee708ccac None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:08:05.061262 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83c52343-af9f-47f2-a988-341ee708ccac None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:08:05.079718 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application [None req-83c52343-af9f-47f2-a988-341ee708ccac None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 03:08:05.079718 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 03:08:05.079718 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 03:08:05.079718 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 03:08:05.079718 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 03:08:05.079718 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 03:08:05.079718 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 03:08:05.079718 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 03:08:05.079718 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 03:08:05.079718 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 03:08:05.079718 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 03:08:05.079718 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 03:08:05.079718 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 13 03:08:05.079718 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 13 03:08:05.079718 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 13 03:08:05.079718 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 13 03:08:05.080836 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 03:08:05.080836 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 03:08:05.080836 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 03:08:05.080836 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 03:08:05.080836 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 03:08:05.080836 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 03:08:05.080836 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 03:08:05.080836 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application with Lock( Sep 13 03:08:05.080836 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 03:08:05.080836 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._enter() Sep 13 03:08:05.080836 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 03:08:05.080836 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 03:08:05.080836 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 03:08:05.080836 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.creator() Sep 13 03:08:05.080836 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 03:08:05.080836 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application created_value = creator( Sep 13 03:08:05.080836 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 13 03:08:05.081777 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 13 03:08:05.081777 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 03:08:05.081777 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Sep 13 03:08:05.081777 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 266/522] 10.222.0.18 () {56 vars in 1108 bytes} [Tue Sep 13 03:08:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 03:08:05.086266 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83c52343-af9f-47f2-a988-341ee708ccac None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:08:05.107192 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c908e69-93fd-4b51-aa04-05a26ca50e06 None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:08:05.146431 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50409ee4-f983-4f07-b854-bb04bb90bd6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:08:05.147075 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50409ee4-f983-4f07-b854-bb04bb90bd6b None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:08:05.147358 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50409ee4-f983-4f07-b854-bb04bb90bd6b None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:08:05.147632 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50409ee4-f983-4f07-b854-bb04bb90bd6b None admin] PATH_INFO: `/v3/projects` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:08:05.148327 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50409ee4-f983-4f07-b854-bb04bb90bd6b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 03:08:05.149161 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50409ee4-f983-4f07-b854-bb04bb90bd6b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:08:05.150055 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50409ee4-f983-4f07-b854-bb04bb90bd6b None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:08:05.169662 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 257/523] 10.222.0.18 () {56 vars in 1148 bytes} [Tue Sep 13 03:08:04 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 03:08:05.169835 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50409ee4-f983-4f07-b854-bb04bb90bd6b None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:08:05.204932 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83c52343-af9f-47f2-a988-341ee708ccac None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:08:05.237403 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:08:05.243760 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd3f38ea-4050-4b7b-b330-afbdd0b40fda None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:08:05.244396 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd3f38ea-4050-4b7b-b330-afbdd0b40fda None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:08:05.244677 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd3f38ea-4050-4b7b-b330-afbdd0b40fda None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:08:05.244950 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd3f38ea-4050-4b7b-b330-afbdd0b40fda None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:08:05.255791 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd3f38ea-4050-4b7b-b330-afbdd0b40fda None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:08:05.257247 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd3f38ea-4050-4b7b-b330-afbdd0b40fda None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:08:05.272147 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application [None req-dd3f38ea-4050-4b7b-b330-afbdd0b40fda None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 03:08:05.272147 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 03:08:05.272147 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 03:08:05.272147 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 03:08:05.272147 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 03:08:05.272147 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 03:08:05.272147 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 03:08:05.272147 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 03:08:05.272147 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 03:08:05.272147 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 03:08:05.272147 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 03:08:05.272147 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 03:08:05.272147 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 13 03:08:05.272147 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 13 03:08:05.272147 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 13 03:08:05.272147 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 13 03:08:05.273481 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 03:08:05.273481 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 03:08:05.273481 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 03:08:05.273481 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 03:08:05.273481 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 03:08:05.273481 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 03:08:05.273481 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 03:08:05.273481 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application with Lock( Sep 13 03:08:05.273481 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 03:08:05.273481 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._enter() Sep 13 03:08:05.273481 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 03:08:05.273481 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 03:08:05.273481 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 03:08:05.273481 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.creator() Sep 13 03:08:05.273481 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 03:08:05.273481 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application created_value = creator( Sep 13 03:08:05.273481 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 13 03:08:05.274341 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 13 03:08:05.274341 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 03:08:05.274341 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Sep 13 03:08:05.274341 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 267/524] 10.222.0.18 () {56 vars in 1108 bytes} [Tue Sep 13 03:08:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 03:08:05.274341 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd3f38ea-4050-4b7b-b330-afbdd0b40fda None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:08:05.282513 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:08:05.294664 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50409ee4-f983-4f07-b854-bb04bb90bd6b None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:08:05.334247 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-236c2e58-9be8-4897-94c5-4455b75ed8ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:08:05.334913 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-236c2e58-9be8-4897-94c5-4455b75ed8ec None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:08:05.335164 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-236c2e58-9be8-4897-94c5-4455b75ed8ec None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:08:05.335431 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-236c2e58-9be8-4897-94c5-4455b75ed8ec None admin] PATH_INFO: `/v3/users/neutron` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:08:05.343862 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-236c2e58-9be8-4897-94c5-4455b75ed8ec None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:08:05.345295 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-236c2e58-9be8-4897-94c5-4455b75ed8ec None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:08:05.357010 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application [None req-236c2e58-9be8-4897-94c5-4455b75ed8ec None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Sep 13 03:08:05.357010 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 03:08:05.357010 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 03:08:05.357010 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 03:08:05.357010 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 03:08:05.357010 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 03:08:05.357010 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 03:08:05.357010 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 03:08:05.357010 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 03:08:05.357010 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 03:08:05.357010 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 03:08:05.357010 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 03:08:05.357010 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 13 03:08:05.357010 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 13 03:08:05.357010 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 13 03:08:05.357010 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 13 03:08:05.358098 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 03:08:05.358098 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 03:08:05.358098 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 13 03:08:05.358098 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 13 03:08:05.358098 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 13 03:08:05.358098 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 13 03:08:05.358098 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 03:08:05.358098 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 03:08:05.358098 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 03:08:05.358098 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 03:08:05.358098 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 03:08:05.358098 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application with Lock( Sep 13 03:08:05.358098 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 03:08:05.358098 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._enter() Sep 13 03:08:05.358098 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 03:08:05.358098 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 03:08:05.358098 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 03:08:05.358098 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.creator() Sep 13 03:08:05.359107 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 03:08:05.359107 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application created_value = creator( Sep 13 03:08:05.359107 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 13 03:08:05.359107 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 13 03:08:05.359107 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 13 03:08:05.359107 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 13 03:08:05.359107 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 13 03:08:05.359107 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 13 03:08:05.359107 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Sep 13 03:08:05.359107 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Sep 13 03:08:05.359107 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 258/525] 10.222.0.18 () {56 vars in 1102 bytes} [Tue Sep 13 03:08:05 2022] GET /identity/v3/users/neutron => generated 85 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 03:08:05.359681 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-236c2e58-9be8-4897-94c5-4455b75ed8ec None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:08:05.391060 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd3f38ea-4050-4b7b-b330-afbdd0b40fda None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:08:05.428897 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ce1399c-8934-4cab-b454-548b9c90ab2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:08:05.429594 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ce1399c-8934-4cab-b454-548b9c90ab2b None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:08:05.430195 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ce1399c-8934-4cab-b454-548b9c90ab2b None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:08:05.430561 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ce1399c-8934-4cab-b454-548b9c90ab2b None admin] PATH_INFO: `/v3/domains` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:08:05.431246 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ce1399c-8934-4cab-b454-548b9c90ab2b None admin] RBAC: Adding query filter params (name=Default) {{(pid=81400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 03:08:05.431553 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ce1399c-8934-4cab-b454-548b9c90ab2b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:08:05.432952 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ce1399c-8934-4cab-b454-548b9c90ab2b None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:08:05.446889 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 268/526] 10.222.0.18 () {56 vars in 1109 bytes} [Tue Sep 13 03:08:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 03:08:05.447722 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ce1399c-8934-4cab-b454-548b9c90ab2b None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:08:05.474703 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-236c2e58-9be8-4897-94c5-4455b75ed8ec None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:08:05.495289 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:08:05.510871 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:08:05.515689 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf64c556-384f-4d3d-b750-39875aeee1fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:08:05.516481 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf64c556-384f-4d3d-b750-39875aeee1fe None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:08:05.517136 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf64c556-384f-4d3d-b750-39875aeee1fe None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:08:05.517136 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf64c556-384f-4d3d-b750-39875aeee1fe None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:08:05.517742 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf64c556-384f-4d3d-b750-39875aeee1fe None admin] RBAC: Adding query filter params (role.id=817d6837cd044890b41cf16b303a092b, scope.project.id=b9d94f6a68f7470d9494f112175620dc, user.id=c7377f5bee4a49959182e1bd34bc97a4) {{(pid=81399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 03:08:05.518049 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf64c556-384f-4d3d-b750-39875aeee1fe None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:08:05.519505 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf64c556-384f-4d3d-b750-39875aeee1fe None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:08:05.525081 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 259/527] 10.222.0.18 () {56 vars in 1374 bytes} [Tue Sep 13 03:08:05 2022] GET /identity/v3/role_assignments?user.id=c7377f5bee4a49959182e1bd34bc97a4&scope.project.id=b9d94f6a68f7470d9494f112175620dc&role.id=817d6837cd044890b41cf16b303a092b => generated 259 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 03:08:05.535999 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf64c556-384f-4d3d-b750-39875aeee1fe None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:08:05.554599 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ce1399c-8934-4cab-b454-548b9c90ab2b None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:08:05.602458 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9990fa64-e522-4512-aa2f-f87bd468726d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:08:05.604227 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9990fa64-e522-4512-aa2f-f87bd468726d None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:08:05.604781 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9990fa64-e522-4512-aa2f-f87bd468726d None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:08:05.605500 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9990fa64-e522-4512-aa2f-f87bd468726d None admin] PATH_INFO: `/v3/domains` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:08:05.606376 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9990fa64-e522-4512-aa2f-f87bd468726d None admin] RBAC: Adding query filter params (name=Default) {{(pid=81400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 03:08:05.607132 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9990fa64-e522-4512-aa2f-f87bd468726d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:08:05.609319 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9990fa64-e522-4512-aa2f-f87bd468726d None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:08:05.628070 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 269/528] 10.222.0.18 () {56 vars in 1109 bytes} [Tue Sep 13 03:08:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 03:08:05.629529 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9990fa64-e522-4512-aa2f-f87bd468726d None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:08:05.683192 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf64c556-384f-4d3d-b750-39875aeee1fe None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:08:05.733191 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f11c111-b9ec-4b5a-a111-4829accab8f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:08:05.734523 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f11c111-b9ec-4b5a-a111-4829accab8f0 None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:08:05.734886 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f11c111-b9ec-4b5a-a111-4829accab8f0 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:08:05.735271 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f11c111-b9ec-4b5a-a111-4829accab8f0 None admin] PATH_INFO: `/v3/users` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:08:05.736064 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f11c111-b9ec-4b5a-a111-4829accab8f0 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=81399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 03:08:05.736478 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f11c111-b9ec-4b5a-a111-4829accab8f0 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:08:05.737976 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f11c111-b9ec-4b5a-a111-4829accab8f0 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:08:05.746650 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9990fa64-e522-4512-aa2f-f87bd468726d None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:08:05.757358 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 260/529] 10.222.0.18 () {56 vars in 1139 bytes} [Tue Sep 13 03:08:05 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 03:08:05.758002 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f11c111-b9ec-4b5a-a111-4829accab8f0 None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:08:05.777211 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:08:05.790276 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf1c1756-41cd-4cbd-8e74-e73a41e5ca19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:08:05.791201 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf1c1756-41cd-4cbd-8e74-e73a41e5ca19 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:08:05.791753 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf1c1756-41cd-4cbd-8e74-e73a41e5ca19 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:08:05.792343 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf1c1756-41cd-4cbd-8e74-e73a41e5ca19 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:08:05.799846 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf1c1756-41cd-4cbd-8e74-e73a41e5ca19 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:08:05.801687 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf1c1756-41cd-4cbd-8e74-e73a41e5ca19 None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:08:05.804006 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:08:05.816157 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application [None req-bf1c1756-41cd-4cbd-8e74-e73a41e5ca19 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 13 03:08:05.816157 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 03:08:05.816157 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 03:08:05.816157 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 03:08:05.816157 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 03:08:05.816157 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 03:08:05.816157 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 03:08:05.816157 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 03:08:05.816157 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 03:08:05.816157 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 03:08:05.816157 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 03:08:05.816157 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 03:08:05.816157 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 13 03:08:05.816157 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 13 03:08:05.816157 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 13 03:08:05.817049 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 13 03:08:05.817049 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 03:08:05.817049 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 03:08:05.817049 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 03:08:05.817049 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 03:08:05.817049 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 03:08:05.817049 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 03:08:05.817049 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 03:08:05.817049 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application with Lock( Sep 13 03:08:05.817049 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 03:08:05.817049 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._enter() Sep 13 03:08:05.817049 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 03:08:05.817049 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 03:08:05.817049 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 03:08:05.817049 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.creator() Sep 13 03:08:05.817049 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 03:08:05.817049 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application created_value = creator( Sep 13 03:08:05.817908 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 13 03:08:05.817908 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 13 03:08:05.817908 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 13 03:08:05.817908 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 13 03:08:05.817908 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 13 03:08:05.817908 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 13 03:08:05.817908 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 13 03:08:05.817908 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Sep 13 03:08:05.820506 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 270/530] 10.222.0.18 () {56 vars in 1111 bytes} [Tue Sep 13 03:08:05 2022] GET /identity/v3/projects/service => generated 88 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 03:08:05.821600 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf1c1756-41cd-4cbd-8e74-e73a41e5ca19 None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:08:05.873386 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f11c111-b9ec-4b5a-a111-4829accab8f0 None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:08:05.911695 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08ebae78-e2f4-449f-94de-02961106e597 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:08:05.912380 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ebae78-e2f4-449f-94de-02961106e597 None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:08:05.912827 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ebae78-e2f4-449f-94de-02961106e597 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:08:05.913101 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ebae78-e2f4-449f-94de-02961106e597 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:08:05.921362 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08ebae78-e2f4-449f-94de-02961106e597 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:08:05.923029 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08ebae78-e2f4-449f-94de-02961106e597 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:08:05.932661 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf1c1756-41cd-4cbd-8e74-e73a41e5ca19 None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:08:05.936195 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application [None req-08ebae78-e2f4-449f-94de-02961106e597 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 13 03:08:05.936195 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 03:08:05.936195 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 03:08:05.936195 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 03:08:05.936195 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 03:08:05.936195 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 03:08:05.936195 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 03:08:05.936195 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 03:08:05.936195 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 03:08:05.936195 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 03:08:05.936195 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 03:08:05.936195 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 03:08:05.936195 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 13 03:08:05.936195 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 13 03:08:05.936195 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 13 03:08:05.937068 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 13 03:08:05.937068 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 03:08:05.937068 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 03:08:05.937068 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 03:08:05.937068 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 03:08:05.937068 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 03:08:05.937068 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 03:08:05.937068 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 03:08:05.937068 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application with Lock( Sep 13 03:08:05.937068 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 03:08:05.937068 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._enter() Sep 13 03:08:05.937068 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 03:08:05.937068 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 03:08:05.937068 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 03:08:05.937068 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.creator() Sep 13 03:08:05.937068 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 03:08:05.937068 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application created_value = creator( Sep 13 03:08:05.937958 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 13 03:08:05.937958 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 13 03:08:05.937958 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 13 03:08:05.937958 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 13 03:08:05.937958 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 13 03:08:05.937958 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 13 03:08:05.937958 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 13 03:08:05.937958 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Sep 13 03:08:05.942125 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 261/531] 10.222.0.18 () {56 vars in 1111 bytes} [Tue Sep 13 03:08:05 2022] GET /identity/v3/projects/service => generated 88 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 03:08:05.942964 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08ebae78-e2f4-449f-94de-02961106e597 None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:08:05.969208 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34d673de-d840-4adf-8280-7a1e12f067c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:08:05.969932 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d673de-d840-4adf-8280-7a1e12f067c8 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:08:05.970231 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d673de-d840-4adf-8280-7a1e12f067c8 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:08:05.970574 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d673de-d840-4adf-8280-7a1e12f067c8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:08:06.027281 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:08:06.045117 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34d673de-d840-4adf-8280-7a1e12f067c8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:08:06.046757 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34d673de-d840-4adf-8280-7a1e12f067c8 None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:08:06.051959 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:08:06.056093 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08ebae78-e2f4-449f-94de-02961106e597 None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:08:06.097896 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d8d13ba-ae80-450f-a898-9364788d266c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:08:06.098756 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d8d13ba-ae80-450f-a898-9364788d266c None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:08:06.099265 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d8d13ba-ae80-450f-a898-9364788d266c None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:08:06.099690 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d8d13ba-ae80-450f-a898-9364788d266c None admin] PATH_INFO: `/v3/projects` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:08:06.100518 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d8d13ba-ae80-450f-a898-9364788d266c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 03:08:06.100972 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d8d13ba-ae80-450f-a898-9364788d266c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:08:06.102793 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d8d13ba-ae80-450f-a898-9364788d266c None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:08:06.120888 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 262/532] 10.222.0.18 () {56 vars in 1148 bytes} [Tue Sep 13 03:08:05 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 03:08:06.122148 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d8d13ba-ae80-450f-a898-9364788d266c None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:08:06.164900 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 271/533] 10.222.0.18 () {58 vars in 1282 bytes} [Tue Sep 13 03:08:05 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 343 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 13 03:08:06.166103 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34d673de-d840-4adf-8280-7a1e12f067c8 None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:08:06.230141 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d8d13ba-ae80-450f-a898-9364788d266c None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:08:06.255640 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:08:06.269995 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a2e0bda-5d64-449e-9f41-a2142fd69718 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:08:06.271031 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a2e0bda-5d64-449e-9f41-a2142fd69718 None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:08:06.271598 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a2e0bda-5d64-449e-9f41-a2142fd69718 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:08:06.272038 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a2e0bda-5d64-449e-9f41-a2142fd69718 None admin] PATH_INFO: `/v3/projects` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:08:06.272898 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a2e0bda-5d64-449e-9f41-a2142fd69718 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 03:08:06.273368 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a2e0bda-5d64-449e-9f41-a2142fd69718 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:08:06.274416 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34d673de-d840-4adf-8280-7a1e12f067c8 None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:08:06.276025 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a2e0bda-5d64-449e-9f41-a2142fd69718 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:08:06.293447 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 263/534] 10.222.0.18 () {56 vars in 1148 bytes} [Tue Sep 13 03:08:06 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 03:08:06.294586 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a2e0bda-5d64-449e-9f41-a2142fd69718 None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:08:06.311705 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3fc422e-3d49-4b20-a579-437c5dfe1ece None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:08:06.312618 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3fc422e-3d49-4b20-a579-437c5dfe1ece None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:08:06.312949 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3fc422e-3d49-4b20-a579-437c5dfe1ece None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:08:06.313224 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3fc422e-3d49-4b20-a579-437c5dfe1ece None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:08:06.313944 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3fc422e-3d49-4b20-a579-437c5dfe1ece None admin] RBAC: Adding query filter params (role.id=817d6837cd044890b41cf16b303a092b, scope.project.id=b9d94f6a68f7470d9494f112175620dc, user.id=b5fdc2ff46a14638b409a13e73ba5500) {{(pid=81400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 03:08:06.314258 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3fc422e-3d49-4b20-a579-437c5dfe1ece None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:08:06.315827 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3fc422e-3d49-4b20-a579-437c5dfe1ece None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:08:06.320137 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:08:06.322201 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 272/535] 10.222.0.18 () {56 vars in 1374 bytes} [Tue Sep 13 03:08:06 2022] GET /identity/v3/role_assignments?user.id=b5fdc2ff46a14638b409a13e73ba5500&scope.project.id=b9d94f6a68f7470d9494f112175620dc&role.id=817d6837cd044890b41cf16b303a092b => generated 259 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 03:08:06.325594 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3fc422e-3d49-4b20-a579-437c5dfe1ece None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:08:06.480142 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3fc422e-3d49-4b20-a579-437c5dfe1ece None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:08:06.494044 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a2e0bda-5d64-449e-9f41-a2142fd69718 None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:08:06.532749 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdd1b09a-333c-428d-af16-1363b9019ead None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:08:06.533362 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd1b09a-333c-428d-af16-1363b9019ead None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:08:06.533639 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd1b09a-333c-428d-af16-1363b9019ead None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:08:06.533910 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd1b09a-333c-428d-af16-1363b9019ead None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:08:06.538691 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdd1b09a-333c-428d-af16-1363b9019ead None admin] RBAC: Adding query filter params (role.id=817d6837cd044890b41cf16b303a092b, scope.project.id=b9d94f6a68f7470d9494f112175620dc, user.id=924ee3f608d34a18b0ddc4695f102a56) {{(pid=81400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 03:08:06.539661 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdd1b09a-333c-428d-af16-1363b9019ead None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:08:06.541293 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdd1b09a-333c-428d-af16-1363b9019ead None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:08:06.547864 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfd80015-62b2-4d33-8f59-0ba6d2b7c780 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:08:06.549130 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd80015-62b2-4d33-8f59-0ba6d2b7c780 None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:08:06.549584 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd80015-62b2-4d33-8f59-0ba6d2b7c780 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:08:06.550037 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd80015-62b2-4d33-8f59-0ba6d2b7c780 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:08:06.556533 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 273/536] 10.222.0.18 () {56 vars in 1374 bytes} [Tue Sep 13 03:08:06 2022] GET /identity/v3/role_assignments?user.id=924ee3f608d34a18b0ddc4695f102a56&scope.project.id=b9d94f6a68f7470d9494f112175620dc&role.id=817d6837cd044890b41cf16b303a092b => generated 259 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 03:08:06.566919 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfd80015-62b2-4d33-8f59-0ba6d2b7c780 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:08:06.568749 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfd80015-62b2-4d33-8f59-0ba6d2b7c780 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:08:06.585221 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application [None req-bfd80015-62b2-4d33-8f59-0ba6d2b7c780 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 03:08:06.585221 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 03:08:06.585221 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 03:08:06.585221 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 03:08:06.585221 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 03:08:06.585221 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 03:08:06.585221 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 03:08:06.585221 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 03:08:06.585221 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 03:08:06.585221 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 03:08:06.585221 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 03:08:06.585221 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 03:08:06.585221 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 13 03:08:06.585221 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 13 03:08:06.585221 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 13 03:08:06.585221 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 13 03:08:06.586171 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 03:08:06.586171 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 03:08:06.586171 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 03:08:06.586171 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 03:08:06.586171 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 03:08:06.586171 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 03:08:06.586171 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 03:08:06.586171 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application with Lock( Sep 13 03:08:06.586171 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 03:08:06.586171 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._enter() Sep 13 03:08:06.586171 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 03:08:06.586171 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 03:08:06.586171 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 03:08:06.586171 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.creator() Sep 13 03:08:06.586171 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 03:08:06.586171 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application created_value = creator( Sep 13 03:08:06.586171 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 13 03:08:06.587163 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 13 03:08:06.587163 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 03:08:06.587163 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Sep 13 03:08:06.588763 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 264/537] 10.222.0.18 () {56 vars in 1108 bytes} [Tue Sep 13 03:08:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 295 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 03:08:06.594713 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdd1b09a-333c-428d-af16-1363b9019ead None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:08:06.613973 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:08:06.731181 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdd1b09a-333c-428d-af16-1363b9019ead None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:08:06.784748 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f03b6f71-9cab-4d0f-b7ff-3bcf86d74245 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:08:06.785363 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f03b6f71-9cab-4d0f-b7ff-3bcf86d74245 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:08:06.785638 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f03b6f71-9cab-4d0f-b7ff-3bcf86d74245 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:08:06.785907 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f03b6f71-9cab-4d0f-b7ff-3bcf86d74245 None admin] PATH_INFO: `/v3/domains` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:08:06.786551 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f03b6f71-9cab-4d0f-b7ff-3bcf86d74245 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 03:08:06.786883 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f03b6f71-9cab-4d0f-b7ff-3bcf86d74245 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:08:06.788289 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f03b6f71-9cab-4d0f-b7ff-3bcf86d74245 None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:08:06.803028 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 274/538] 10.222.0.18 () {56 vars in 1109 bytes} [Tue Sep 13 03:08:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 03:08:06.808389 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfd80015-62b2-4d33-8f59-0ba6d2b7c780 None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:08:06.892261 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:08:06.926916 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfd80015-62b2-4d33-8f59-0ba6d2b7c780 None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:08:06.961208 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e545392-ba1f-4e15-a6d4-08e403a95bce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:08:06.962153 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e545392-ba1f-4e15-a6d4-08e403a95bce None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:08:06.962672 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e545392-ba1f-4e15-a6d4-08e403a95bce None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:08:06.963113 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e545392-ba1f-4e15-a6d4-08e403a95bce None admin] PATH_INFO: `/v3/projects/service` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:08:06.970772 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e545392-ba1f-4e15-a6d4-08e403a95bce None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:08:06.972433 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e545392-ba1f-4e15-a6d4-08e403a95bce None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:08:06.985090 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application [None req-6e545392-ba1f-4e15-a6d4-08e403a95bce None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 13 03:08:06.985090 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 03:08:06.985090 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 03:08:06.985090 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 03:08:06.985090 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 03:08:06.985090 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 03:08:06.985090 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 03:08:06.985090 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 03:08:06.985090 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 03:08:06.985090 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 03:08:06.985090 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 03:08:06.985090 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 03:08:06.985090 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 13 03:08:06.985090 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 13 03:08:06.985090 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 13 03:08:06.985957 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 13 03:08:06.985957 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 03:08:06.985957 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 03:08:06.985957 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 03:08:06.985957 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 03:08:06.985957 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 03:08:06.985957 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 03:08:06.985957 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 03:08:06.985957 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application with Lock( Sep 13 03:08:06.985957 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 03:08:06.985957 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._enter() Sep 13 03:08:06.985957 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 03:08:06.985957 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 03:08:06.985957 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 03:08:06.985957 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.creator() Sep 13 03:08:06.985957 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 03:08:06.985957 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application created_value = creator( Sep 13 03:08:06.986824 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 13 03:08:06.986824 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 13 03:08:06.986824 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 13 03:08:06.986824 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 13 03:08:06.986824 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 13 03:08:06.986824 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 13 03:08:06.986824 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 13 03:08:06.986824 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Sep 13 03:08:06.988882 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 265/539] 10.222.0.18 () {56 vars in 1111 bytes} [Tue Sep 13 03:08:06 2022] GET /identity/v3/projects/service => generated 88 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 03:08:06.995690 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f03b6f71-9cab-4d0f-b7ff-3bcf86d74245 None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:08:07.103706 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:08:07.145386 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f03b6f71-9cab-4d0f-b7ff-3bcf86d74245 None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:08:07.196357 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b41ac26-a8c1-4f80-a2e8-63194be2df08 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:08:07.196965 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b41ac26-a8c1-4f80-a2e8-63194be2df08 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:08:07.197240 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b41ac26-a8c1-4f80-a2e8-63194be2df08 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:08:07.197509 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b41ac26-a8c1-4f80-a2e8-63194be2df08 None admin] PATH_INFO: `/v3/projects` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:08:07.198198 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b41ac26-a8c1-4f80-a2e8-63194be2df08 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 03:08:07.198498 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b41ac26-a8c1-4f80-a2e8-63194be2df08 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:08:07.199956 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b41ac26-a8c1-4f80-a2e8-63194be2df08 None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:08:07.222272 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 275/540] 10.222.0.18 () {56 vars in 1148 bytes} [Tue Sep 13 03:08:06 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 03:08:07.228320 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e545392-ba1f-4e15-a6d4-08e403a95bce None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:08:07.367105 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e545392-ba1f-4e15-a6d4-08e403a95bce None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:08:07.411679 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:08:07.416691 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ccaae6b-10aa-482d-a4d7-0d41e4c93817 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:08:07.417310 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ccaae6b-10aa-482d-a4d7-0d41e4c93817 None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:08:07.417590 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ccaae6b-10aa-482d-a4d7-0d41e4c93817 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:08:07.417862 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ccaae6b-10aa-482d-a4d7-0d41e4c93817 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:08:07.418597 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ccaae6b-10aa-482d-a4d7-0d41e4c93817 None admin] RBAC: Adding query filter params (role.id=817d6837cd044890b41cf16b303a092b, scope.project.id=b9d94f6a68f7470d9494f112175620dc, user.id=cdffbc851086441ebd01022d1e4c2609) {{(pid=81399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 03:08:07.418956 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ccaae6b-10aa-482d-a4d7-0d41e4c93817 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:08:07.420497 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ccaae6b-10aa-482d-a4d7-0d41e4c93817 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:08:07.433511 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 266/541] 10.222.0.18 () {56 vars in 1374 bytes} [Tue Sep 13 03:08:07 2022] GET /identity/v3/role_assignments?user.id=cdffbc851086441ebd01022d1e4c2609&scope.project.id=b9d94f6a68f7470d9494f112175620dc&role.id=817d6837cd044890b41cf16b303a092b => generated 259 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 03:08:07.453329 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd25b2a9-e80c-4d51-b644-2e3dd0c1ca3b None None] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:08:07.454208 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd25b2a9-e80c-4d51-b644-2e3dd0c1ca3b None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:08:07.454697 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd25b2a9-e80c-4d51-b644-2e3dd0c1ca3b None None] PATH_INFO: `/` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:08:07.456473 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 276/542] 10.222.0.18 () {54 vars in 927 bytes} [Tue Sep 13 03:08:07 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 03:08:07.462078 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7002064c-874c-483d-be45-a42d6aaa8db0 None None] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:08:07.462672 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7002064c-874c-483d-be45-a42d6aaa8db0 None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:08:07.463120 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7002064c-874c-483d-be45-a42d6aaa8db0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:08:07.498238 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.auth.core [None req-7002064c-874c-483d-be45-a42d6aaa8db0 None None] MFA Rules not processed for user `49f280ab086b4610855c4ffa378517d9`. Rule list: `[]` (Enabled: `True`). {{(pid=81399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 03:08:07.500316 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-7002064c-874c-483d-be45-a42d6aaa8db0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:08:07.640447 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 267/543] 10.222.0.18 () {58 vars in 1024 bytes} [Tue Sep 13 03:08:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 13 03:08:07.976453 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc0711a-4af7-4616-b007-c8c088c87868 None None] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:08:07.976737 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc0711a-4af7-4616-b007-c8c088c87868 None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:08:07.977220 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc0711a-4af7-4616-b007-c8c088c87868 None None] PATH_INFO: `/` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:08:07.978916 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 277/544] 10.222.0.18 () {54 vars in 927 bytes} [Tue Sep 13 03:08:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 03:08:07.984396 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a065014a-7dd9-4aa5-8f76-ba2797882916 None None] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:08:07.984757 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a065014a-7dd9-4aa5-8f76-ba2797882916 None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:08:07.985102 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a065014a-7dd9-4aa5-8f76-ba2797882916 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:08:08.002892 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dc0711a-4af7-4616-b007-c8c088c87868 None None] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:08:08.004376 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.auth.core [None req-a065014a-7dd9-4aa5-8f76-ba2797882916 None None] MFA Rules not processed for user `49f280ab086b4610855c4ffa378517d9`. Rule list: `[]` (Enabled: `True`). {{(pid=81399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 03:08:08.005556 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-a065014a-7dd9-4aa5-8f76-ba2797882916 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:08:08.101332 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 268/545] 10.222.0.18 () {58 vars in 1024 bytes} [Tue Sep 13 03:08:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 13 03:08:08.143452 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dc0711a-4af7-4616-b007-c8c088c87868 None None] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:08:08.187632 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:08:08.191607 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cb86384-aad3-43e2-b99c-8cf6fdc5e564 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:08:08.192665 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cb86384-aad3-43e2-b99c-8cf6fdc5e564 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:08:08.193247 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cb86384-aad3-43e2-b99c-8cf6fdc5e564 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:08:08.193960 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cb86384-aad3-43e2-b99c-8cf6fdc5e564 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:08:08.207169 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cb86384-aad3-43e2-b99c-8cf6fdc5e564 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 03:08:08.212163 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application [None req-8cb86384-aad3-43e2-b99c-8cf6fdc5e564 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 13 03:08:08.212163 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 03:08:08.212163 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 03:08:08.212163 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 03:08:08.212163 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 03:08:08.212163 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 03:08:08.212163 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 03:08:08.212163 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 03:08:08.212163 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 03:08:08.212163 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 03:08:08.212163 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 03:08:08.212163 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 03:08:08.212163 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 03:08:08.212163 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 03:08:08.212163 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 03:08:08.212163 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 03:08:08.213045 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 03:08:08.213045 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 03:08:08.213045 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 03:08:08.213045 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 03:08:08.213045 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 03:08:08.213045 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 03:08:08.213045 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 03:08:08.213045 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 03:08:08.213045 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 03:08:08.213045 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 03:08:08.213045 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 03:08:08.213045 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application with Lock( Sep 13 03:08:08.213045 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 03:08:08.213045 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._enter() Sep 13 03:08:08.213045 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 03:08:08.213045 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 03:08:08.213835 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 03:08:08.213835 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.creator() Sep 13 03:08:08.213835 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 03:08:08.213835 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application created_value = creator( Sep 13 03:08:08.213835 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 03:08:08.213835 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 03:08:08.213835 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 03:08:08.213835 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 03:08:08.213835 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 03:08:08.213835 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 03:08:08.213835 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 13 03:08:08.213835 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Sep 13 03:08:08.213835 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 278/546] 10.222.0.18 () {56 vars in 1102 bytes} [Tue Sep 13 03:08:08 2022] GET /identity/v3/roles/service => generated 85 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 03:08:08.219264 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a065014a-7dd9-4aa5-8f76-ba2797882916 None None] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:08:08.298176 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:08:08.350860 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a065014a-7dd9-4aa5-8f76-ba2797882916 None None] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:08:08.391188 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24292cd0-063d-40c4-8f3b-26fc5acedf0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:08:08.391806 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24292cd0-063d-40c4-8f3b-26fc5acedf0e None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:08:08.392084 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24292cd0-063d-40c4-8f3b-26fc5acedf0e None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:08:08.392356 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24292cd0-063d-40c4-8f3b-26fc5acedf0e None admin] PATH_INFO: `/v3/roles` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:08:08.393032 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24292cd0-063d-40c4-8f3b-26fc5acedf0e None admin] RBAC: Adding query filter params (name=service) {{(pid=81399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 03:08:08.393333 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24292cd0-063d-40c4-8f3b-26fc5acedf0e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:08:08.394821 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24292cd0-063d-40c4-8f3b-26fc5acedf0e None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:08:08.403522 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 269/547] 10.222.0.18 () {56 vars in 1103 bytes} [Tue Sep 13 03:08:08 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 03:08:08.409770 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cb86384-aad3-43e2-b99c-8cf6fdc5e564 None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:08:08.504690 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24292cd0-063d-40c4-8f3b-26fc5acedf0e None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:08:08.538498 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cb86384-aad3-43e2-b99c-8cf6fdc5e564 None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:08:08.578886 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-396d3281-e376-47fe-aa27-160e0402b1d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:08:08.579429 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-396d3281-e376-47fe-aa27-160e0402b1d6 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:08:08.579706 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-396d3281-e376-47fe-aa27-160e0402b1d6 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:08:08.579976 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-396d3281-e376-47fe-aa27-160e0402b1d6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:08:08.603003 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-396d3281-e376-47fe-aa27-160e0402b1d6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:08:08.607980 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-396d3281-e376-47fe-aa27-160e0402b1d6 None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:08:08.633637 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application [None req-396d3281-e376-47fe-aa27-160e0402b1d6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 03:08:08.633637 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 03:08:08.633637 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 03:08:08.633637 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 03:08:08.633637 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 03:08:08.633637 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 03:08:08.633637 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 03:08:08.633637 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 03:08:08.633637 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 03:08:08.633637 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 03:08:08.633637 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 03:08:08.633637 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 03:08:08.633637 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 13 03:08:08.633637 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 13 03:08:08.633637 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 13 03:08:08.633637 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 13 03:08:08.634560 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 03:08:08.634560 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 03:08:08.634560 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 03:08:08.634560 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 03:08:08.634560 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 03:08:08.634560 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 03:08:08.634560 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 03:08:08.634560 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application with Lock( Sep 13 03:08:08.634560 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 03:08:08.634560 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._enter() Sep 13 03:08:08.634560 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 03:08:08.634560 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 03:08:08.634560 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 03:08:08.634560 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.creator() Sep 13 03:08:08.634560 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 03:08:08.634560 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application created_value = creator( Sep 13 03:08:08.634560 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 13 03:08:08.635457 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 13 03:08:08.635457 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 03:08:08.635457 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Sep 13 03:08:08.638160 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 279/548] 10.222.0.18 () {56 vars in 1108 bytes} [Tue Sep 13 03:08:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 229 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 03:08:08.645705 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-396d3281-e376-47fe-aa27-160e0402b1d6 None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:08:08.712466 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24292cd0-063d-40c4-8f3b-26fc5acedf0e None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:08:08.746973 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:08:08.751065 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:08:08.774241 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92600abc-da24-43ca-9d96-68df363e773b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:08:08.774905 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92600abc-da24-43ca-9d96-68df363e773b None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:08:08.775178 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92600abc-da24-43ca-9d96-68df363e773b None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:08:08.775452 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92600abc-da24-43ca-9d96-68df363e773b None admin] PATH_INFO: `/v3/roles/service` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:08:08.786813 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92600abc-da24-43ca-9d96-68df363e773b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 03:08:08.792337 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application [None req-92600abc-da24-43ca-9d96-68df363e773b None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 13 03:08:08.792337 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 03:08:08.792337 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 03:08:08.792337 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 03:08:08.792337 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 03:08:08.792337 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 03:08:08.792337 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 03:08:08.792337 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 03:08:08.792337 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 03:08:08.792337 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 03:08:08.792337 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 03:08:08.792337 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 03:08:08.792337 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 03:08:08.792337 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 03:08:08.792337 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 03:08:08.792337 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 03:08:08.793792 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 03:08:08.793792 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 03:08:08.793792 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 03:08:08.793792 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 03:08:08.793792 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 03:08:08.793792 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 03:08:08.793792 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 03:08:08.793792 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 03:08:08.793792 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 03:08:08.793792 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 03:08:08.793792 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 03:08:08.793792 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application with Lock( Sep 13 03:08:08.793792 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 03:08:08.793792 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._enter() Sep 13 03:08:08.793792 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 03:08:08.793792 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 03:08:08.794824 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 03:08:08.794824 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.creator() Sep 13 03:08:08.794824 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 03:08:08.794824 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application created_value = creator( Sep 13 03:08:08.794824 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 03:08:08.794824 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 03:08:08.794824 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 03:08:08.794824 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 03:08:08.794824 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 03:08:08.794824 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 03:08:08.794824 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 13 03:08:08.794824 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Sep 13 03:08:08.794824 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 270/549] 10.222.0.18 () {56 vars in 1102 bytes} [Tue Sep 13 03:08:08 2022] GET /identity/v3/roles/service => generated 85 bytes in 289 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 13 03:08:08.802639 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92600abc-da24-43ca-9d96-68df363e773b None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:08:08.903895 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-396d3281-e376-47fe-aa27-160e0402b1d6 None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:08:08.912751 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92600abc-da24-43ca-9d96-68df363e773b None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:08:08.949966 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87e481be-7c97-46d4-8b26-4d11b5c88495 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:08:08.950596 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e481be-7c97-46d4-8b26-4d11b5c88495 None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:08:08.950925 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e481be-7c97-46d4-8b26-4d11b5c88495 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:08:08.951199 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e481be-7c97-46d4-8b26-4d11b5c88495 None admin] PATH_INFO: `/v3/roles` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:08:08.951911 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87e481be-7c97-46d4-8b26-4d11b5c88495 None admin] RBAC: Adding query filter params (name=service) {{(pid=81399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 03:08:08.952224 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87e481be-7c97-46d4-8b26-4d11b5c88495 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:08:08.953626 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87e481be-7c97-46d4-8b26-4d11b5c88495 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:08:08.962490 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 271/550] 10.222.0.18 () {56 vars in 1103 bytes} [Tue Sep 13 03:08:08 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 03:08:08.968282 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87e481be-7c97-46d4-8b26-4d11b5c88495 None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:08:08.988199 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4f8b55f-cd06-430c-b863-5eb527d1e3c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:08:08.988858 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f8b55f-cd06-430c-b863-5eb527d1e3c5 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:08:08.989137 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f8b55f-cd06-430c-b863-5eb527d1e3c5 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:08:08.989404 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f8b55f-cd06-430c-b863-5eb527d1e3c5 None admin] PATH_INFO: `/v3/domains` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:08:08.990044 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4f8b55f-cd06-430c-b863-5eb527d1e3c5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 03:08:08.990347 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4f8b55f-cd06-430c-b863-5eb527d1e3c5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:08:08.991919 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4f8b55f-cd06-430c-b863-5eb527d1e3c5 None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:08:09.005867 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 280/551] 10.222.0.18 () {56 vars in 1109 bytes} [Tue Sep 13 03:08:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 360 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 03:08:09.007000 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4374aab4-dda8-4fce-bcca-9063feea5b7b None None] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:08:09.007375 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4374aab4-dda8-4fce-bcca-9063feea5b7b None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:08:09.007650 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4374aab4-dda8-4fce-bcca-9063feea5b7b None None] PATH_INFO: `/` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:08:09.013076 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 281/552] 10.222.0.18 () {54 vars in 927 bytes} [Tue Sep 13 03:08:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 03:08:09.013845 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4374aab4-dda8-4fce-bcca-9063feea5b7b None None] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:08:09.019173 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:08:09.088391 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87e481be-7c97-46d4-8b26-4d11b5c88495 None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:08:09.124957 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e5e890c-ac59-49cc-80db-3444f2682aec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:08:09.125596 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e5e890c-ac59-49cc-80db-3444f2682aec None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:08:09.125876 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e5e890c-ac59-49cc-80db-3444f2682aec None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:08:09.126142 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e5e890c-ac59-49cc-80db-3444f2682aec None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:08:09.137080 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4374aab4-dda8-4fce-bcca-9063feea5b7b None None] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:08:09.138307 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e5e890c-ac59-49cc-80db-3444f2682aec None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:08:09.139799 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e5e890c-ac59-49cc-80db-3444f2682aec None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:08:09.140372 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:08:09.154725 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application [None req-6e5e890c-ac59-49cc-80db-3444f2682aec None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 03:08:09.154725 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 03:08:09.154725 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 03:08:09.154725 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 03:08:09.154725 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 03:08:09.154725 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 03:08:09.154725 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 03:08:09.154725 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 03:08:09.154725 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 03:08:09.154725 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 03:08:09.154725 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 03:08:09.154725 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 03:08:09.154725 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 13 03:08:09.154725 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 13 03:08:09.154725 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 13 03:08:09.154725 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 13 03:08:09.156337 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 03:08:09.156337 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 03:08:09.156337 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 03:08:09.156337 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 03:08:09.156337 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 03:08:09.156337 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 03:08:09.156337 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 03:08:09.156337 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application with Lock( Sep 13 03:08:09.156337 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 03:08:09.156337 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._enter() Sep 13 03:08:09.156337 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 03:08:09.156337 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 03:08:09.156337 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 03:08:09.156337 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.creator() Sep 13 03:08:09.156337 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 03:08:09.156337 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application created_value = creator( Sep 13 03:08:09.156337 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 13 03:08:09.157243 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 13 03:08:09.157243 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 03:08:09.157243 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Sep 13 03:08:09.157243 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 272/553] 10.222.0.18 () {56 vars in 1108 bytes} [Tue Sep 13 03:08:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 03:08:09.157944 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e5351f3-57b2-49e1-8089-a8dd2adaff94 None None] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:08:09.158178 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e5351f3-57b2-49e1-8089-a8dd2adaff94 None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:08:09.158451 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e5351f3-57b2-49e1-8089-a8dd2adaff94 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:08:09.179038 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.auth.core [None req-9e5351f3-57b2-49e1-8089-a8dd2adaff94 None None] MFA Rules not processed for user `49f280ab086b4610855c4ffa378517d9`. Rule list: `[]` (Enabled: `True`). {{(pid=81399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 03:08:09.180596 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-9e5351f3-57b2-49e1-8089-a8dd2adaff94 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:08:09.183558 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba10de2b-5603-40d3-b0aa-1bcd6f274bac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:08:09.184482 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba10de2b-5603-40d3-b0aa-1bcd6f274bac None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:08:09.184971 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba10de2b-5603-40d3-b0aa-1bcd6f274bac None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:08:09.185438 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba10de2b-5603-40d3-b0aa-1bcd6f274bac None admin] PATH_INFO: `/v3/users/swift` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:08:09.193605 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba10de2b-5603-40d3-b0aa-1bcd6f274bac None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:08:09.195646 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba10de2b-5603-40d3-b0aa-1bcd6f274bac None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:08:09.208936 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application [None req-ba10de2b-5603-40d3-b0aa-1bcd6f274bac None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Sep 13 03:08:09.208936 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 03:08:09.208936 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 03:08:09.208936 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 03:08:09.208936 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 03:08:09.208936 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 03:08:09.208936 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 03:08:09.208936 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 03:08:09.208936 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 03:08:09.208936 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 03:08:09.208936 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 03:08:09.208936 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 03:08:09.208936 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 13 03:08:09.208936 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 13 03:08:09.208936 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 13 03:08:09.208936 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 13 03:08:09.209884 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 03:08:09.209884 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 03:08:09.209884 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 13 03:08:09.209884 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 13 03:08:09.209884 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 13 03:08:09.209884 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 13 03:08:09.209884 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 03:08:09.209884 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 03:08:09.209884 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 03:08:09.209884 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 03:08:09.209884 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 03:08:09.209884 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application with Lock( Sep 13 03:08:09.209884 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 03:08:09.209884 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._enter() Sep 13 03:08:09.209884 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 03:08:09.209884 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 03:08:09.209884 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 03:08:09.209884 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.creator() Sep 13 03:08:09.210904 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 03:08:09.210904 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application created_value = creator( Sep 13 03:08:09.210904 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 13 03:08:09.210904 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 13 03:08:09.210904 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 13 03:08:09.210904 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 13 03:08:09.210904 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 13 03:08:09.210904 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 13 03:08:09.210904 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Sep 13 03:08:09.210904 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Sep 13 03:08:09.213424 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 282/554] 10.222.0.18 () {56 vars in 1096 bytes} [Tue Sep 13 03:08:09 2022] GET /identity/v3/users/swift => generated 83 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 03:08:09.214391 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba10de2b-5603-40d3-b0aa-1bcd6f274bac None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:08:09.262636 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 273/555] 10.222.0.18 () {58 vars in 1024 bytes} [Tue Sep 13 03:08:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 13 03:08:09.262636 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e5351f3-57b2-49e1-8089-a8dd2adaff94 None None] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:08:09.285659 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:08:09.372129 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e5351f3-57b2-49e1-8089-a8dd2adaff94 None None] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:08:09.379901 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba10de2b-5603-40d3-b0aa-1bcd6f274bac None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:08:09.409492 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0566a9f-f64b-4fbd-863a-bc68300b65c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:08:09.410037 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0566a9f-f64b-4fbd-863a-bc68300b65c8 None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:08:09.410317 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0566a9f-f64b-4fbd-863a-bc68300b65c8 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:08:09.410589 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0566a9f-f64b-4fbd-863a-bc68300b65c8 None admin] PATH_INFO: `/v3/users` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:08:09.411380 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0566a9f-f64b-4fbd-863a-bc68300b65c8 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 03:08:09.411734 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0566a9f-f64b-4fbd-863a-bc68300b65c8 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:08:09.416433 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0566a9f-f64b-4fbd-863a-bc68300b65c8 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:08:09.438074 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 274/556] 10.222.0.18 () {56 vars in 1135 bytes} [Tue Sep 13 03:08:09 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 03:08:09.439461 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba2f0ce-37dc-48ca-8df6-1cd161278a98 None None] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:08:09.439746 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba2f0ce-37dc-48ca-8df6-1cd161278a98 None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:08:09.440018 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba2f0ce-37dc-48ca-8df6-1cd161278a98 None None] PATH_INFO: `/` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:08:09.441445 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 275/557] 10.222.0.18 () {54 vars in 927 bytes} [Tue Sep 13 03:08:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 03:08:09.442259 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1274ec4-de40-4276-b76a-f05021b3058e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:08:09.443124 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1274ec4-de40-4276-b76a-f05021b3058e None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:08:09.443599 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1274ec4-de40-4276-b76a-f05021b3058e None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:08:09.444078 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1274ec4-de40-4276-b76a-f05021b3058e None admin] PATH_INFO: `/v3/domains` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:08:09.444908 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1274ec4-de40-4276-b76a-f05021b3058e None admin] RBAC: Adding query filter params (name=Default) {{(pid=81400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 03:08:09.445420 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1274ec4-de40-4276-b76a-f05021b3058e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:08:09.445983 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ba2f0ce-37dc-48ca-8df6-1cd161278a98 None None] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:08:09.447336 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1274ec4-de40-4276-b76a-f05021b3058e None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:08:09.462025 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 283/558] 10.222.0.18 () {56 vars in 1109 bytes} [Tue Sep 13 03:08:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 248 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 03:08:09.463868 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abaf4f29-6788-4da0-b693-eac27e9bdbb2 None None] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:08:09.464351 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abaf4f29-6788-4da0-b693-eac27e9bdbb2 None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:08:09.464817 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abaf4f29-6788-4da0-b693-eac27e9bdbb2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:08:09.486366 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.auth.core [None req-abaf4f29-6788-4da0-b693-eac27e9bdbb2 None None] MFA Rules not processed for user `49f280ab086b4610855c4ffa378517d9`. Rule list: `[]` (Enabled: `True`). {{(pid=81400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 03:08:09.488199 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-abaf4f29-6788-4da0-b693-eac27e9bdbb2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:08:09.507842 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:08:09.561664 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:08:09.574788 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 284/559] 10.222.0.18 () {58 vars in 1024 bytes} [Tue Sep 13 03:08:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 13 03:08:09.574788 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abaf4f29-6788-4da0-b693-eac27e9bdbb2 None None] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:08:09.605681 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ba2f0ce-37dc-48ca-8df6-1cd161278a98 None None] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:08:09.657577 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-890f4788-db37-4bc0-a870-729f6964144b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:08:09.658450 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-890f4788-db37-4bc0-a870-729f6964144b None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:08:09.658934 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-890f4788-db37-4bc0-a870-729f6964144b None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:08:09.659455 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-890f4788-db37-4bc0-a870-729f6964144b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:08:09.727651 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abaf4f29-6788-4da0-b693-eac27e9bdbb2 None None] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:08:09.732433 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-890f4788-db37-4bc0-a870-729f6964144b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:08:09.734074 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-890f4788-db37-4bc0-a870-729f6964144b None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:08:09.775633 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e8d31f7-e678-49ea-9d4f-9f937b6b1b18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:08:09.776291 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e8d31f7-e678-49ea-9d4f-9f937b6b1b18 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:08:09.776592 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e8d31f7-e678-49ea-9d4f-9f937b6b1b18 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:08:09.777571 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e8d31f7-e678-49ea-9d4f-9f937b6b1b18 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:08:09.788946 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e8d31f7-e678-49ea-9d4f-9f937b6b1b18 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:08:09.790994 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e8d31f7-e678-49ea-9d4f-9f937b6b1b18 None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:08:09.813458 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application [None req-4e8d31f7-e678-49ea-9d4f-9f937b6b1b18 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 13 03:08:09.813458 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 03:08:09.813458 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 03:08:09.813458 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 03:08:09.813458 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 03:08:09.813458 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 03:08:09.813458 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 03:08:09.813458 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 03:08:09.813458 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 03:08:09.813458 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 03:08:09.813458 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 03:08:09.813458 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 03:08:09.813458 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 13 03:08:09.813458 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 13 03:08:09.813458 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 13 03:08:09.813458 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 13 03:08:09.814452 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 03:08:09.814452 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 03:08:09.814452 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 13 03:08:09.814452 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 13 03:08:09.814452 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 13 03:08:09.814452 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 13 03:08:09.814452 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 03:08:09.814452 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 03:08:09.814452 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 03:08:09.814452 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 03:08:09.814452 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 03:08:09.814452 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application with Lock( Sep 13 03:08:09.814452 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 03:08:09.814452 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._enter() Sep 13 03:08:09.814452 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 03:08:09.814452 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 03:08:09.814452 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 03:08:09.814452 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.creator() Sep 13 03:08:09.815447 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 03:08:09.815447 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application created_value = creator( Sep 13 03:08:09.815447 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 13 03:08:09.815447 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 13 03:08:09.815447 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 13 03:08:09.815447 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 13 03:08:09.815447 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 13 03:08:09.815447 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 13 03:08:09.815447 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 13 03:08:09.815447 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Sep 13 03:08:09.818688 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 285/560] 10.222.0.18 () {56 vars in 1093 bytes} [Tue Sep 13 03:08:09 2022] GET /identity/v3/users/nova => generated 82 bytes in 247 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 03:08:09.819703 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e8d31f7-e678-49ea-9d4f-9f937b6b1b18 None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:08:09.836548 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:08:09.840492 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:08:09.862175 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 276/561] 10.222.0.18 () {58 vars in 1282 bytes} [Tue Sep 13 03:08:09 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 416 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 13 03:08:09.863745 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-890f4788-db37-4bc0-a870-729f6964144b None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:08:09.923650 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e8d31f7-e678-49ea-9d4f-9f937b6b1b18 None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:08:09.957261 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7baeda09-bc5a-4136-acac-35e398dc0c7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:08:09.957904 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7baeda09-bc5a-4136-acac-35e398dc0c7c None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:08:09.958159 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7baeda09-bc5a-4136-acac-35e398dc0c7c None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:08:09.958430 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7baeda09-bc5a-4136-acac-35e398dc0c7c None admin] PATH_INFO: `/v3/roles/service` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:08:09.961827 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-890f4788-db37-4bc0-a870-729f6964144b None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:08:09.971094 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7baeda09-bc5a-4136-acac-35e398dc0c7c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 03:08:09.976321 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application [None req-7baeda09-bc5a-4136-acac-35e398dc0c7c None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 13 03:08:09.976321 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 03:08:09.976321 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 03:08:09.976321 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 03:08:09.976321 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 03:08:09.976321 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 03:08:09.976321 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 03:08:09.976321 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 03:08:09.976321 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 03:08:09.976321 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 03:08:09.976321 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 03:08:09.976321 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 03:08:09.976321 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 03:08:09.976321 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 03:08:09.976321 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 03:08:09.976321 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 03:08:09.977426 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 03:08:09.977426 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 03:08:09.977426 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 03:08:09.977426 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 03:08:09.977426 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 03:08:09.977426 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 03:08:09.977426 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 03:08:09.977426 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 03:08:09.977426 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 03:08:09.977426 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 03:08:09.977426 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 03:08:09.977426 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application with Lock( Sep 13 03:08:09.977426 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 03:08:09.977426 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._enter() Sep 13 03:08:09.977426 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 03:08:09.977426 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 03:08:09.978318 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 03:08:09.978318 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.creator() Sep 13 03:08:09.978318 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 03:08:09.978318 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application created_value = creator( Sep 13 03:08:09.978318 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 03:08:09.978318 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 03:08:09.978318 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 03:08:09.978318 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 03:08:09.978318 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 03:08:09.978318 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 03:08:09.978318 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 13 03:08:09.978318 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Sep 13 03:08:09.978318 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 286/562] 10.222.0.18 () {56 vars in 1102 bytes} [Tue Sep 13 03:08:09 2022] GET /identity/v3/roles/service => generated 85 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 03:08:09.978318 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7baeda09-bc5a-4136-acac-35e398dc0c7c None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:08:09.996118 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73f8d345-bfea-4c71-9f56-3eac345b3221 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:08:09.996928 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f8d345-bfea-4c71-9f56-3eac345b3221 None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:08:09.997354 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f8d345-bfea-4c71-9f56-3eac345b3221 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:08:09.997622 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f8d345-bfea-4c71-9f56-3eac345b3221 None admin] PATH_INFO: `/v3/users` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:08:09.998331 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73f8d345-bfea-4c71-9f56-3eac345b3221 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 03:08:09.998672 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73f8d345-bfea-4c71-9f56-3eac345b3221 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:08:10.000611 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73f8d345-bfea-4c71-9f56-3eac345b3221 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:08:10.018002 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 277/563] 10.222.0.18 () {56 vars in 1133 bytes} [Tue Sep 13 03:08:09 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 03:08:10.019296 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f509d9-148a-4ece-a1b0-cbe56c80330a None None] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:08:10.019672 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f509d9-148a-4ece-a1b0-cbe56c80330a None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:08:10.020036 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f509d9-148a-4ece-a1b0-cbe56c80330a None None] PATH_INFO: `/` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:08:10.021458 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 278/564] 10.222.0.18 () {54 vars in 927 bytes} [Tue Sep 13 03:08:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 03:08:10.023355 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37f509d9-148a-4ece-a1b0-cbe56c80330a None None] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:08:10.054733 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:08:10.067953 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:08:10.083734 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7baeda09-bc5a-4136-acac-35e398dc0c7c None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:08:10.116124 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb850d2f-9e4a-4237-a14d-2b65e55a618f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:08:10.116743 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb850d2f-9e4a-4237-a14d-2b65e55a618f None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:08:10.117023 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb850d2f-9e4a-4237-a14d-2b65e55a618f None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:08:10.117294 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb850d2f-9e4a-4237-a14d-2b65e55a618f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:08:10.127136 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37f509d9-148a-4ece-a1b0-cbe56c80330a None None] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:08:10.128050 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb850d2f-9e4a-4237-a14d-2b65e55a618f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:08:10.129498 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb850d2f-9e4a-4237-a14d-2b65e55a618f None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:08:10.143920 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application [None req-bb850d2f-9e4a-4237-a14d-2b65e55a618f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 03:08:10.143920 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 03:08:10.143920 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 03:08:10.143920 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 03:08:10.143920 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 03:08:10.143920 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 03:08:10.143920 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 03:08:10.143920 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 03:08:10.143920 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 03:08:10.143920 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 03:08:10.143920 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 03:08:10.143920 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 03:08:10.143920 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 13 03:08:10.143920 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 13 03:08:10.143920 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 13 03:08:10.143920 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 13 03:08:10.145019 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 03:08:10.145019 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 03:08:10.145019 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 03:08:10.145019 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 03:08:10.145019 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 03:08:10.145019 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 03:08:10.145019 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 03:08:10.145019 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application with Lock( Sep 13 03:08:10.145019 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 03:08:10.145019 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._enter() Sep 13 03:08:10.145019 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 03:08:10.145019 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 03:08:10.145019 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 03:08:10.145019 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.creator() Sep 13 03:08:10.145019 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 03:08:10.145019 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application created_value = creator( Sep 13 03:08:10.145019 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 13 03:08:10.150751 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 13 03:08:10.150751 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 03:08:10.150751 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Sep 13 03:08:10.150751 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 287/565] 10.222.0.18 () {56 vars in 1108 bytes} [Tue Sep 13 03:08:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 03:08:10.150751 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb850d2f-9e4a-4237-a14d-2b65e55a618f None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:08:10.167640 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73291447-b5a9-4ab0-a9b2-04f8428c1f29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:08:10.168265 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73291447-b5a9-4ab0-a9b2-04f8428c1f29 None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:08:10.168545 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73291447-b5a9-4ab0-a9b2-04f8428c1f29 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:08:10.168815 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73291447-b5a9-4ab0-a9b2-04f8428c1f29 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:08:10.178723 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73291447-b5a9-4ab0-a9b2-04f8428c1f29 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 03:08:10.183838 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application [None req-73291447-b5a9-4ab0-a9b2-04f8428c1f29 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 13 03:08:10.183838 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 03:08:10.183838 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 03:08:10.183838 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 03:08:10.183838 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 03:08:10.183838 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 03:08:10.183838 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 03:08:10.183838 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 03:08:10.183838 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 03:08:10.183838 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 03:08:10.183838 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 03:08:10.183838 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 03:08:10.183838 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 03:08:10.183838 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 03:08:10.183838 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 03:08:10.183838 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 03:08:10.184938 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 03:08:10.184938 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 03:08:10.184938 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 03:08:10.184938 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 03:08:10.184938 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 03:08:10.184938 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 03:08:10.184938 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 03:08:10.184938 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 03:08:10.184938 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 03:08:10.184938 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 03:08:10.184938 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 03:08:10.184938 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application with Lock( Sep 13 03:08:10.184938 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 03:08:10.184938 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._enter() Sep 13 03:08:10.184938 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 03:08:10.184938 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 03:08:10.185985 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 03:08:10.185985 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.creator() Sep 13 03:08:10.185985 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 03:08:10.185985 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application created_value = creator( Sep 13 03:08:10.185985 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 03:08:10.185985 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 03:08:10.185985 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 03:08:10.185985 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 03:08:10.185985 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 03:08:10.185985 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 03:08:10.185985 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 13 03:08:10.185985 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Sep 13 03:08:10.185985 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 279/566] 10.222.0.18 () {56 vars in 1102 bytes} [Tue Sep 13 03:08:10 2022] GET /identity/v3/roles/service => generated 85 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 03:08:10.185985 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73291447-b5a9-4ab0-a9b2-04f8428c1f29 None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:08:10.247157 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb850d2f-9e4a-4237-a14d-2b65e55a618f None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:08:10.272114 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:08:10.279823 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:08:10.280693 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19621503-5833-485e-b6a2-cbb86ca550a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:08:10.281341 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19621503-5833-485e-b6a2-cbb86ca550a4 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:08:10.281680 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19621503-5833-485e-b6a2-cbb86ca550a4 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:08:10.281949 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19621503-5833-485e-b6a2-cbb86ca550a4 None admin] PATH_INFO: `/v3/roles` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:08:10.282595 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19621503-5833-485e-b6a2-cbb86ca550a4 None admin] RBAC: Adding query filter params (name=service) {{(pid=81400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 03:08:10.282914 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19621503-5833-485e-b6a2-cbb86ca550a4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:08:10.283954 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73291447-b5a9-4ab0-a9b2-04f8428c1f29 None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:08:10.284279 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19621503-5833-485e-b6a2-cbb86ca550a4 None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:08:10.291927 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 288/567] 10.222.0.18 () {56 vars in 1103 bytes} [Tue Sep 13 03:08:10 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 03:08:10.293048 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d86047e-53d9-43ae-aaf9-39cb386339fe None None] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:08:10.293321 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d86047e-53d9-43ae-aaf9-39cb386339fe None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:08:10.293584 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d86047e-53d9-43ae-aaf9-39cb386339fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:08:10.311443 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.auth.core [None req-7d86047e-53d9-43ae-aaf9-39cb386339fe None None] MFA Rules not processed for user `49f280ab086b4610855c4ffa378517d9`. Rule list: `[]` (Enabled: `True`). {{(pid=81400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 03:08:10.313219 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-7d86047e-53d9-43ae-aaf9-39cb386339fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:08:10.319896 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33223161-45c3-4409-88f4-1d3d1029285b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:08:10.320526 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33223161-45c3-4409-88f4-1d3d1029285b None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:08:10.320805 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33223161-45c3-4409-88f4-1d3d1029285b None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:08:10.321075 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33223161-45c3-4409-88f4-1d3d1029285b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:08:10.330904 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33223161-45c3-4409-88f4-1d3d1029285b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:08:10.332343 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33223161-45c3-4409-88f4-1d3d1029285b None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:08:10.346967 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application [None req-33223161-45c3-4409-88f4-1d3d1029285b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 03:08:10.346967 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 03:08:10.346967 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 03:08:10.346967 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 03:08:10.346967 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 03:08:10.346967 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 03:08:10.346967 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 03:08:10.346967 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 03:08:10.346967 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 03:08:10.346967 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 03:08:10.346967 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 03:08:10.346967 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 03:08:10.346967 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 13 03:08:10.346967 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 13 03:08:10.346967 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 13 03:08:10.346967 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 13 03:08:10.348112 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 03:08:10.348112 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 03:08:10.348112 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 03:08:10.348112 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 03:08:10.348112 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 03:08:10.348112 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 03:08:10.348112 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 03:08:10.348112 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application with Lock( Sep 13 03:08:10.348112 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 03:08:10.348112 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._enter() Sep 13 03:08:10.348112 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 03:08:10.348112 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 03:08:10.348112 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 03:08:10.348112 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.creator() Sep 13 03:08:10.348112 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 03:08:10.348112 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application created_value = creator( Sep 13 03:08:10.348112 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 13 03:08:10.349237 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 13 03:08:10.349237 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 03:08:10.349237 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Sep 13 03:08:10.349237 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 280/568] 10.222.0.18 () {56 vars in 1108 bytes} [Tue Sep 13 03:08:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 03:08:10.349237 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33223161-45c3-4409-88f4-1d3d1029285b None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:08:10.380516 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 289/569] 10.222.0.18 () {58 vars in 1024 bytes} [Tue Sep 13 03:08:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 13 03:08:10.381332 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d86047e-53d9-43ae-aaf9-39cb386339fe None None] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:08:10.454500 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33223161-45c3-4409-88f4-1d3d1029285b None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:08:10.482784 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d86047e-53d9-43ae-aaf9-39cb386339fe None None] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:08:10.488599 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-170de795-c753-4dfd-ac35-c56a4fdb5092 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:08:10.489199 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-170de795-c753-4dfd-ac35-c56a4fdb5092 None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:08:10.489474 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-170de795-c753-4dfd-ac35-c56a4fdb5092 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:08:10.489741 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-170de795-c753-4dfd-ac35-c56a4fdb5092 None admin] PATH_INFO: `/v3/domains` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:08:10.490646 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-170de795-c753-4dfd-ac35-c56a4fdb5092 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 03:08:10.490646 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-170de795-c753-4dfd-ac35-c56a4fdb5092 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:08:10.492032 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-170de795-c753-4dfd-ac35-c56a4fdb5092 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:08:10.502022 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:08:10.507277 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 281/570] 10.222.0.18 () {56 vars in 1109 bytes} [Tue Sep 13 03:08:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 03:08:10.508059 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-170de795-c753-4dfd-ac35-c56a4fdb5092 None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:08:10.521064 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:08:10.526449 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e5445cc-1105-4c8f-be8a-c2f75cca7c0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:08:10.527093 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e5445cc-1105-4c8f-be8a-c2f75cca7c0b None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:08:10.527379 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e5445cc-1105-4c8f-be8a-c2f75cca7c0b None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:08:10.527654 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e5445cc-1105-4c8f-be8a-c2f75cca7c0b None admin] PATH_INFO: `/v3/roles` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:08:10.528319 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e5445cc-1105-4c8f-be8a-c2f75cca7c0b None admin] RBAC: Adding query filter params (name=service) {{(pid=81400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 03:08:10.528627 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e5445cc-1105-4c8f-be8a-c2f75cca7c0b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:08:10.529987 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e5445cc-1105-4c8f-be8a-c2f75cca7c0b None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:08:10.538204 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 290/571] 10.222.0.18 () {56 vars in 1103 bytes} [Tue Sep 13 03:08:10 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 03:08:10.538997 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e5445cc-1105-4c8f-be8a-c2f75cca7c0b None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:08:10.618759 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-170de795-c753-4dfd-ac35-c56a4fdb5092 None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:08:10.641502 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e5445cc-1105-4c8f-be8a-c2f75cca7c0b None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:08:10.654830 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d4fdb1c-2c05-4cbc-9171-4aa2854e2029 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:08:10.656117 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d4fdb1c-2c05-4cbc-9171-4aa2854e2029 None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:08:10.656403 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d4fdb1c-2c05-4cbc-9171-4aa2854e2029 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:08:10.656675 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d4fdb1c-2c05-4cbc-9171-4aa2854e2029 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:08:10.666735 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d4fdb1c-2c05-4cbc-9171-4aa2854e2029 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:08:10.668181 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d4fdb1c-2c05-4cbc-9171-4aa2854e2029 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:08:10.675011 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7a02b72-9cf7-4208-8f98-048f3ae31790 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:08:10.675602 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a02b72-9cf7-4208-8f98-048f3ae31790 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:08:10.675872 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a02b72-9cf7-4208-8f98-048f3ae31790 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:08:10.676139 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a02b72-9cf7-4208-8f98-048f3ae31790 None admin] PATH_INFO: `/v3/domains` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:08:10.676761 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7a02b72-9cf7-4208-8f98-048f3ae31790 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 03:08:10.677054 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7a02b72-9cf7-4208-8f98-048f3ae31790 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:08:10.678399 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7a02b72-9cf7-4208-8f98-048f3ae31790 None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:08:10.683888 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application [None req-3d4fdb1c-2c05-4cbc-9171-4aa2854e2029 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 03:08:10.683888 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 03:08:10.683888 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 03:08:10.683888 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 03:08:10.683888 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 03:08:10.683888 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 03:08:10.683888 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 03:08:10.683888 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 03:08:10.683888 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 03:08:10.683888 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 03:08:10.683888 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 03:08:10.683888 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 03:08:10.683888 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 13 03:08:10.683888 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 13 03:08:10.683888 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 13 03:08:10.683888 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 13 03:08:10.684960 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 03:08:10.684960 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 03:08:10.684960 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 03:08:10.684960 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 03:08:10.684960 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 03:08:10.684960 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 03:08:10.684960 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 03:08:10.684960 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application with Lock( Sep 13 03:08:10.684960 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 03:08:10.684960 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._enter() Sep 13 03:08:10.684960 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 03:08:10.684960 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 03:08:10.684960 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 03:08:10.684960 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.creator() Sep 13 03:08:10.684960 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 03:08:10.684960 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application created_value = creator( Sep 13 03:08:10.684960 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 13 03:08:10.685947 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 13 03:08:10.685947 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 03:08:10.685947 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Sep 13 03:08:10.685947 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 282/572] 10.222.0.18 () {56 vars in 1108 bytes} [Tue Sep 13 03:08:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 03:08:10.686208 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d4fdb1c-2c05-4cbc-9171-4aa2854e2029 None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:08:10.694533 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 291/573] 10.222.0.18 () {56 vars in 1109 bytes} [Tue Sep 13 03:08:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 03:08:10.695483 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7a02b72-9cf7-4208-8f98-048f3ae31790 None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:08:10.737753 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:08:10.747455 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:08:10.786294 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d4fdb1c-2c05-4cbc-9171-4aa2854e2029 None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:08:10.793889 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7a02b72-9cf7-4208-8f98-048f3ae31790 None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:08:10.818442 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72c6151c-9018-4fce-912c-201137f3a007 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:08:10.819093 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72c6151c-9018-4fce-912c-201137f3a007 None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:08:10.819367 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72c6151c-9018-4fce-912c-201137f3a007 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:08:10.819636 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72c6151c-9018-4fce-912c-201137f3a007 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:08:10.826237 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72c6151c-9018-4fce-912c-201137f3a007 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:08:10.827456 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b430de0-5874-4851-b114-1b37b34fede3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:08:10.827737 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72c6151c-9018-4fce-912c-201137f3a007 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:08:10.828064 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b430de0-5874-4851-b114-1b37b34fede3 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:08:10.828337 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b430de0-5874-4851-b114-1b37b34fede3 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:08:10.828603 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b430de0-5874-4851-b114-1b37b34fede3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:08:10.838259 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b430de0-5874-4851-b114-1b37b34fede3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:08:10.838920 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application [None req-72c6151c-9018-4fce-912c-201137f3a007 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 13 03:08:10.838920 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 03:08:10.838920 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 03:08:10.838920 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 03:08:10.838920 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 03:08:10.838920 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 03:08:10.838920 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 03:08:10.838920 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 03:08:10.838920 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 03:08:10.838920 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 03:08:10.838920 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 03:08:10.838920 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 03:08:10.838920 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 13 03:08:10.838920 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 13 03:08:10.838920 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 13 03:08:10.839945 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 13 03:08:10.839945 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 03:08:10.839945 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 03:08:10.839945 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 03:08:10.839945 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 03:08:10.839945 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 03:08:10.839945 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 03:08:10.839945 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 03:08:10.839945 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application with Lock( Sep 13 03:08:10.839945 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 03:08:10.839945 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._enter() Sep 13 03:08:10.839945 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 03:08:10.839945 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 03:08:10.839945 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 03:08:10.839945 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.creator() Sep 13 03:08:10.839945 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 03:08:10.839945 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application created_value = creator( Sep 13 03:08:10.841007 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 13 03:08:10.841007 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 13 03:08:10.841007 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 13 03:08:10.841007 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 13 03:08:10.841007 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 13 03:08:10.841007 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 13 03:08:10.841007 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 13 03:08:10.841007 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Sep 13 03:08:10.841495 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b430de0-5874-4851-b114-1b37b34fede3 None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:08:10.845246 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 283/574] 10.222.0.18 () {56 vars in 1111 bytes} [Tue Sep 13 03:08:10 2022] GET /identity/v3/projects/service => generated 88 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 03:08:10.845932 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72c6151c-9018-4fce-912c-201137f3a007 None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:08:10.854979 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application [None req-9b430de0-5874-4851-b114-1b37b34fede3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 03:08:10.854979 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 03:08:10.854979 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 03:08:10.854979 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 03:08:10.854979 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 03:08:10.854979 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 03:08:10.854979 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 03:08:10.854979 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 03:08:10.854979 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 03:08:10.854979 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 03:08:10.854979 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 03:08:10.854979 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 03:08:10.854979 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 13 03:08:10.854979 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 13 03:08:10.854979 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 13 03:08:10.854979 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 13 03:08:10.856141 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 03:08:10.856141 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 03:08:10.856141 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 03:08:10.856141 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 03:08:10.856141 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 03:08:10.856141 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 03:08:10.856141 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 03:08:10.856141 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application with Lock( Sep 13 03:08:10.856141 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 03:08:10.856141 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._enter() Sep 13 03:08:10.856141 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 03:08:10.856141 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 03:08:10.856141 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 03:08:10.856141 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.creator() Sep 13 03:08:10.856141 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 03:08:10.856141 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application created_value = creator( Sep 13 03:08:10.856141 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 13 03:08:10.857252 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 13 03:08:10.857252 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 03:08:10.857252 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Sep 13 03:08:10.857252 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 292/575] 10.222.0.18 () {56 vars in 1108 bytes} [Tue Sep 13 03:08:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 03:08:10.857252 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b430de0-5874-4851-b114-1b37b34fede3 None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:08:10.944412 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72c6151c-9018-4fce-912c-201137f3a007 None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:08:10.948835 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:08:10.958056 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b430de0-5874-4851-b114-1b37b34fede3 None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:08:10.966271 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:08:10.978145 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19fd25df-7786-47c4-94a3-ce1a705b9411 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:08:10.978794 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19fd25df-7786-47c4-94a3-ce1a705b9411 None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:08:10.979069 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19fd25df-7786-47c4-94a3-ce1a705b9411 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:08:10.979334 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19fd25df-7786-47c4-94a3-ce1a705b9411 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:08:10.989546 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19fd25df-7786-47c4-94a3-ce1a705b9411 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 03:08:10.991253 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b55e468e-b21f-4ee7-8bdc-6aa3caeb0d1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:08:10.991847 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b55e468e-b21f-4ee7-8bdc-6aa3caeb0d1e None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:08:10.992121 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b55e468e-b21f-4ee7-8bdc-6aa3caeb0d1e None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:08:10.992386 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b55e468e-b21f-4ee7-8bdc-6aa3caeb0d1e None admin] PATH_INFO: `/v3/domains` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:08:10.993008 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b55e468e-b21f-4ee7-8bdc-6aa3caeb0d1e None admin] RBAC: Adding query filter params (name=Default) {{(pid=81400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 03:08:10.993303 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b55e468e-b21f-4ee7-8bdc-6aa3caeb0d1e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:08:10.994696 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b55e468e-b21f-4ee7-8bdc-6aa3caeb0d1e None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:08:10.994815 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application [None req-19fd25df-7786-47c4-94a3-ce1a705b9411 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 13 03:08:10.994815 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 03:08:10.994815 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 03:08:10.994815 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 03:08:10.994815 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 03:08:10.994815 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 03:08:10.994815 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 03:08:10.994815 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 03:08:10.994815 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 03:08:10.994815 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 03:08:10.994815 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 03:08:10.994815 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 03:08:10.994815 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 03:08:10.994815 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 03:08:10.994815 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 03:08:10.994815 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 03:08:10.995870 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 03:08:10.995870 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 03:08:10.995870 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 03:08:10.995870 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 03:08:10.995870 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 03:08:10.995870 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 03:08:10.995870 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 03:08:10.995870 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 03:08:10.995870 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 03:08:10.995870 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 03:08:10.995870 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 03:08:10.995870 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application with Lock( Sep 13 03:08:10.995870 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 03:08:10.995870 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._enter() Sep 13 03:08:10.995870 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 03:08:10.995870 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 03:08:10.996793 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 03:08:10.996793 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.creator() Sep 13 03:08:10.996793 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 03:08:10.996793 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application created_value = creator( Sep 13 03:08:10.996793 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 03:08:10.996793 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 03:08:10.996793 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 03:08:10.996793 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 03:08:10.996793 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 03:08:10.996793 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 03:08:10.996793 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 13 03:08:10.996793 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Sep 13 03:08:10.996793 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 284/576] 10.222.0.18 () {56 vars in 1102 bytes} [Tue Sep 13 03:08:10 2022] GET /identity/v3/roles/service => generated 85 bytes in 150 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 03:08:10.997490 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19fd25df-7786-47c4-94a3-ce1a705b9411 None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:08:11.012955 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 293/577] 10.222.0.18 () {56 vars in 1109 bytes} [Tue Sep 13 03:08:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 03:08:11.012955 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b55e468e-b21f-4ee7-8bdc-6aa3caeb0d1e None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:08:11.103420 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19fd25df-7786-47c4-94a3-ce1a705b9411 None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:08:11.107558 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b55e468e-b21f-4ee7-8bdc-6aa3caeb0d1e None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:08:11.136768 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-801f21f6-0632-47ff-a6fc-ee32f762145c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:08:11.137395 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-801f21f6-0632-47ff-a6fc-ee32f762145c None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:08:11.137673 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-801f21f6-0632-47ff-a6fc-ee32f762145c None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:08:11.137946 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-801f21f6-0632-47ff-a6fc-ee32f762145c None admin] PATH_INFO: `/v3/projects/service` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:08:11.139633 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d27f651d-cd6c-4ab7-88d4-488a5322de5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:08:11.140753 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d27f651d-cd6c-4ab7-88d4-488a5322de5c None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:08:11.141025 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d27f651d-cd6c-4ab7-88d4-488a5322de5c None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:08:11.141286 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d27f651d-cd6c-4ab7-88d4-488a5322de5c None admin] PATH_INFO: `/v3/projects` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:08:11.141955 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d27f651d-cd6c-4ab7-88d4-488a5322de5c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 03:08:11.142248 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d27f651d-cd6c-4ab7-88d4-488a5322de5c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:08:11.143730 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d27f651d-cd6c-4ab7-88d4-488a5322de5c None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:08:11.145036 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-801f21f6-0632-47ff-a6fc-ee32f762145c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:08:11.146481 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-801f21f6-0632-47ff-a6fc-ee32f762145c None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:08:11.155430 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 294/578] 10.222.0.18 () {56 vars in 1148 bytes} [Tue Sep 13 03:08:11 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 03:08:11.156786 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d27f651d-cd6c-4ab7-88d4-488a5322de5c None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:08:11.158174 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application [None req-801f21f6-0632-47ff-a6fc-ee32f762145c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 13 03:08:11.158174 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 03:08:11.158174 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 03:08:11.158174 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 03:08:11.158174 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 03:08:11.158174 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 03:08:11.158174 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 03:08:11.158174 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 03:08:11.158174 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 03:08:11.158174 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 03:08:11.158174 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 03:08:11.158174 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 03:08:11.158174 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 13 03:08:11.158174 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 13 03:08:11.158174 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 13 03:08:11.159072 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 13 03:08:11.159072 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 03:08:11.159072 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 03:08:11.159072 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 03:08:11.159072 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 03:08:11.159072 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 03:08:11.159072 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 03:08:11.159072 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 03:08:11.159072 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application with Lock( Sep 13 03:08:11.159072 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 03:08:11.159072 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._enter() Sep 13 03:08:11.159072 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 03:08:11.159072 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 03:08:11.159072 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 03:08:11.159072 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.creator() Sep 13 03:08:11.159072 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 03:08:11.159072 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application created_value = creator( Sep 13 03:08:11.159978 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 13 03:08:11.159978 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 13 03:08:11.159978 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 13 03:08:11.159978 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 13 03:08:11.159978 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 13 03:08:11.159978 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 13 03:08:11.159978 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 13 03:08:11.159978 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Sep 13 03:08:11.159978 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 285/579] 10.222.0.18 () {56 vars in 1111 bytes} [Tue Sep 13 03:08:10 2022] GET /identity/v3/projects/service => generated 88 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 03:08:11.160465 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-801f21f6-0632-47ff-a6fc-ee32f762145c None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:08:11.174702 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:08:11.202917 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:08:11.258476 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-801f21f6-0632-47ff-a6fc-ee32f762145c None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:08:11.259529 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d27f651d-cd6c-4ab7-88d4-488a5322de5c None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:08:11.290828 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59bdddf1-dba3-4199-8bb1-8693e2b36e28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:08:11.291418 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59bdddf1-dba3-4199-8bb1-8693e2b36e28 None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:08:11.291692 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59bdddf1-dba3-4199-8bb1-8693e2b36e28 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:08:11.291956 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59bdddf1-dba3-4199-8bb1-8693e2b36e28 None admin] PATH_INFO: `/v3/roles` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:08:11.292600 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59bdddf1-dba3-4199-8bb1-8693e2b36e28 None admin] RBAC: Adding query filter params (name=service) {{(pid=81399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 03:08:11.292713 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cc285ed-af4e-4252-bf6f-15e263893d84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:08:11.292896 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59bdddf1-dba3-4199-8bb1-8693e2b36e28 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:08:11.293271 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc285ed-af4e-4252-bf6f-15e263893d84 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:08:11.293575 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc285ed-af4e-4252-bf6f-15e263893d84 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:08:11.293848 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc285ed-af4e-4252-bf6f-15e263893d84 None admin] PATH_INFO: `/v3/domains` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:08:11.294240 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59bdddf1-dba3-4199-8bb1-8693e2b36e28 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:08:11.294476 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cc285ed-af4e-4252-bf6f-15e263893d84 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 03:08:11.294806 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cc285ed-af4e-4252-bf6f-15e263893d84 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:08:11.296410 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cc285ed-af4e-4252-bf6f-15e263893d84 None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:08:11.301827 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 286/580] 10.222.0.18 () {56 vars in 1103 bytes} [Tue Sep 13 03:08:11 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 03:08:11.302591 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59bdddf1-dba3-4199-8bb1-8693e2b36e28 None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:08:11.315933 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 295/581] 10.222.0.18 () {56 vars in 1109 bytes} [Tue Sep 13 03:08:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 03:08:11.315933 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cc285ed-af4e-4252-bf6f-15e263893d84 None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:08:11.385101 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:08:11.400930 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59bdddf1-dba3-4199-8bb1-8693e2b36e28 None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:08:11.403214 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:08:11.415194 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cc285ed-af4e-4252-bf6f-15e263893d84 None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:08:11.437976 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6890416-d22e-4ed2-8537-862684258c6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:08:11.438951 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6890416-d22e-4ed2-8537-862684258c6e None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:08:11.439235 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6890416-d22e-4ed2-8537-862684258c6e None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:08:11.439510 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6890416-d22e-4ed2-8537-862684258c6e None admin] PATH_INFO: `/v3/users/glance` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:08:11.446840 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6890416-d22e-4ed2-8537-862684258c6e None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:08:11.448281 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6890416-d22e-4ed2-8537-862684258c6e None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:08:11.449514 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10a5c9f3-61e4-49d4-9ee9-f65e4a6a23aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:08:11.450527 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a5c9f3-61e4-49d4-9ee9-f65e4a6a23aa None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:08:11.451030 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a5c9f3-61e4-49d4-9ee9-f65e4a6a23aa None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:08:11.451460 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a5c9f3-61e4-49d4-9ee9-f65e4a6a23aa None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:08:11.452388 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10a5c9f3-61e4-49d4-9ee9-f65e4a6a23aa None admin] RBAC: Adding query filter params (role.id=817d6837cd044890b41cf16b303a092b, scope.project.id=b9d94f6a68f7470d9494f112175620dc, user.id=893b0c0d85964b7dbc9705c0a091682c) {{(pid=81400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 03:08:11.453185 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10a5c9f3-61e4-49d4-9ee9-f65e4a6a23aa None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:08:11.454579 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10a5c9f3-61e4-49d4-9ee9-f65e4a6a23aa None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:08:11.459885 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 296/582] 10.222.0.18 () {56 vars in 1374 bytes} [Tue Sep 13 03:08:11 2022] GET /identity/v3/role_assignments?user.id=893b0c0d85964b7dbc9705c0a091682c&scope.project.id=b9d94f6a68f7470d9494f112175620dc&role.id=817d6837cd044890b41cf16b303a092b => generated 259 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 03:08:11.460246 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application [None req-f6890416-d22e-4ed2-8537-862684258c6e None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Sep 13 03:08:11.460246 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 03:08:11.460246 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 03:08:11.460246 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 03:08:11.460246 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 03:08:11.460246 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 03:08:11.460246 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 03:08:11.460246 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 03:08:11.460246 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 03:08:11.460246 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 03:08:11.460246 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 03:08:11.460246 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 03:08:11.460246 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 13 03:08:11.460246 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 13 03:08:11.460246 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 13 03:08:11.460246 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 13 03:08:11.461168 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 03:08:11.461168 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 03:08:11.461168 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 13 03:08:11.461168 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 13 03:08:11.461168 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 13 03:08:11.461168 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 13 03:08:11.461168 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 03:08:11.461168 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 03:08:11.461168 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 03:08:11.461168 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 03:08:11.461168 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 03:08:11.461168 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application with Lock( Sep 13 03:08:11.461168 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 03:08:11.461168 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._enter() Sep 13 03:08:11.461168 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 03:08:11.461168 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 03:08:11.461168 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 03:08:11.461168 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.creator() Sep 13 03:08:11.462161 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 03:08:11.462161 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application created_value = creator( Sep 13 03:08:11.462161 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 13 03:08:11.462161 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 13 03:08:11.462161 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 13 03:08:11.462161 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 13 03:08:11.462161 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 13 03:08:11.462161 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 13 03:08:11.462161 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Sep 13 03:08:11.462161 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Sep 13 03:08:11.462746 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10a5c9f3-61e4-49d4-9ee9-f65e4a6a23aa None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:08:11.462820 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 287/583] 10.222.0.18 () {56 vars in 1099 bytes} [Tue Sep 13 03:08:11 2022] GET /identity/v3/users/glance => generated 84 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 03:08:11.462820 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6890416-d22e-4ed2-8537-862684258c6e None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:08:11.579838 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6890416-d22e-4ed2-8537-862684258c6e None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:08:11.591078 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10a5c9f3-61e4-49d4-9ee9-f65e4a6a23aa None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:08:11.617867 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcfb5319-0e49-4694-94ff-d1ba151541a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:08:11.618515 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcfb5319-0e49-4694-94ff-d1ba151541a7 None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:08:11.618824 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcfb5319-0e49-4694-94ff-d1ba151541a7 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:08:11.619357 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcfb5319-0e49-4694-94ff-d1ba151541a7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:08:11.629310 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-491af10d-4b00-466d-8c04-24cf1ca22899 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:08:11.630317 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-491af10d-4b00-466d-8c04-24cf1ca22899 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:08:11.630604 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-491af10d-4b00-466d-8c04-24cf1ca22899 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:08:11.631145 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-491af10d-4b00-466d-8c04-24cf1ca22899 None admin] PATH_INFO: `/v3/projects` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:08:11.631757 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcfb5319-0e49-4694-94ff-d1ba151541a7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:08:11.631863 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-491af10d-4b00-466d-8c04-24cf1ca22899 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 03:08:11.632134 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-491af10d-4b00-466d-8c04-24cf1ca22899 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:08:11.633212 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcfb5319-0e49-4694-94ff-d1ba151541a7 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:08:11.633638 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-491af10d-4b00-466d-8c04-24cf1ca22899 None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:08:11.636678 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:08:11.650933 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 297/584] 10.222.0.18 () {56 vars in 1148 bytes} [Tue Sep 13 03:08:11 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 03:08:11.652534 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-491af10d-4b00-466d-8c04-24cf1ca22899 None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:08:11.653368 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application [None req-fcfb5319-0e49-4694-94ff-d1ba151541a7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 03:08:11.653368 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 03:08:11.653368 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 03:08:11.653368 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 03:08:11.653368 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 03:08:11.653368 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 03:08:11.653368 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 03:08:11.653368 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 03:08:11.653368 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 03:08:11.653368 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 03:08:11.653368 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 03:08:11.653368 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 03:08:11.653368 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 13 03:08:11.653368 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 13 03:08:11.653368 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 13 03:08:11.653368 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 13 03:08:11.654303 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 03:08:11.654303 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 03:08:11.654303 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 03:08:11.654303 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 03:08:11.654303 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 03:08:11.654303 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 03:08:11.654303 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 03:08:11.654303 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application with Lock( Sep 13 03:08:11.654303 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 03:08:11.654303 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._enter() Sep 13 03:08:11.654303 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 03:08:11.654303 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 03:08:11.654303 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 03:08:11.654303 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.creator() Sep 13 03:08:11.654303 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 03:08:11.654303 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application created_value = creator( Sep 13 03:08:11.654303 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 13 03:08:11.655794 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 13 03:08:11.655794 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 03:08:11.655794 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Sep 13 03:08:11.659190 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 288/585] 10.222.0.18 () {56 vars in 1108 bytes} [Tue Sep 13 03:08:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 03:08:11.659831 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcfb5319-0e49-4694-94ff-d1ba151541a7 None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:08:11.678266 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:08:11.766502 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-491af10d-4b00-466d-8c04-24cf1ca22899 None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:08:11.774916 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcfb5319-0e49-4694-94ff-d1ba151541a7 None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:08:11.810291 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37114846-5e0a-495c-bca9-66831d104475 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:08:11.810936 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37114846-5e0a-495c-bca9-66831d104475 None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:08:11.811217 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37114846-5e0a-495c-bca9-66831d104475 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:08:11.811505 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37114846-5e0a-495c-bca9-66831d104475 None admin] PATH_INFO: `/v3/users` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:08:11.812764 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37114846-5e0a-495c-bca9-66831d104475 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=81399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 03:08:11.812997 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f128562-14ea-4e36-9871-58b8e36ffebf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:08:11.813125 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37114846-5e0a-495c-bca9-66831d104475 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:08:11.813875 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f128562-14ea-4e36-9871-58b8e36ffebf None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:08:11.814299 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f128562-14ea-4e36-9871-58b8e36ffebf None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:08:11.814482 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37114846-5e0a-495c-bca9-66831d104475 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:08:11.814887 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f128562-14ea-4e36-9871-58b8e36ffebf None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:08:11.825188 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f128562-14ea-4e36-9871-58b8e36ffebf None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:08:11.827009 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f128562-14ea-4e36-9871-58b8e36ffebf None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:08:11.832839 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 289/586] 10.222.0.18 () {56 vars in 1137 bytes} [Tue Sep 13 03:08:11 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 03:08:11.833782 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37114846-5e0a-495c-bca9-66831d104475 None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:08:11.840876 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application [None req-2f128562-14ea-4e36-9871-58b8e36ffebf None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Sep 13 03:08:11.840876 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 03:08:11.840876 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 03:08:11.840876 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 03:08:11.840876 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 03:08:11.840876 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 03:08:11.840876 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 03:08:11.840876 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 03:08:11.840876 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 03:08:11.840876 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 03:08:11.840876 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 03:08:11.840876 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 03:08:11.840876 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 13 03:08:11.840876 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 13 03:08:11.840876 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 13 03:08:11.840876 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 13 03:08:11.842047 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 03:08:11.842047 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 03:08:11.842047 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 13 03:08:11.842047 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 13 03:08:11.842047 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 13 03:08:11.842047 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 13 03:08:11.842047 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 03:08:11.842047 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 03:08:11.842047 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 03:08:11.842047 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 03:08:11.842047 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 03:08:11.842047 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application with Lock( Sep 13 03:08:11.842047 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 03:08:11.842047 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._enter() Sep 13 03:08:11.842047 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 03:08:11.842047 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 03:08:11.842047 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 03:08:11.842047 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.creator() Sep 13 03:08:11.843341 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 03:08:11.843341 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application created_value = creator( Sep 13 03:08:11.843341 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 13 03:08:11.843341 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 13 03:08:11.843341 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 13 03:08:11.843341 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 13 03:08:11.843341 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 13 03:08:11.843341 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 13 03:08:11.843341 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Sep 13 03:08:11.843341 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Sep 13 03:08:11.846700 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 298/587] 10.222.0.18 () {56 vars in 1099 bytes} [Tue Sep 13 03:08:11 2022] GET /identity/v3/users/cinder => generated 84 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 03:08:11.847362 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f128562-14ea-4e36-9871-58b8e36ffebf None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:08:11.906049 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:08:11.915263 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:08:11.946388 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37114846-5e0a-495c-bca9-66831d104475 None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:08:11.955252 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f128562-14ea-4e36-9871-58b8e36ffebf None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:08:11.982019 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73a98dc4-c9a3-4252-a452-b5fb8b29cd20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:08:11.982783 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a98dc4-c9a3-4252-a452-b5fb8b29cd20 None admin] REQUEST_METHOD: `PUT` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:08:11.983065 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a98dc4-c9a3-4252-a452-b5fb8b29cd20 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:08:11.983336 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a98dc4-c9a3-4252-a452-b5fb8b29cd20 None admin] PATH_INFO: `/v3/projects/b9d94f6a68f7470d9494f112175620dc/users/c7377f5bee4a49959182e1bd34bc97a4/roles/817d6837cd044890b41cf16b303a092b` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:08:11.990378 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6a27e2f-fb96-42d1-a8aa-66cfc0e193da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:08:11.991012 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6a27e2f-fb96-42d1-a8aa-66cfc0e193da None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:08:11.991293 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6a27e2f-fb96-42d1-a8aa-66cfc0e193da None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:08:11.991567 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6a27e2f-fb96-42d1-a8aa-66cfc0e193da None admin] PATH_INFO: `/v3/domains` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:08:11.992232 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6a27e2f-fb96-42d1-a8aa-66cfc0e193da None admin] RBAC: Adding query filter params (name=Default) {{(pid=81400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 03:08:11.992540 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6a27e2f-fb96-42d1-a8aa-66cfc0e193da None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:08:11.993904 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6a27e2f-fb96-42d1-a8aa-66cfc0e193da None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:08:11.999249 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73a98dc4-c9a3-4252-a452-b5fb8b29cd20 None admin] RBAC: Authorizing `identity:create_grant(project_id=b9d94f6a68f7470d9494f112175620dc, user_id=c7377f5bee4a49959182e1bd34bc97a4, role_id=817d6837cd044890b41cf16b303a092b)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:08:12.001157 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73a98dc4-c9a3-4252-a452-b5fb8b29cd20 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:08:12.012170 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 299/588] 10.222.0.18 () {56 vars in 1109 bytes} [Tue Sep 13 03:08:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 03:08:12.012170 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6a27e2f-fb96-42d1-a8aa-66cfc0e193da None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:08:12.019386 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 290/589] 10.222.0.18 () {58 vars in 1439 bytes} [Tue Sep 13 03:08:11 2022] PUT /identity/v3/projects/b9d94f6a68f7470d9494f112175620dc/users/c7377f5bee4a49959182e1bd34bc97a4/roles/817d6837cd044890b41cf16b303a092b => generated 0 bytes in 186 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 03:08:12.020072 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73a98dc4-c9a3-4252-a452-b5fb8b29cd20 None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:08:12.152199 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73a98dc4-c9a3-4252-a452-b5fb8b29cd20 None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:08:12.182023 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:08:12.184719 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6a27e2f-fb96-42d1-a8aa-66cfc0e193da None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:08:12.197161 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11a29984-74ea-47a5-98c7-6ec77c5ac0eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:08:12.197922 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a29984-74ea-47a5-98c7-6ec77c5ac0eb None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:08:12.198985 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a29984-74ea-47a5-98c7-6ec77c5ac0eb None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:08:12.199357 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a29984-74ea-47a5-98c7-6ec77c5ac0eb None admin] PATH_INFO: `/v3/users` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:08:12.200191 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11a29984-74ea-47a5-98c7-6ec77c5ac0eb None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=81399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 03:08:12.200594 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11a29984-74ea-47a5-98c7-6ec77c5ac0eb None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:08:12.202202 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11a29984-74ea-47a5-98c7-6ec77c5ac0eb None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:08:12.209826 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:08:12.222758 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 291/590] 10.222.0.18 () {56 vars in 1137 bytes} [Tue Sep 13 03:08:12 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 03:08:12.227772 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11a29984-74ea-47a5-98c7-6ec77c5ac0eb None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:08:12.231266 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb7f799e-455d-4069-8e86-1746c78e8a05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:08:12.232737 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb7f799e-455d-4069-8e86-1746c78e8a05 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:08:12.233024 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb7f799e-455d-4069-8e86-1746c78e8a05 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:08:12.233304 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb7f799e-455d-4069-8e86-1746c78e8a05 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:08:12.250913 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb7f799e-455d-4069-8e86-1746c78e8a05 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:08:12.252827 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb7f799e-455d-4069-8e86-1746c78e8a05 None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:08:12.282024 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application [None req-eb7f799e-455d-4069-8e86-1746c78e8a05 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 03:08:12.282024 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 03:08:12.282024 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 03:08:12.282024 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 03:08:12.282024 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 03:08:12.282024 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 03:08:12.282024 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 03:08:12.282024 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 03:08:12.282024 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 03:08:12.282024 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 03:08:12.282024 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 03:08:12.282024 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 03:08:12.282024 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 13 03:08:12.282024 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 13 03:08:12.282024 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 13 03:08:12.282024 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 13 03:08:12.283277 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 03:08:12.283277 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 03:08:12.283277 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 03:08:12.283277 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 03:08:12.283277 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 03:08:12.283277 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 03:08:12.283277 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 03:08:12.283277 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application with Lock( Sep 13 03:08:12.283277 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 03:08:12.283277 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._enter() Sep 13 03:08:12.283277 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 03:08:12.283277 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 03:08:12.283277 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 03:08:12.283277 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.creator() Sep 13 03:08:12.283277 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 03:08:12.283277 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application created_value = creator( Sep 13 03:08:12.283277 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 13 03:08:12.284508 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 13 03:08:12.284508 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 03:08:12.284508 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Sep 13 03:08:12.284508 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 300/591] 10.222.0.18 () {56 vars in 1108 bytes} [Tue Sep 13 03:08:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 275 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 03:08:12.284508 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb7f799e-455d-4069-8e86-1746c78e8a05 None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:08:12.347495 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11a29984-74ea-47a5-98c7-6ec77c5ac0eb None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:08:12.384791 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da3b7a42-41c3-4160-a383-4e83fb30d899 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:08:12.385722 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da3b7a42-41c3-4160-a383-4e83fb30d899 None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:08:12.386167 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da3b7a42-41c3-4160-a383-4e83fb30d899 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:08:12.386583 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da3b7a42-41c3-4160-a383-4e83fb30d899 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:08:12.394863 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da3b7a42-41c3-4160-a383-4e83fb30d899 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:08:12.396525 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da3b7a42-41c3-4160-a383-4e83fb30d899 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:08:12.407590 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb7f799e-455d-4069-8e86-1746c78e8a05 None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:08:12.410967 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application [None req-da3b7a42-41c3-4160-a383-4e83fb30d899 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Sep 13 03:08:12.410967 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 03:08:12.410967 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 03:08:12.410967 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 03:08:12.410967 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 03:08:12.410967 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 03:08:12.410967 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 03:08:12.410967 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 03:08:12.410967 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 03:08:12.410967 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 03:08:12.410967 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 03:08:12.410967 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 03:08:12.410967 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 13 03:08:12.410967 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 13 03:08:12.410967 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 13 03:08:12.410967 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 13 03:08:12.412110 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 03:08:12.412110 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 03:08:12.412110 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 13 03:08:12.412110 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 13 03:08:12.412110 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 13 03:08:12.412110 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 13 03:08:12.412110 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 03:08:12.412110 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 03:08:12.412110 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 03:08:12.412110 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 03:08:12.412110 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 03:08:12.412110 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application with Lock( Sep 13 03:08:12.412110 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 03:08:12.412110 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._enter() Sep 13 03:08:12.412110 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 03:08:12.412110 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 03:08:12.412110 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 03:08:12.412110 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.creator() Sep 13 03:08:12.413235 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 03:08:12.413235 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application created_value = creator( Sep 13 03:08:12.413235 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 13 03:08:12.413235 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 13 03:08:12.413235 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 13 03:08:12.413235 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 13 03:08:12.413235 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 13 03:08:12.413235 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 13 03:08:12.413235 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Sep 13 03:08:12.413235 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Sep 13 03:08:12.415536 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 292/592] 10.222.0.18 () {56 vars in 1102 bytes} [Tue Sep 13 03:08:12 2022] GET /identity/v3/users/neutron => generated 85 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 03:08:12.416474 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da3b7a42-41c3-4160-a383-4e83fb30d899 None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:08:12.449203 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b47dcc7-01b8-4833-8743-b2449c9b4f02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:08:12.449750 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b47dcc7-01b8-4833-8743-b2449c9b4f02 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:08:12.450029 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b47dcc7-01b8-4833-8743-b2449c9b4f02 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:08:12.450300 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b47dcc7-01b8-4833-8743-b2449c9b4f02 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:08:12.461814 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b47dcc7-01b8-4833-8743-b2449c9b4f02 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:08:12.463312 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b47dcc7-01b8-4833-8743-b2449c9b4f02 None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:08:12.465741 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:08:12.480440 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application [None req-0b47dcc7-01b8-4833-8743-b2449c9b4f02 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 03:08:12.480440 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 03:08:12.480440 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 03:08:12.480440 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 03:08:12.480440 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 03:08:12.480440 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 03:08:12.480440 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 03:08:12.480440 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 03:08:12.480440 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 03:08:12.480440 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 03:08:12.480440 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 03:08:12.480440 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 03:08:12.480440 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 13 03:08:12.480440 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 13 03:08:12.480440 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 13 03:08:12.480440 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 13 03:08:12.481620 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 03:08:12.481620 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 03:08:12.481620 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 03:08:12.481620 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 03:08:12.481620 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 03:08:12.481620 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 03:08:12.481620 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 03:08:12.481620 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application with Lock( Sep 13 03:08:12.481620 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 03:08:12.481620 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._enter() Sep 13 03:08:12.481620 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 03:08:12.481620 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 03:08:12.481620 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 03:08:12.481620 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.creator() Sep 13 03:08:12.481620 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 03:08:12.481620 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application created_value = creator( Sep 13 03:08:12.481620 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 13 03:08:12.482806 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 13 03:08:12.482806 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 03:08:12.482806 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Sep 13 03:08:12.486928 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 301/593] 10.222.0.18 () {56 vars in 1108 bytes} [Tue Sep 13 03:08:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 03:08:12.487768 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b47dcc7-01b8-4833-8743-b2449c9b4f02 None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:08:12.512299 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:08:12.533626 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da3b7a42-41c3-4160-a383-4e83fb30d899 None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:08:12.570993 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-265b4787-eba7-4bdb-9c9c-e5dd29e2880e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:08:12.571876 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-265b4787-eba7-4bdb-9c9c-e5dd29e2880e None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:08:12.572398 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-265b4787-eba7-4bdb-9c9c-e5dd29e2880e None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:08:12.572815 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-265b4787-eba7-4bdb-9c9c-e5dd29e2880e None admin] PATH_INFO: `/v3/domains` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:08:12.573595 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-265b4787-eba7-4bdb-9c9c-e5dd29e2880e None admin] RBAC: Adding query filter params (name=Default) {{(pid=81399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 03:08:12.574071 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-265b4787-eba7-4bdb-9c9c-e5dd29e2880e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:08:12.575760 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-265b4787-eba7-4bdb-9c9c-e5dd29e2880e None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:08:12.589770 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 293/594] 10.222.0.18 () {56 vars in 1109 bytes} [Tue Sep 13 03:08:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 03:08:12.590888 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-265b4787-eba7-4bdb-9c9c-e5dd29e2880e None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:08:12.604422 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b47dcc7-01b8-4833-8743-b2449c9b4f02 None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:08:12.641888 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b5a3072-ac6f-4396-b077-8f7e8fccff56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:08:12.642257 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b5a3072-ac6f-4396-b077-8f7e8fccff56 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:08:12.642540 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b5a3072-ac6f-4396-b077-8f7e8fccff56 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:08:12.642858 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b5a3072-ac6f-4396-b077-8f7e8fccff56 None admin] PATH_INFO: `/v3/users` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:08:12.643594 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b5a3072-ac6f-4396-b077-8f7e8fccff56 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=81400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 03:08:12.643935 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b5a3072-ac6f-4396-b077-8f7e8fccff56 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:08:12.645482 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b5a3072-ac6f-4396-b077-8f7e8fccff56 None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:08:12.683347 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 302/595] 10.222.0.18 () {56 vars in 1139 bytes} [Tue Sep 13 03:08:12 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 03:08:12.683996 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b5a3072-ac6f-4396-b077-8f7e8fccff56 None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:08:12.700156 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-265b4787-eba7-4bdb-9c9c-e5dd29e2880e None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:08:12.703229 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:08:12.753900 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d47928b-91fe-4db1-ba1e-84dc85c34cce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:08:12.754624 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d47928b-91fe-4db1-ba1e-84dc85c34cce None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:08:12.754912 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d47928b-91fe-4db1-ba1e-84dc85c34cce None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:08:12.755182 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d47928b-91fe-4db1-ba1e-84dc85c34cce None admin] PATH_INFO: `/v3/domains` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:08:12.755812 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d47928b-91fe-4db1-ba1e-84dc85c34cce None admin] RBAC: Adding query filter params (name=Default) {{(pid=81399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 03:08:12.756109 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d47928b-91fe-4db1-ba1e-84dc85c34cce None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:08:12.757535 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d47928b-91fe-4db1-ba1e-84dc85c34cce None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:08:12.771640 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:08:12.772292 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 294/596] 10.222.0.18 () {56 vars in 1109 bytes} [Tue Sep 13 03:08:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 03:08:12.773305 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d47928b-91fe-4db1-ba1e-84dc85c34cce None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:08:12.800065 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b5a3072-ac6f-4396-b077-8f7e8fccff56 None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:08:12.836266 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc2432c6-f77e-4f74-b006-9708efd3f32e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:08:12.836895 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc2432c6-f77e-4f74-b006-9708efd3f32e None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:08:12.837178 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc2432c6-f77e-4f74-b006-9708efd3f32e None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:08:12.837449 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc2432c6-f77e-4f74-b006-9708efd3f32e None admin] PATH_INFO: `/v3/projects/service` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:08:12.846706 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc2432c6-f77e-4f74-b006-9708efd3f32e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:08:12.848771 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc2432c6-f77e-4f74-b006-9708efd3f32e None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:08:12.863480 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application [None req-dc2432c6-f77e-4f74-b006-9708efd3f32e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 13 03:08:12.863480 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 03:08:12.863480 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 03:08:12.863480 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 03:08:12.863480 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 03:08:12.863480 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 03:08:12.863480 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 03:08:12.863480 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 03:08:12.863480 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 03:08:12.863480 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 03:08:12.863480 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 03:08:12.863480 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 03:08:12.863480 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 13 03:08:12.863480 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 13 03:08:12.863480 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 13 03:08:12.864574 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 13 03:08:12.864574 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 03:08:12.864574 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 03:08:12.864574 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 03:08:12.864574 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 03:08:12.864574 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 03:08:12.864574 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 03:08:12.864574 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 03:08:12.864574 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application with Lock( Sep 13 03:08:12.864574 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 03:08:12.864574 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._enter() Sep 13 03:08:12.864574 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 03:08:12.864574 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 03:08:12.864574 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 03:08:12.864574 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.creator() Sep 13 03:08:12.864574 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 03:08:12.864574 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application created_value = creator( Sep 13 03:08:12.865617 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 13 03:08:12.865617 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 13 03:08:12.865617 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 13 03:08:12.865617 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 13 03:08:12.865617 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 13 03:08:12.865617 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 13 03:08:12.865617 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 13 03:08:12.865617 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Sep 13 03:08:12.872925 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 303/597] 10.222.0.18 () {56 vars in 1111 bytes} [Tue Sep 13 03:08:12 2022] GET /identity/v3/projects/service => generated 88 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 03:08:12.873873 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc2432c6-f77e-4f74-b006-9708efd3f32e None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:08:12.903573 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d47928b-91fe-4db1-ba1e-84dc85c34cce None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:08:12.948371 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05b2fc1e-64d0-4fc8-a173-9f95335f4dba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:08:12.950041 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b2fc1e-64d0-4fc8-a173-9f95335f4dba None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:08:12.950549 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b2fc1e-64d0-4fc8-a173-9f95335f4dba None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:08:12.951008 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b2fc1e-64d0-4fc8-a173-9f95335f4dba None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:08:12.968010 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05b2fc1e-64d0-4fc8-a173-9f95335f4dba None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:08:12.969795 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05b2fc1e-64d0-4fc8-a173-9f95335f4dba None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:08:12.988601 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application [None req-05b2fc1e-64d0-4fc8-a173-9f95335f4dba None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 03:08:12.988601 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 03:08:12.988601 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 03:08:12.988601 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 03:08:12.988601 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 03:08:12.988601 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 03:08:12.988601 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 03:08:12.988601 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 03:08:12.988601 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 03:08:12.988601 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 03:08:12.988601 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 03:08:12.988601 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 03:08:12.988601 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 13 03:08:12.988601 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 13 03:08:12.988601 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 13 03:08:12.988601 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 13 03:08:12.989846 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 03:08:12.989846 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 03:08:12.989846 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 03:08:12.989846 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 03:08:12.989846 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 03:08:12.989846 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 03:08:12.989846 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 03:08:12.989846 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application with Lock( Sep 13 03:08:12.989846 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 03:08:12.989846 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._enter() Sep 13 03:08:12.989846 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 03:08:12.989846 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 03:08:12.989846 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 03:08:12.989846 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.creator() Sep 13 03:08:12.989846 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 03:08:12.989846 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application created_value = creator( Sep 13 03:08:12.989846 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 13 03:08:12.991081 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 13 03:08:12.991081 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 03:08:12.991081 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Sep 13 03:08:12.993009 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 295/598] 10.222.0.18 () {56 vars in 1108 bytes} [Tue Sep 13 03:08:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 03:08:12.993982 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05b2fc1e-64d0-4fc8-a173-9f95335f4dba None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:08:13.008648 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc2432c6-f77e-4f74-b006-9708efd3f32e None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:08:13.017663 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:08:13.049558 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e8d2ea4-3a30-46cf-9c80-32c42025aafe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:08:13.050510 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e8d2ea4-3a30-46cf-9c80-32c42025aafe None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:08:13.050948 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e8d2ea4-3a30-46cf-9c80-32c42025aafe None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:08:13.051450 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e8d2ea4-3a30-46cf-9c80-32c42025aafe None admin] PATH_INFO: `/v3/projects/service` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:08:13.059082 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e8d2ea4-3a30-46cf-9c80-32c42025aafe None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:08:13.061440 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e8d2ea4-3a30-46cf-9c80-32c42025aafe None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:08:13.063984 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:08:13.077909 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application [None req-4e8d2ea4-3a30-46cf-9c80-32c42025aafe None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 13 03:08:13.077909 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 03:08:13.077909 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 03:08:13.077909 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 03:08:13.077909 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 03:08:13.077909 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 03:08:13.077909 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 03:08:13.077909 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 03:08:13.077909 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 03:08:13.077909 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 03:08:13.077909 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 03:08:13.077909 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 03:08:13.077909 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 13 03:08:13.077909 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 13 03:08:13.077909 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 13 03:08:13.078968 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 13 03:08:13.078968 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 03:08:13.078968 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 03:08:13.078968 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 03:08:13.078968 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 03:08:13.078968 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 03:08:13.078968 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 03:08:13.078968 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 03:08:13.078968 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application with Lock( Sep 13 03:08:13.078968 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 03:08:13.078968 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._enter() Sep 13 03:08:13.078968 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 03:08:13.078968 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 03:08:13.078968 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 03:08:13.078968 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.creator() Sep 13 03:08:13.078968 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 03:08:13.078968 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application created_value = creator( Sep 13 03:08:13.080064 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 13 03:08:13.080064 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 13 03:08:13.080064 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 13 03:08:13.080064 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 13 03:08:13.080064 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 13 03:08:13.080064 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 13 03:08:13.080064 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 13 03:08:13.080064 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Sep 13 03:08:13.082092 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 304/599] 10.222.0.18 () {56 vars in 1111 bytes} [Tue Sep 13 03:08:12 2022] GET /identity/v3/projects/service => generated 88 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 03:08:13.083024 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e8d2ea4-3a30-46cf-9c80-32c42025aafe None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:08:13.121910 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05b2fc1e-64d0-4fc8-a173-9f95335f4dba None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:08:13.158312 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c10ed565-ed6c-4538-89e4-b62ef91a4201 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:08:13.159108 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c10ed565-ed6c-4538-89e4-b62ef91a4201 None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:08:13.159584 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c10ed565-ed6c-4538-89e4-b62ef91a4201 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:08:13.159968 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c10ed565-ed6c-4538-89e4-b62ef91a4201 None admin] PATH_INFO: `/v3/projects` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:08:13.160770 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c10ed565-ed6c-4538-89e4-b62ef91a4201 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 03:08:13.161189 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c10ed565-ed6c-4538-89e4-b62ef91a4201 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:08:13.162775 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c10ed565-ed6c-4538-89e4-b62ef91a4201 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:08:13.180417 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 296/600] 10.222.0.18 () {56 vars in 1148 bytes} [Tue Sep 13 03:08:12 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 03:08:13.181447 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c10ed565-ed6c-4538-89e4-b62ef91a4201 None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:08:13.197345 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e8d2ea4-3a30-46cf-9c80-32c42025aafe None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:08:13.234931 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ab1db87-cf3a-4146-8415-56d45a052166 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:08:13.235941 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ab1db87-cf3a-4146-8415-56d45a052166 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:08:13.236373 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ab1db87-cf3a-4146-8415-56d45a052166 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:08:13.236779 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ab1db87-cf3a-4146-8415-56d45a052166 None admin] PATH_INFO: `/v3/domains` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:08:13.237650 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ab1db87-cf3a-4146-8415-56d45a052166 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 03:08:13.238079 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ab1db87-cf3a-4146-8415-56d45a052166 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:08:13.239754 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ab1db87-cf3a-4146-8415-56d45a052166 None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:08:13.255141 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 305/601] 10.222.0.18 () {56 vars in 1109 bytes} [Tue Sep 13 03:08:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 03:08:13.256160 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ab1db87-cf3a-4146-8415-56d45a052166 None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:08:13.267636 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:08:13.306077 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c10ed565-ed6c-4538-89e4-b62ef91a4201 None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:08:13.329401 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:08:13.354010 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e601031-5c09-4541-be86-f92cb52979e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:08:13.355155 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e601031-5c09-4541-be86-f92cb52979e2 None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:08:13.355573 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e601031-5c09-4541-be86-f92cb52979e2 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:08:13.356430 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e601031-5c09-4541-be86-f92cb52979e2 None admin] PATH_INFO: `/v3/projects` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:08:13.357519 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e601031-5c09-4541-be86-f92cb52979e2 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 03:08:13.357964 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e601031-5c09-4541-be86-f92cb52979e2 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:08:13.360762 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e601031-5c09-4541-be86-f92cb52979e2 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:08:13.376662 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ab1db87-cf3a-4146-8415-56d45a052166 None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:08:13.377392 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 297/602] 10.222.0.18 () {56 vars in 1148 bytes} [Tue Sep 13 03:08:13 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 03:08:13.378359 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e601031-5c09-4541-be86-f92cb52979e2 None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:08:13.419807 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-833a77b2-c401-4942-ab69-5d66357911bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:08:13.420627 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-833a77b2-c401-4942-ab69-5d66357911bd None admin] REQUEST_METHOD: `PUT` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:08:13.421614 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-833a77b2-c401-4942-ab69-5d66357911bd None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:08:13.421974 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-833a77b2-c401-4942-ab69-5d66357911bd None admin] PATH_INFO: `/v3/projects/b9d94f6a68f7470d9494f112175620dc/users/b5fdc2ff46a14638b409a13e73ba5500/roles/817d6837cd044890b41cf16b303a092b` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:08:13.438052 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-833a77b2-c401-4942-ab69-5d66357911bd None admin] RBAC: Authorizing `identity:create_grant(project_id=b9d94f6a68f7470d9494f112175620dc, user_id=b5fdc2ff46a14638b409a13e73ba5500, role_id=817d6837cd044890b41cf16b303a092b)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:08:13.439744 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-833a77b2-c401-4942-ab69-5d66357911bd None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:08:13.458829 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 306/603] 10.222.0.18 () {58 vars in 1439 bytes} [Tue Sep 13 03:08:13 2022] PUT /identity/v3/projects/b9d94f6a68f7470d9494f112175620dc/users/b5fdc2ff46a14638b409a13e73ba5500/roles/817d6837cd044890b41cf16b303a092b => generated 0 bytes in 198 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 03:08:13.460154 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-833a77b2-c401-4942-ab69-5d66357911bd None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:08:13.503961 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e601031-5c09-4541-be86-f92cb52979e2 None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:08:13.552009 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86104f31-91b0-407b-8af0-3e70475014f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:08:13.553065 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86104f31-91b0-407b-8af0-3e70475014f5 None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:08:13.553490 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86104f31-91b0-407b-8af0-3e70475014f5 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:08:13.553895 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86104f31-91b0-407b-8af0-3e70475014f5 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:08:13.556651 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:08:13.566776 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86104f31-91b0-407b-8af0-3e70475014f5 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:08:13.568463 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86104f31-91b0-407b-8af0-3e70475014f5 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:08:13.582924 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application [None req-86104f31-91b0-407b-8af0-3e70475014f5 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 13 03:08:13.582924 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 03:08:13.582924 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 03:08:13.582924 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 03:08:13.582924 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 03:08:13.582924 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 03:08:13.582924 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 03:08:13.582924 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 03:08:13.582924 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 03:08:13.582924 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 03:08:13.582924 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 03:08:13.582924 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 03:08:13.582924 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 13 03:08:13.582924 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 13 03:08:13.582924 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 13 03:08:13.584064 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 13 03:08:13.584064 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 03:08:13.584064 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 03:08:13.584064 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 03:08:13.584064 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 03:08:13.584064 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 03:08:13.584064 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 03:08:13.584064 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 03:08:13.584064 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application with Lock( Sep 13 03:08:13.584064 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 03:08:13.584064 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._enter() Sep 13 03:08:13.584064 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 03:08:13.584064 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 03:08:13.584064 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 03:08:13.584064 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.creator() Sep 13 03:08:13.584064 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 03:08:13.584064 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application created_value = creator( Sep 13 03:08:13.585160 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 13 03:08:13.585160 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 13 03:08:13.585160 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 13 03:08:13.585160 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 13 03:08:13.585160 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 13 03:08:13.585160 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 13 03:08:13.585160 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 13 03:08:13.585160 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Sep 13 03:08:13.587251 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 298/604] 10.222.0.18 () {56 vars in 1111 bytes} [Tue Sep 13 03:08:13 2022] GET /identity/v3/projects/service => generated 88 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 03:08:13.598662 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86104f31-91b0-407b-8af0-3e70475014f5 None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:08:13.662563 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-833a77b2-c401-4942-ab69-5d66357911bd None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:08:13.674026 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:08:13.707693 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99ea4380-8dc4-413c-bfaf-3ab2a9a7b321 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:08:13.708534 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99ea4380-8dc4-413c-bfaf-3ab2a9a7b321 None admin] REQUEST_METHOD: `PUT` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:08:13.708715 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99ea4380-8dc4-413c-bfaf-3ab2a9a7b321 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:08:13.708989 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99ea4380-8dc4-413c-bfaf-3ab2a9a7b321 None admin] PATH_INFO: `/v3/projects/b9d94f6a68f7470d9494f112175620dc/users/924ee3f608d34a18b0ddc4695f102a56/roles/817d6837cd044890b41cf16b303a092b` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:08:13.730497 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99ea4380-8dc4-413c-bfaf-3ab2a9a7b321 None admin] RBAC: Authorizing `identity:create_grant(project_id=b9d94f6a68f7470d9494f112175620dc, user_id=924ee3f608d34a18b0ddc4695f102a56, role_id=817d6837cd044890b41cf16b303a092b)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:08:13.732976 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99ea4380-8dc4-413c-bfaf-3ab2a9a7b321 None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:08:13.752922 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 307/605] 10.222.0.18 () {58 vars in 1439 bytes} [Tue Sep 13 03:08:13 2022] PUT /identity/v3/projects/b9d94f6a68f7470d9494f112175620dc/users/924ee3f608d34a18b0ddc4695f102a56/roles/817d6837cd044890b41cf16b303a092b => generated 0 bytes in 294 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 03:08:13.754203 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e370032-be6d-40f0-a848-0453c1bf53e4 None None] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:08:13.754424 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e370032-be6d-40f0-a848-0453c1bf53e4 None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:08:13.755543 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e370032-be6d-40f0-a848-0453c1bf53e4 None None] PATH_INFO: `/` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:08:13.757989 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 308/606] 10.222.0.18 () {54 vars in 927 bytes} [Tue Sep 13 03:08:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 03:08:13.764625 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e9a4a1b-f859-42d8-86f7-6dd6740d76e5 None None] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:08:13.765001 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e9a4a1b-f859-42d8-86f7-6dd6740d76e5 None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:08:13.765254 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e9a4a1b-f859-42d8-86f7-6dd6740d76e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:08:13.766918 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86104f31-91b0-407b-8af0-3e70475014f5 None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:08:13.786277 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.auth.core [None req-2e9a4a1b-f859-42d8-86f7-6dd6740d76e5 None None] MFA Rules not processed for user `49f280ab086b4610855c4ffa378517d9`. Rule list: `[]` (Enabled: `True`). {{(pid=81400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 03:08:13.787910 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-2e9a4a1b-f859-42d8-86f7-6dd6740d76e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:08:13.809157 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15504f42-6908-430b-83f6-ef9b007363f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:08:13.809945 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15504f42-6908-430b-83f6-ef9b007363f8 None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:08:13.810235 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15504f42-6908-430b-83f6-ef9b007363f8 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:08:13.810499 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15504f42-6908-430b-83f6-ef9b007363f8 None admin] PATH_INFO: `/v3/projects` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:08:13.811243 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15504f42-6908-430b-83f6-ef9b007363f8 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 03:08:13.811542 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15504f42-6908-430b-83f6-ef9b007363f8 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:08:13.813277 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15504f42-6908-430b-83f6-ef9b007363f8 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:08:13.829682 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 299/607] 10.222.0.18 () {56 vars in 1148 bytes} [Tue Sep 13 03:08:13 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 03:08:13.834577 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15504f42-6908-430b-83f6-ef9b007363f8 None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:08:13.871991 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 309/608] 10.222.0.18 () {58 vars in 1024 bytes} [Tue Sep 13 03:08:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 13 03:08:13.929674 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:08:13.979352 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15504f42-6908-430b-83f6-ef9b007363f8 None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:08:14.016452 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3e3bd0c-ac73-4132-81d3-db618b661124 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:08:14.017038 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e3bd0c-ac73-4132-81d3-db618b661124 None admin] REQUEST_METHOD: `PUT` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:08:14.017310 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e3bd0c-ac73-4132-81d3-db618b661124 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:08:14.017577 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e3bd0c-ac73-4132-81d3-db618b661124 None admin] PATH_INFO: `/v3/projects/b9d94f6a68f7470d9494f112175620dc/users/cdffbc851086441ebd01022d1e4c2609/roles/817d6837cd044890b41cf16b303a092b` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:08:14.034770 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3e3bd0c-ac73-4132-81d3-db618b661124 None admin] RBAC: Authorizing `identity:create_grant(project_id=b9d94f6a68f7470d9494f112175620dc, user_id=cdffbc851086441ebd01022d1e4c2609, role_id=817d6837cd044890b41cf16b303a092b)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:08:14.035613 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3e3bd0c-ac73-4132-81d3-db618b661124 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:08:14.053431 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 300/609] 10.222.0.18 () {58 vars in 1439 bytes} [Tue Sep 13 03:08:13 2022] PUT /identity/v3/projects/b9d94f6a68f7470d9494f112175620dc/users/cdffbc851086441ebd01022d1e4c2609/roles/817d6837cd044890b41cf16b303a092b => generated 0 bytes in 219 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 03:08:14.201470 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e9a4a1b-f859-42d8-86f7-6dd6740d76e5 None None] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:08:14.312164 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:08:14.325215 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e9a4a1b-f859-42d8-86f7-6dd6740d76e5 None None] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:08:14.364344 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1fded6d-353a-40a9-9f2e-4c5d2011023c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:08:14.364979 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1fded6d-353a-40a9-9f2e-4c5d2011023c None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:08:14.365263 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1fded6d-353a-40a9-9f2e-4c5d2011023c None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:08:14.365532 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1fded6d-353a-40a9-9f2e-4c5d2011023c None admin] PATH_INFO: `/v3/roles/service` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:08:14.376285 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1fded6d-353a-40a9-9f2e-4c5d2011023c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 03:08:14.380480 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f88938b-21ea-444f-9673-3f4e939384e4 None None] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:08:14.381036 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f88938b-21ea-444f-9673-3f4e939384e4 None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:08:14.381524 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f88938b-21ea-444f-9673-3f4e939384e4 None None] PATH_INFO: `/` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:08:14.381749 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application [None req-c1fded6d-353a-40a9-9f2e-4c5d2011023c None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 13 03:08:14.381749 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 03:08:14.381749 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 03:08:14.381749 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 03:08:14.381749 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 03:08:14.381749 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 03:08:14.381749 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 03:08:14.381749 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 03:08:14.381749 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 03:08:14.381749 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 03:08:14.381749 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 03:08:14.381749 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 03:08:14.381749 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 03:08:14.381749 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 03:08:14.381749 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 03:08:14.381749 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 03:08:14.383269 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 03:08:14.383269 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 03:08:14.383269 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 03:08:14.383269 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 03:08:14.383269 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 03:08:14.383269 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 03:08:14.383269 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 03:08:14.383269 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 03:08:14.383269 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 03:08:14.383269 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 03:08:14.383269 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 03:08:14.383269 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application with Lock( Sep 13 03:08:14.383269 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 03:08:14.383269 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._enter() Sep 13 03:08:14.383269 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 03:08:14.383269 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 03:08:14.384164 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 03:08:14.384164 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.creator() Sep 13 03:08:14.384164 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 03:08:14.384164 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application created_value = creator( Sep 13 03:08:14.384164 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 03:08:14.384164 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 03:08:14.384164 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 03:08:14.384164 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 03:08:14.384164 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 03:08:14.384164 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 03:08:14.384164 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 13 03:08:14.384164 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Sep 13 03:08:14.384164 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 310/610] 10.222.0.18 () {56 vars in 1102 bytes} [Tue Sep 13 03:08:14 2022] GET /identity/v3/roles/service => generated 85 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 03:08:14.387376 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 301/611] 10.222.0.18 () {54 vars in 927 bytes} [Tue Sep 13 03:08:14 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 13 03:08:14.390743 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1fded6d-353a-40a9-9f2e-4c5d2011023c None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:08:14.393881 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45dcd9bf-e6e1-4a43-8c5b-51b74d68ccf8 None None] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:08:14.394326 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45dcd9bf-e6e1-4a43-8c5b-51b74d68ccf8 None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:08:14.396393 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45dcd9bf-e6e1-4a43-8c5b-51b74d68ccf8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:08:14.421295 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.auth.core [None req-45dcd9bf-e6e1-4a43-8c5b-51b74d68ccf8 None None] MFA Rules not processed for user `49f280ab086b4610855c4ffa378517d9`. Rule list: `[]` (Enabled: `True`). {{(pid=81399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 03:08:14.422807 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-45dcd9bf-e6e1-4a43-8c5b-51b74d68ccf8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:08:14.517730 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 302/612] 10.222.0.18 () {58 vars in 1024 bytes} [Tue Sep 13 03:08:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 13 03:08:14.536011 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1fded6d-353a-40a9-9f2e-4c5d2011023c None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:08:14.590454 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fedae8b8-2648-4a63-9467-44e143cf70ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:08:14.591013 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fedae8b8-2648-4a63-9467-44e143cf70ac None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:08:14.591292 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fedae8b8-2648-4a63-9467-44e143cf70ac None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:08:14.591564 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fedae8b8-2648-4a63-9467-44e143cf70ac None admin] PATH_INFO: `/v3/roles` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:08:14.592232 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fedae8b8-2648-4a63-9467-44e143cf70ac None admin] RBAC: Adding query filter params (name=service) {{(pid=81400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 03:08:14.592535 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fedae8b8-2648-4a63-9467-44e143cf70ac None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:08:14.594449 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fedae8b8-2648-4a63-9467-44e143cf70ac None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:08:14.603066 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 311/613] 10.222.0.18 () {56 vars in 1103 bytes} [Tue Sep 13 03:08:14 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 03:08:14.609039 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45dcd9bf-e6e1-4a43-8c5b-51b74d68ccf8 None None] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:08:14.670863 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:08:14.759339 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45dcd9bf-e6e1-4a43-8c5b-51b74d68ccf8 None None] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:08:14.806396 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e963cc8-05f9-41f4-9b2f-46f665920ee7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:08:14.807156 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e963cc8-05f9-41f4-9b2f-46f665920ee7 None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:08:14.807355 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e963cc8-05f9-41f4-9b2f-46f665920ee7 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:08:14.807585 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e963cc8-05f9-41f4-9b2f-46f665920ee7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:08:14.817482 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e963cc8-05f9-41f4-9b2f-46f665920ee7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:08:14.818919 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e963cc8-05f9-41f4-9b2f-46f665920ee7 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:08:14.833578 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application [None req-8e963cc8-05f9-41f4-9b2f-46f665920ee7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 03:08:14.833578 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 03:08:14.833578 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 03:08:14.833578 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 03:08:14.833578 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 03:08:14.833578 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 03:08:14.833578 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 03:08:14.833578 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 03:08:14.833578 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 03:08:14.833578 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 03:08:14.833578 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 03:08:14.833578 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 03:08:14.833578 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 13 03:08:14.833578 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 13 03:08:14.833578 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 13 03:08:14.833578 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 13 03:08:14.834538 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 03:08:14.834538 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 03:08:14.834538 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 03:08:14.834538 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 03:08:14.834538 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 03:08:14.834538 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 03:08:14.834538 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 03:08:14.834538 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application with Lock( Sep 13 03:08:14.834538 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 03:08:14.834538 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._enter() Sep 13 03:08:14.834538 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 03:08:14.834538 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 03:08:14.834538 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 03:08:14.834538 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.creator() Sep 13 03:08:14.834538 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 03:08:14.834538 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application created_value = creator( Sep 13 03:08:14.834538 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 13 03:08:14.835480 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 13 03:08:14.835480 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 03:08:14.835480 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Sep 13 03:08:14.835480 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 303/614] 10.222.0.18 () {56 vars in 1108 bytes} [Tue Sep 13 03:08:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 227 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 03:08:14.843761 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fedae8b8-2648-4a63-9467-44e143cf70ac None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:08:14.844218 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e963cc8-05f9-41f4-9b2f-46f665920ee7 None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:08:14.865183 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:08:14.958937 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fedae8b8-2648-4a63-9467-44e143cf70ac None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:08:14.999653 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8457c38b-e2ad-4adf-8347-5c979f9d1630 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:08:15.000719 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8457c38b-e2ad-4adf-8347-5c979f9d1630 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:08:15.001212 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8457c38b-e2ad-4adf-8347-5c979f9d1630 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:08:15.001709 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8457c38b-e2ad-4adf-8347-5c979f9d1630 None admin] PATH_INFO: `/v3/domains` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:08:15.002563 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8457c38b-e2ad-4adf-8347-5c979f9d1630 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 03:08:15.003106 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8457c38b-e2ad-4adf-8347-5c979f9d1630 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:08:15.004684 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8457c38b-e2ad-4adf-8347-5c979f9d1630 None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:08:15.019020 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 312/615] 10.222.0.18 () {56 vars in 1109 bytes} [Tue Sep 13 03:08:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 03:08:15.026080 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8457c38b-e2ad-4adf-8347-5c979f9d1630 None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:08:15.055352 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:08:15.083342 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e963cc8-05f9-41f4-9b2f-46f665920ee7 None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:08:15.128187 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:08:15.173447 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6db59224-ca6f-4ab8-913b-cf1eda560e5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:08:15.175332 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6db59224-ca6f-4ab8-913b-cf1eda560e5a None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:08:15.175854 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6db59224-ca6f-4ab8-913b-cf1eda560e5a None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:08:15.177030 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6db59224-ca6f-4ab8-913b-cf1eda560e5a None admin] PATH_INFO: `/v3/roles/service` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:08:15.180323 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8457c38b-e2ad-4adf-8347-5c979f9d1630 None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:08:15.200564 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6db59224-ca6f-4ab8-913b-cf1eda560e5a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 03:08:15.208178 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application [None req-6db59224-ca6f-4ab8-913b-cf1eda560e5a None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 13 03:08:15.208178 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 03:08:15.208178 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 03:08:15.208178 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 03:08:15.208178 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 03:08:15.208178 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 03:08:15.208178 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 03:08:15.208178 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 03:08:15.208178 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 03:08:15.208178 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 03:08:15.208178 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 03:08:15.208178 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 03:08:15.208178 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 03:08:15.208178 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 03:08:15.208178 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 03:08:15.208178 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 03:08:15.209099 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 03:08:15.209099 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 03:08:15.209099 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 03:08:15.209099 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 03:08:15.209099 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 03:08:15.209099 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 03:08:15.209099 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 03:08:15.209099 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 03:08:15.209099 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 03:08:15.209099 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 03:08:15.209099 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 03:08:15.209099 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application with Lock( Sep 13 03:08:15.209099 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 03:08:15.209099 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._enter() Sep 13 03:08:15.209099 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 03:08:15.209099 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 03:08:15.209949 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 03:08:15.209949 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.creator() Sep 13 03:08:15.209949 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 03:08:15.209949 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application created_value = creator( Sep 13 03:08:15.209949 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 03:08:15.209949 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 03:08:15.209949 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 03:08:15.209949 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 03:08:15.209949 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 03:08:15.209949 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 03:08:15.209949 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 13 03:08:15.209949 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Sep 13 03:08:15.216863 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 304/616] 10.222.0.18 () {56 vars in 1102 bytes} [Tue Sep 13 03:08:14 2022] GET /identity/v3/roles/service => generated 85 bytes in 372 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 03:08:15.220213 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6db59224-ca6f-4ab8-913b-cf1eda560e5a None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:08:15.268400 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a848151f-4cbf-4010-b26b-fe3aa7713e3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:08:15.269042 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a848151f-4cbf-4010-b26b-fe3aa7713e3e None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:08:15.269320 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a848151f-4cbf-4010-b26b-fe3aa7713e3e None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:08:15.269591 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a848151f-4cbf-4010-b26b-fe3aa7713e3e None admin] PATH_INFO: `/v3/users/swift` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:08:15.286138 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a848151f-4cbf-4010-b26b-fe3aa7713e3e None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:08:15.287626 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a848151f-4cbf-4010-b26b-fe3aa7713e3e None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:08:15.307837 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application [None req-a848151f-4cbf-4010-b26b-fe3aa7713e3e None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Sep 13 03:08:15.307837 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 03:08:15.307837 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 03:08:15.307837 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 03:08:15.307837 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 03:08:15.307837 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 03:08:15.307837 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 03:08:15.307837 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 03:08:15.307837 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 03:08:15.307837 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 03:08:15.307837 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 03:08:15.307837 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 03:08:15.307837 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 13 03:08:15.307837 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 13 03:08:15.307837 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 13 03:08:15.307837 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 13 03:08:15.308805 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 03:08:15.308805 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 03:08:15.308805 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 13 03:08:15.308805 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 13 03:08:15.308805 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 13 03:08:15.308805 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 13 03:08:15.308805 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 03:08:15.308805 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 03:08:15.308805 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 03:08:15.308805 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 03:08:15.308805 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 03:08:15.308805 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application with Lock( Sep 13 03:08:15.308805 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 03:08:15.308805 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._enter() Sep 13 03:08:15.308805 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 03:08:15.308805 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 03:08:15.308805 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 03:08:15.308805 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.creator() Sep 13 03:08:15.309758 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 03:08:15.309758 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application created_value = creator( Sep 13 03:08:15.309758 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 13 03:08:15.309758 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 13 03:08:15.309758 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 13 03:08:15.309758 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 13 03:08:15.309758 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 13 03:08:15.309758 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 13 03:08:15.309758 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Sep 13 03:08:15.309758 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Sep 13 03:08:15.309758 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 313/617] 10.222.0.18 () {56 vars in 1096 bytes} [Tue Sep 13 03:08:15 2022] GET /identity/v3/users/swift => generated 83 bytes in 283 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 03:08:15.331840 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a848151f-4cbf-4010-b26b-fe3aa7713e3e None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:08:15.337860 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6db59224-ca6f-4ab8-913b-cf1eda560e5a None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:08:15.372344 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24e865f9-b633-4f4c-88b5-1fecf38d7b2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:08:15.373377 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24e865f9-b633-4f4c-88b5-1fecf38d7b2f None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:08:15.373880 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24e865f9-b633-4f4c-88b5-1fecf38d7b2f None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:08:15.375096 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24e865f9-b633-4f4c-88b5-1fecf38d7b2f None admin] PATH_INFO: `/v3/roles` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:08:15.375811 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24e865f9-b633-4f4c-88b5-1fecf38d7b2f None admin] RBAC: Adding query filter params (name=service) {{(pid=81399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 03:08:15.376125 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24e865f9-b633-4f4c-88b5-1fecf38d7b2f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:08:15.377518 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24e865f9-b633-4f4c-88b5-1fecf38d7b2f None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:08:15.402379 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 305/618] 10.222.0.18 () {56 vars in 1103 bytes} [Tue Sep 13 03:08:15 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 03:08:15.408090 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24e865f9-b633-4f4c-88b5-1fecf38d7b2f None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:08:15.428728 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:08:15.498405 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a848151f-4cbf-4010-b26b-fe3aa7713e3e None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:08:15.546965 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:08:15.558683 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f27f7cfb-d8bc-4a76-827a-884a8779e8c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:08:15.559384 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f27f7cfb-d8bc-4a76-827a-884a8779e8c3 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:08:15.560343 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f27f7cfb-d8bc-4a76-827a-884a8779e8c3 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:08:15.561273 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f27f7cfb-d8bc-4a76-827a-884a8779e8c3 None admin] PATH_INFO: `/v3/users` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:08:15.564312 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f27f7cfb-d8bc-4a76-827a-884a8779e8c3 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 03:08:15.564625 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f27f7cfb-d8bc-4a76-827a-884a8779e8c3 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:08:15.566093 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f27f7cfb-d8bc-4a76-827a-884a8779e8c3 None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:08:15.603220 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 314/619] 10.222.0.18 () {56 vars in 1135 bytes} [Tue Sep 13 03:08:15 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 274 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 03:08:15.609977 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24e865f9-b633-4f4c-88b5-1fecf38d7b2f None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:08:15.619961 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f27f7cfb-d8bc-4a76-827a-884a8779e8c3 None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:08:15.683081 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3404b8d1-10d7-4d46-ba53-2d9117da755f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:08:15.683721 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3404b8d1-10d7-4d46-ba53-2d9117da755f None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:08:15.683996 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3404b8d1-10d7-4d46-ba53-2d9117da755f None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:08:15.684264 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3404b8d1-10d7-4d46-ba53-2d9117da755f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:08:15.703430 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3404b8d1-10d7-4d46-ba53-2d9117da755f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:08:15.704873 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3404b8d1-10d7-4d46-ba53-2d9117da755f None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:08:15.730894 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application [None req-3404b8d1-10d7-4d46-ba53-2d9117da755f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 03:08:15.730894 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 03:08:15.730894 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 03:08:15.730894 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 03:08:15.730894 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 03:08:15.730894 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 03:08:15.730894 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 03:08:15.730894 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 03:08:15.730894 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 03:08:15.730894 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 03:08:15.730894 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 03:08:15.730894 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 03:08:15.730894 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 13 03:08:15.730894 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 13 03:08:15.730894 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 13 03:08:15.730894 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 13 03:08:15.731965 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 03:08:15.731965 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 03:08:15.731965 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 03:08:15.731965 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 03:08:15.731965 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 03:08:15.731965 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 03:08:15.731965 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 03:08:15.731965 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application with Lock( Sep 13 03:08:15.731965 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 03:08:15.731965 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._enter() Sep 13 03:08:15.731965 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 03:08:15.731965 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 03:08:15.731965 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 03:08:15.731965 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.creator() Sep 13 03:08:15.731965 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 03:08:15.731965 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application created_value = creator( Sep 13 03:08:15.731965 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 13 03:08:15.732881 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 13 03:08:15.732881 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 03:08:15.732881 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Sep 13 03:08:15.737764 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 306/620] 10.222.0.18 () {56 vars in 1108 bytes} [Tue Sep 13 03:08:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 328 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 13 03:08:15.741027 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3404b8d1-10d7-4d46-ba53-2d9117da755f None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:08:15.803064 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f27f7cfb-d8bc-4a76-827a-884a8779e8c3 None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:08:15.856852 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:08:15.876093 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e02434c1-4886-481a-96df-3499f716c16b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:08:15.876760 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e02434c1-4886-481a-96df-3499f716c16b None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:08:15.877001 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e02434c1-4886-481a-96df-3499f716c16b None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:08:15.877271 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e02434c1-4886-481a-96df-3499f716c16b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:08:15.898709 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e02434c1-4886-481a-96df-3499f716c16b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:08:15.901331 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e02434c1-4886-481a-96df-3499f716c16b None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:08:15.935507 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3404b8d1-10d7-4d46-ba53-2d9117da755f None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:08:15.937517 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application [None req-e02434c1-4886-481a-96df-3499f716c16b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 03:08:15.937517 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 03:08:15.937517 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 03:08:15.937517 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 03:08:15.937517 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 03:08:15.937517 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 03:08:15.937517 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 03:08:15.937517 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 03:08:15.937517 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 03:08:15.937517 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 03:08:15.937517 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 03:08:15.937517 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 03:08:15.937517 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 13 03:08:15.937517 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 13 03:08:15.937517 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 13 03:08:15.937517 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 13 03:08:15.938472 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 03:08:15.938472 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 03:08:15.938472 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 03:08:15.938472 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 03:08:15.938472 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 03:08:15.938472 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 03:08:15.938472 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 03:08:15.938472 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application with Lock( Sep 13 03:08:15.938472 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 03:08:15.938472 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._enter() Sep 13 03:08:15.938472 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 03:08:15.938472 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 03:08:15.938472 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 03:08:15.938472 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.creator() Sep 13 03:08:15.938472 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 03:08:15.938472 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application created_value = creator( Sep 13 03:08:15.938472 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 13 03:08:15.939436 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 13 03:08:15.939436 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 03:08:15.939436 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Sep 13 03:08:15.939436 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 315/621] 10.222.0.18 () {56 vars in 1108 bytes} [Tue Sep 13 03:08:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 319 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 13 03:08:15.944603 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e02434c1-4886-481a-96df-3499f716c16b None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:08:16.007426 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52784a1c-100f-40dd-a111-5c9dfdb7a55d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:08:16.008041 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52784a1c-100f-40dd-a111-5c9dfdb7a55d None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:08:16.008319 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52784a1c-100f-40dd-a111-5c9dfdb7a55d None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:08:16.008593 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52784a1c-100f-40dd-a111-5c9dfdb7a55d None admin] PATH_INFO: `/v3/domains` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:08:16.009231 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52784a1c-100f-40dd-a111-5c9dfdb7a55d None admin] RBAC: Adding query filter params (name=Default) {{(pid=81399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 03:08:16.009763 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52784a1c-100f-40dd-a111-5c9dfdb7a55d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:08:16.013173 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52784a1c-100f-40dd-a111-5c9dfdb7a55d None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:08:16.027502 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:08:16.037538 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 307/622] 10.222.0.18 () {56 vars in 1109 bytes} [Tue Sep 13 03:08:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 297 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 03:08:16.042183 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52784a1c-100f-40dd-a111-5c9dfdb7a55d None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:08:16.153593 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e02434c1-4886-481a-96df-3499f716c16b None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:08:16.173612 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52784a1c-100f-40dd-a111-5c9dfdb7a55d None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:08:16.209363 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:08:16.223740 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df0d9b0c-f622-48bb-b8f7-85f9c23841cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:08:16.224772 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df0d9b0c-f622-48bb-b8f7-85f9c23841cc None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:08:16.225268 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df0d9b0c-f622-48bb-b8f7-85f9c23841cc None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:08:16.225551 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df0d9b0c-f622-48bb-b8f7-85f9c23841cc None admin] PATH_INFO: `/v3/domains` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:08:16.226337 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df0d9b0c-f622-48bb-b8f7-85f9c23841cc None admin] RBAC: Adding query filter params (name=Default) {{(pid=81400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 03:08:16.226794 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df0d9b0c-f622-48bb-b8f7-85f9c23841cc None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:08:16.228417 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df0d9b0c-f622-48bb-b8f7-85f9c23841cc None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:08:16.242119 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df7f94d6-2901-4c0e-a12b-db0c038b2967 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:08:16.242739 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 316/623] 10.222.0.18 () {56 vars in 1109 bytes} [Tue Sep 13 03:08:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 298 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 03:08:16.243386 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df7f94d6-2901-4c0e-a12b-db0c038b2967 None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:08:16.243873 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df7f94d6-2901-4c0e-a12b-db0c038b2967 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:08:16.243993 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af475cd6-811e-41cd-8493-92e73a5499f7 None None] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:08:16.244217 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af475cd6-811e-41cd-8493-92e73a5499f7 None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:08:16.244492 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af475cd6-811e-41cd-8493-92e73a5499f7 None None] PATH_INFO: `/` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:08:16.244902 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df7f94d6-2901-4c0e-a12b-db0c038b2967 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:08:16.246116 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 317/624] 10.222.0.18 () {54 vars in 927 bytes} [Tue Sep 13 03:08:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 03:08:16.249395 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af475cd6-811e-41cd-8493-92e73a5499f7 None None] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:08:16.261720 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df7f94d6-2901-4c0e-a12b-db0c038b2967 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:08:16.263564 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df7f94d6-2901-4c0e-a12b-db0c038b2967 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:08:16.293599 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application [None req-df7f94d6-2901-4c0e-a12b-db0c038b2967 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 13 03:08:16.293599 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 03:08:16.293599 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 03:08:16.293599 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 03:08:16.293599 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 03:08:16.293599 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 03:08:16.293599 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 03:08:16.293599 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 03:08:16.293599 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 03:08:16.293599 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 03:08:16.293599 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 03:08:16.293599 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 03:08:16.293599 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 13 03:08:16.293599 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 13 03:08:16.293599 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 13 03:08:16.293599 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 13 03:08:16.294668 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 03:08:16.294668 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 03:08:16.294668 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 13 03:08:16.294668 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 13 03:08:16.294668 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 13 03:08:16.294668 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 13 03:08:16.294668 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 03:08:16.294668 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 03:08:16.294668 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 03:08:16.294668 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 03:08:16.294668 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 03:08:16.294668 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application with Lock( Sep 13 03:08:16.294668 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 03:08:16.294668 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._enter() Sep 13 03:08:16.294668 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 03:08:16.294668 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 03:08:16.294668 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 03:08:16.294668 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.creator() Sep 13 03:08:16.295651 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 03:08:16.295651 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application created_value = creator( Sep 13 03:08:16.295651 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 13 03:08:16.295651 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 13 03:08:16.295651 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 13 03:08:16.295651 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 13 03:08:16.295651 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 13 03:08:16.295651 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 13 03:08:16.295651 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 13 03:08:16.295651 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Sep 13 03:08:16.310427 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 308/625] 10.222.0.18 () {56 vars in 1093 bytes} [Tue Sep 13 03:08:16 2022] GET /identity/v3/users/nova => generated 82 bytes in 268 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 03:08:16.312151 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a2e1158-d2ac-421f-a2db-66db208cb9c4 None None] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:08:16.312659 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a2e1158-d2ac-421f-a2db-66db208cb9c4 None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:08:16.313138 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a2e1158-d2ac-421f-a2db-66db208cb9c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:08:16.349463 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.auth.core [None req-8a2e1158-d2ac-421f-a2db-66db208cb9c4 None None] MFA Rules not processed for user `49f280ab086b4610855c4ffa378517d9`. Rule list: `[]` (Enabled: `True`). {{(pid=81399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 03:08:16.351069 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-8a2e1158-d2ac-421f-a2db-66db208cb9c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:08:16.416060 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:08:16.423637 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af475cd6-811e-41cd-8493-92e73a5499f7 None None] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:08:16.501861 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 309/626] 10.222.0.18 () {58 vars in 1024 bytes} [Tue Sep 13 03:08:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 13 03:08:16.506323 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a2e1158-d2ac-421f-a2db-66db208cb9c4 None None] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:08:16.523013 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb392b28-783b-4c4c-8388-9db4251053ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:08:16.524250 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb392b28-783b-4c4c-8388-9db4251053ff None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:08:16.527150 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb392b28-783b-4c4c-8388-9db4251053ff None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:08:16.529162 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb392b28-783b-4c4c-8388-9db4251053ff None admin] PATH_INFO: `/v3/projects/service` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:08:16.553456 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb392b28-783b-4c4c-8388-9db4251053ff None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:08:16.555228 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb392b28-783b-4c4c-8388-9db4251053ff None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:08:16.577845 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application [None req-bb392b28-783b-4c4c-8388-9db4251053ff None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 13 03:08:16.577845 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 03:08:16.577845 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 03:08:16.577845 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 03:08:16.577845 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 03:08:16.577845 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 03:08:16.577845 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 03:08:16.577845 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 03:08:16.577845 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 03:08:16.577845 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 03:08:16.577845 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 03:08:16.577845 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 03:08:16.577845 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 13 03:08:16.577845 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 13 03:08:16.577845 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 13 03:08:16.578859 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 13 03:08:16.578859 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 03:08:16.578859 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 03:08:16.578859 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 03:08:16.578859 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 03:08:16.578859 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 03:08:16.578859 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 03:08:16.578859 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 03:08:16.578859 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application with Lock( Sep 13 03:08:16.578859 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 03:08:16.578859 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._enter() Sep 13 03:08:16.578859 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 03:08:16.578859 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 03:08:16.578859 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 03:08:16.578859 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.creator() Sep 13 03:08:16.578859 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 03:08:16.578859 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application created_value = creator( Sep 13 03:08:16.579751 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 13 03:08:16.579751 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 13 03:08:16.579751 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 13 03:08:16.579751 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 13 03:08:16.579751 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 13 03:08:16.579751 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 13 03:08:16.579751 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 13 03:08:16.579751 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Sep 13 03:08:16.587072 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 318/627] 10.222.0.18 () {56 vars in 1111 bytes} [Tue Sep 13 03:08:16 2022] GET /identity/v3/projects/service => generated 88 bytes in 338 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 03:08:16.593472 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22d840c1-d5a4-4c8f-a517-807f680fb5be None None] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:08:16.593754 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22d840c1-d5a4-4c8f-a517-807f680fb5be None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:08:16.594025 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22d840c1-d5a4-4c8f-a517-807f680fb5be None None] PATH_INFO: `/` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:08:16.595334 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 319/628] 10.222.0.18 () {54 vars in 927 bytes} [Tue Sep 13 03:08:16 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 03:08:16.596020 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22d840c1-d5a4-4c8f-a517-807f680fb5be None None] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:08:16.709456 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a2e1158-d2ac-421f-a2db-66db208cb9c4 None None] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:08:16.723510 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:08:16.753109 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13527cf2-6baa-4d76-9cf3-4006fc271925 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:08:16.753637 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13527cf2-6baa-4d76-9cf3-4006fc271925 None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:08:16.753911 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13527cf2-6baa-4d76-9cf3-4006fc271925 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:08:16.754182 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13527cf2-6baa-4d76-9cf3-4006fc271925 None admin] PATH_INFO: `/v3/users` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:08:16.755109 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13527cf2-6baa-4d76-9cf3-4006fc271925 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 03:08:16.755415 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13527cf2-6baa-4d76-9cf3-4006fc271925 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:08:16.757184 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13527cf2-6baa-4d76-9cf3-4006fc271925 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:08:16.774994 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22d840c1-d5a4-4c8f-a517-807f680fb5be None None] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:08:16.778696 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 310/629] 10.222.0.18 () {56 vars in 1133 bytes} [Tue Sep 13 03:08:16 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 276 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 03:08:16.779893 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd77c533-2388-4bda-a009-fe627263620a None None] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:08:16.780097 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd77c533-2388-4bda-a009-fe627263620a None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:08:16.781922 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd77c533-2388-4bda-a009-fe627263620a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:08:16.800452 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.auth.core [None req-dd77c533-2388-4bda-a009-fe627263620a None None] MFA Rules not processed for user `49f280ab086b4610855c4ffa378517d9`. Rule list: `[]` (Enabled: `True`). {{(pid=81399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 03:08:16.801635 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-dd77c533-2388-4bda-a009-fe627263620a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:08:16.810113 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6de805f0-4c2f-4799-a6fa-f60a83c1f0da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:08:16.810751 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de805f0-4c2f-4799-a6fa-f60a83c1f0da None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:08:16.811032 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de805f0-4c2f-4799-a6fa-f60a83c1f0da None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:08:16.811303 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de805f0-4c2f-4799-a6fa-f60a83c1f0da None admin] PATH_INFO: `/v3/projects` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:08:16.812513 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6de805f0-4c2f-4799-a6fa-f60a83c1f0da None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 03:08:16.812819 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6de805f0-4c2f-4799-a6fa-f60a83c1f0da None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:08:16.814249 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6de805f0-4c2f-4799-a6fa-f60a83c1f0da None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:08:16.825897 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 320/630] 10.222.0.18 () {56 vars in 1148 bytes} [Tue Sep 13 03:08:16 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 03:08:16.826691 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6de805f0-4c2f-4799-a6fa-f60a83c1f0da None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:08:16.849396 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:08:16.869615 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 311/631] 10.222.0.18 () {58 vars in 1024 bytes} [Tue Sep 13 03:08:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 13 03:08:16.870447 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd77c533-2388-4bda-a009-fe627263620a None None] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:08:16.930603 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6de805f0-4c2f-4799-a6fa-f60a83c1f0da None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:08:16.967056 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:08:16.968472 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6883502-c0a9-433d-b34e-77a690920510 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:08:16.969130 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6883502-c0a9-433d-b34e-77a690920510 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:08:16.969415 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6883502-c0a9-433d-b34e-77a690920510 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:08:16.970369 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6883502-c0a9-433d-b34e-77a690920510 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:08:16.971945 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd77c533-2388-4bda-a009-fe627263620a None None] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:08:16.981172 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6883502-c0a9-433d-b34e-77a690920510 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 03:08:16.986433 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application [None req-f6883502-c0a9-433d-b34e-77a690920510 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 13 03:08:16.986433 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 03:08:16.986433 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 03:08:16.986433 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 03:08:16.986433 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 03:08:16.986433 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 03:08:16.986433 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 03:08:16.986433 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 03:08:16.986433 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 03:08:16.986433 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 03:08:16.986433 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 03:08:16.986433 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 03:08:16.986433 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 03:08:16.986433 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 03:08:16.986433 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 03:08:16.986433 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 03:08:16.987431 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 03:08:16.987431 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 03:08:16.987431 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 03:08:16.987431 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 03:08:16.987431 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 03:08:16.987431 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 03:08:16.987431 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 03:08:16.987431 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 03:08:16.987431 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 03:08:16.987431 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 03:08:16.987431 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 03:08:16.987431 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application with Lock( Sep 13 03:08:16.987431 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 03:08:16.987431 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._enter() Sep 13 03:08:16.987431 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 03:08:16.987431 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 03:08:16.988305 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 03:08:16.988305 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.creator() Sep 13 03:08:16.988305 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 03:08:16.988305 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application created_value = creator( Sep 13 03:08:16.988305 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 03:08:16.988305 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 03:08:16.988305 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 03:08:16.988305 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 03:08:16.988305 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 03:08:16.988305 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 03:08:16.988305 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 13 03:08:16.988305 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Sep 13 03:08:16.988305 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 321/632] 10.222.0.18 () {56 vars in 1102 bytes} [Tue Sep 13 03:08:16 2022] GET /identity/v3/roles/service => generated 85 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 03:08:16.989019 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-601ef660-6efe-4f33-82cf-0075213cc226 None None] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:08:16.989019 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-601ef660-6efe-4f33-82cf-0075213cc226 None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:08:16.990814 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-601ef660-6efe-4f33-82cf-0075213cc226 None None] PATH_INFO: `/` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:08:16.990814 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 322/633] 10.222.0.18 () {54 vars in 927 bytes} [Tue Sep 13 03:08:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 03:08:16.991460 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-601ef660-6efe-4f33-82cf-0075213cc226 None None] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:08:17.019381 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8d266ba-e6d4-4afc-a2b3-0803aed4feda None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:08:17.019922 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8d266ba-e6d4-4afc-a2b3-0803aed4feda None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:08:17.020200 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8d266ba-e6d4-4afc-a2b3-0803aed4feda None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:08:17.020468 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8d266ba-e6d4-4afc-a2b3-0803aed4feda None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:08:17.089539 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:08:17.097099 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8d266ba-e6d4-4afc-a2b3-0803aed4feda None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:08:17.098685 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8d266ba-e6d4-4afc-a2b3-0803aed4feda None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:08:17.111637 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-601ef660-6efe-4f33-82cf-0075213cc226 None None] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:08:17.146246 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea503c3e-d1be-4245-b1bf-de3ab5a9462d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:08:17.146929 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea503c3e-d1be-4245-b1bf-de3ab5a9462d None admin] REQUEST_METHOD: `PUT` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:08:17.147210 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea503c3e-d1be-4245-b1bf-de3ab5a9462d None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:08:17.147483 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea503c3e-d1be-4245-b1bf-de3ab5a9462d None admin] PATH_INFO: `/v3/projects/b9d94f6a68f7470d9494f112175620dc/users/893b0c0d85964b7dbc9705c0a091682c/roles/817d6837cd044890b41cf16b303a092b` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:08:17.161435 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea503c3e-d1be-4245-b1bf-de3ab5a9462d None admin] RBAC: Authorizing `identity:create_grant(project_id=b9d94f6a68f7470d9494f112175620dc, user_id=893b0c0d85964b7dbc9705c0a091682c, role_id=817d6837cd044890b41cf16b303a092b)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:08:17.163497 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea503c3e-d1be-4245-b1bf-de3ab5a9462d None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:08:17.175676 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 323/634] 10.222.0.18 () {58 vars in 1439 bytes} [Tue Sep 13 03:08:16 2022] PUT /identity/v3/projects/b9d94f6a68f7470d9494f112175620dc/users/893b0c0d85964b7dbc9705c0a091682c/roles/817d6837cd044890b41cf16b303a092b => generated 0 bytes in 184 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 03:08:17.176860 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c617259-0b41-479d-8f2f-ad2b5524dfa1 None None] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:08:17.177144 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c617259-0b41-479d-8f2f-ad2b5524dfa1 None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:08:17.177419 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c617259-0b41-479d-8f2f-ad2b5524dfa1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:08:17.200867 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:08:17.201081 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.auth.core [None req-8c617259-0b41-479d-8f2f-ad2b5524dfa1 None None] MFA Rules not processed for user `49f280ab086b4610855c4ffa378517d9`. Rule list: `[]` (Enabled: `True`). {{(pid=81400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 03:08:17.202290 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-8c617259-0b41-479d-8f2f-ad2b5524dfa1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:08:17.211055 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 312/635] 10.222.0.18 () {58 vars in 1282 bytes} [Tue Sep 13 03:08:16 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 341 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 13 03:08:17.212057 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8d266ba-e6d4-4afc-a2b3-0803aed4feda None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:08:17.290727 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 324/636] 10.222.0.18 () {58 vars in 1024 bytes} [Tue Sep 13 03:08:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 13 03:08:17.290727 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c617259-0b41-479d-8f2f-ad2b5524dfa1 None None] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:08:17.339583 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8d266ba-e6d4-4afc-a2b3-0803aed4feda None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:08:17.370486 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:08:17.383364 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b64cb574-83b0-4248-b9b1-b15bb780f867 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:08:17.383976 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b64cb574-83b0-4248-b9b1-b15bb780f867 None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:08:17.384255 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b64cb574-83b0-4248-b9b1-b15bb780f867 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:08:17.384524 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b64cb574-83b0-4248-b9b1-b15bb780f867 None admin] PATH_INFO: `/v3/roles` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:08:17.385189 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b64cb574-83b0-4248-b9b1-b15bb780f867 None admin] RBAC: Adding query filter params (name=service) {{(pid=81399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 03:08:17.385490 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b64cb574-83b0-4248-b9b1-b15bb780f867 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:08:17.386938 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b64cb574-83b0-4248-b9b1-b15bb780f867 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:08:17.396129 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 313/637] 10.222.0.18 () {56 vars in 1103 bytes} [Tue Sep 13 03:08:17 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 03:08:17.396930 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b64cb574-83b0-4248-b9b1-b15bb780f867 None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:08:17.425138 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c617259-0b41-479d-8f2f-ad2b5524dfa1 None None] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:08:17.470098 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:08:17.472541 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28365dba-cf64-4b62-a7ba-fbcd86d7382e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:08:17.473812 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28365dba-cf64-4b62-a7ba-fbcd86d7382e None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:08:17.474853 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28365dba-cf64-4b62-a7ba-fbcd86d7382e None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:08:17.475143 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28365dba-cf64-4b62-a7ba-fbcd86d7382e None admin] PATH_INFO: `/v3/roles/service` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:08:17.490739 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28365dba-cf64-4b62-a7ba-fbcd86d7382e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 03:08:17.497818 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application [None req-28365dba-cf64-4b62-a7ba-fbcd86d7382e None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 13 03:08:17.497818 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 03:08:17.497818 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 03:08:17.497818 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 03:08:17.497818 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 03:08:17.497818 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 03:08:17.497818 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 03:08:17.497818 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 03:08:17.497818 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 03:08:17.497818 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 03:08:17.497818 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 03:08:17.497818 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 03:08:17.497818 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 03:08:17.497818 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 03:08:17.497818 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 03:08:17.497818 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 03:08:17.498774 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 03:08:17.498774 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 03:08:17.498774 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 03:08:17.498774 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 03:08:17.498774 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 03:08:17.498774 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 03:08:17.498774 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 03:08:17.498774 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 03:08:17.498774 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 03:08:17.498774 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 03:08:17.498774 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 03:08:17.498774 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application with Lock( Sep 13 03:08:17.498774 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 03:08:17.498774 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._enter() Sep 13 03:08:17.498774 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 03:08:17.498774 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 03:08:17.499668 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 03:08:17.499668 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.creator() Sep 13 03:08:17.499668 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 03:08:17.499668 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application created_value = creator( Sep 13 03:08:17.499668 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 03:08:17.499668 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 03:08:17.499668 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 03:08:17.499668 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 03:08:17.499668 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 03:08:17.499668 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 03:08:17.499668 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 13 03:08:17.499668 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Sep 13 03:08:17.500304 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 325/638] 10.222.0.18 () {56 vars in 1102 bytes} [Tue Sep 13 03:08:17 2022] GET /identity/v3/roles/service => generated 85 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 03:08:17.505341 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28365dba-cf64-4b62-a7ba-fbcd86d7382e None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:08:17.518240 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b64cb574-83b0-4248-b9b1-b15bb780f867 None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:08:17.556263 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5f33252-25d8-4856-a638-42faa45950d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:08:17.556847 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f33252-25d8-4856-a638-42faa45950d5 None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:08:17.557186 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f33252-25d8-4856-a638-42faa45950d5 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:08:17.557571 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f33252-25d8-4856-a638-42faa45950d5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:08:17.569651 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5f33252-25d8-4856-a638-42faa45950d5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:08:17.571249 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5f33252-25d8-4856-a638-42faa45950d5 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:08:17.587719 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application [None req-b5f33252-25d8-4856-a638-42faa45950d5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 03:08:17.587719 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 03:08:17.587719 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 03:08:17.587719 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 03:08:17.587719 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 03:08:17.587719 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 03:08:17.587719 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 03:08:17.587719 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 03:08:17.587719 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 03:08:17.587719 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 03:08:17.587719 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 03:08:17.587719 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 03:08:17.587719 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 13 03:08:17.587719 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 13 03:08:17.587719 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 13 03:08:17.587719 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 13 03:08:17.588775 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 03:08:17.588775 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 03:08:17.588775 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 03:08:17.588775 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 03:08:17.588775 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 03:08:17.588775 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 03:08:17.588775 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 03:08:17.588775 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application with Lock( Sep 13 03:08:17.588775 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 03:08:17.588775 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._enter() Sep 13 03:08:17.588775 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 03:08:17.588775 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 03:08:17.588775 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 03:08:17.588775 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.creator() Sep 13 03:08:17.588775 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 03:08:17.588775 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application created_value = creator( Sep 13 03:08:17.588775 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 13 03:08:17.589674 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 13 03:08:17.589674 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 03:08:17.589674 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Sep 13 03:08:17.589674 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 314/639] 10.222.0.18 () {56 vars in 1108 bytes} [Tue Sep 13 03:08:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 03:08:17.589674 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5f33252-25d8-4856-a638-42faa45950d5 None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:08:17.631060 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28365dba-cf64-4b62-a7ba-fbcd86d7382e None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:08:17.653294 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:08:17.672229 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-854ecb97-4ad9-4ef7-abd7-b23b2c2e35bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:08:17.672866 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-854ecb97-4ad9-4ef7-abd7-b23b2c2e35bf None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:08:17.673127 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-854ecb97-4ad9-4ef7-abd7-b23b2c2e35bf None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:08:17.673400 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-854ecb97-4ad9-4ef7-abd7-b23b2c2e35bf None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:08:17.683667 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-854ecb97-4ad9-4ef7-abd7-b23b2c2e35bf None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:08:17.685115 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-854ecb97-4ad9-4ef7-abd7-b23b2c2e35bf None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:08:17.698485 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5f33252-25d8-4856-a638-42faa45950d5 None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:08:17.699624 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application [None req-854ecb97-4ad9-4ef7-abd7-b23b2c2e35bf None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 03:08:17.699624 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 03:08:17.699624 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 03:08:17.699624 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 03:08:17.699624 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 03:08:17.699624 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 03:08:17.699624 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 03:08:17.699624 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 03:08:17.699624 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 03:08:17.699624 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 03:08:17.699624 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 03:08:17.699624 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 03:08:17.699624 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 13 03:08:17.699624 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 13 03:08:17.699624 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 13 03:08:17.699624 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 13 03:08:17.700740 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 03:08:17.700740 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 03:08:17.700740 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 03:08:17.700740 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 03:08:17.700740 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 03:08:17.700740 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 03:08:17.700740 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 03:08:17.700740 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application with Lock( Sep 13 03:08:17.700740 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 03:08:17.700740 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._enter() Sep 13 03:08:17.700740 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 03:08:17.700740 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 03:08:17.700740 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 03:08:17.700740 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.creator() Sep 13 03:08:17.700740 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 03:08:17.700740 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application created_value = creator( Sep 13 03:08:17.700740 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 13 03:08:17.701783 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 13 03:08:17.701783 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 03:08:17.701783 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Sep 13 03:08:17.701783 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 326/640] 10.222.0.18 () {56 vars in 1108 bytes} [Tue Sep 13 03:08:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 03:08:17.706122 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-854ecb97-4ad9-4ef7-abd7-b23b2c2e35bf None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:08:17.716401 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:08:17.740061 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8d202c0-fea2-4352-a2b5-5175a5aa3a2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:08:17.740668 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8d202c0-fea2-4352-a2b5-5175a5aa3a2c None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:08:17.740948 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8d202c0-fea2-4352-a2b5-5175a5aa3a2c None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:08:17.741218 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8d202c0-fea2-4352-a2b5-5175a5aa3a2c None admin] PATH_INFO: `/v3/roles` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:08:17.741875 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8d202c0-fea2-4352-a2b5-5175a5aa3a2c None admin] RBAC: Adding query filter params (name=service) {{(pid=81399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 03:08:17.742182 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8d202c0-fea2-4352-a2b5-5175a5aa3a2c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:08:17.743580 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8d202c0-fea2-4352-a2b5-5175a5aa3a2c None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:08:17.751823 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 315/641] 10.222.0.18 () {56 vars in 1103 bytes} [Tue Sep 13 03:08:17 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 03:08:17.752683 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8d202c0-fea2-4352-a2b5-5175a5aa3a2c None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:08:17.815700 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-854ecb97-4ad9-4ef7-abd7-b23b2c2e35bf None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:08:17.850921 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b1dae05-595f-4b10-b4ef-058caba1d66c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:08:17.851825 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b1dae05-595f-4b10-b4ef-058caba1d66c None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:08:17.852274 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b1dae05-595f-4b10-b4ef-058caba1d66c None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:08:17.852551 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b1dae05-595f-4b10-b4ef-058caba1d66c None admin] PATH_INFO: `/v3/roles/service` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:08:17.855034 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8d202c0-fea2-4352-a2b5-5175a5aa3a2c None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:08:17.863125 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b1dae05-595f-4b10-b4ef-058caba1d66c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 03:08:17.868472 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application [None req-3b1dae05-595f-4b10-b4ef-058caba1d66c None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 13 03:08:17.868472 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 03:08:17.868472 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 03:08:17.868472 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 03:08:17.868472 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 03:08:17.868472 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 03:08:17.868472 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 03:08:17.868472 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 03:08:17.868472 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 03:08:17.868472 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 03:08:17.868472 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 03:08:17.868472 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 03:08:17.868472 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 03:08:17.868472 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 03:08:17.868472 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 03:08:17.868472 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 03:08:17.869658 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 03:08:17.869658 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 03:08:17.869658 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 03:08:17.869658 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 03:08:17.869658 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 03:08:17.869658 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 03:08:17.869658 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 03:08:17.869658 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 03:08:17.869658 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 03:08:17.869658 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 03:08:17.869658 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 03:08:17.869658 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application with Lock( Sep 13 03:08:17.869658 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 03:08:17.869658 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._enter() Sep 13 03:08:17.869658 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 03:08:17.869658 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 03:08:17.871056 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 03:08:17.871056 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.creator() Sep 13 03:08:17.871056 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 03:08:17.871056 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application created_value = creator( Sep 13 03:08:17.871056 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 03:08:17.871056 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 03:08:17.871056 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 03:08:17.871056 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 03:08:17.871056 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 03:08:17.871056 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 03:08:17.871056 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 13 03:08:17.871056 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Sep 13 03:08:17.871056 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 327/642] 10.222.0.18 () {56 vars in 1102 bytes} [Tue Sep 13 03:08:17 2022] GET /identity/v3/roles/service => generated 85 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 03:08:17.871056 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b1dae05-595f-4b10-b4ef-058caba1d66c None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:08:17.895787 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d1db3ae-2df7-4251-b2b3-ed84a98f7351 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:08:17.896522 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d1db3ae-2df7-4251-b2b3-ed84a98f7351 None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:08:17.896806 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d1db3ae-2df7-4251-b2b3-ed84a98f7351 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:08:17.897109 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d1db3ae-2df7-4251-b2b3-ed84a98f7351 None admin] PATH_INFO: `/v3/domains` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:08:17.897826 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d1db3ae-2df7-4251-b2b3-ed84a98f7351 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 03:08:17.898133 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d1db3ae-2df7-4251-b2b3-ed84a98f7351 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:08:17.899648 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d1db3ae-2df7-4251-b2b3-ed84a98f7351 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:08:17.903371 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:08:17.918912 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 316/643] 10.222.0.18 () {56 vars in 1109 bytes} [Tue Sep 13 03:08:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 03:08:17.918912 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d1db3ae-2df7-4251-b2b3-ed84a98f7351 None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:08:17.953898 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:08:17.981347 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b1dae05-595f-4b10-b4ef-058caba1d66c None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:08:18.018945 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26681528-6d5e-4752-ae8c-074284ad60bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:08:18.019549 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26681528-6d5e-4752-ae8c-074284ad60bc None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:08:18.019941 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26681528-6d5e-4752-ae8c-074284ad60bc None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:08:18.020215 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26681528-6d5e-4752-ae8c-074284ad60bc None admin] PATH_INFO: `/v3/domains` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:08:18.020967 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26681528-6d5e-4752-ae8c-074284ad60bc None admin] RBAC: Adding query filter params (name=Default) {{(pid=81400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 03:08:18.021327 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26681528-6d5e-4752-ae8c-074284ad60bc None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:08:18.022722 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26681528-6d5e-4752-ae8c-074284ad60bc None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:08:18.023694 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d1db3ae-2df7-4251-b2b3-ed84a98f7351 None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:08:18.035736 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 328/644] 10.222.0.18 () {56 vars in 1109 bytes} [Tue Sep 13 03:08:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 03:08:18.036533 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26681528-6d5e-4752-ae8c-074284ad60bc None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:08:18.072207 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7f518a6-7622-4e6d-a72b-8d65c9d5dfa0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:08:18.073332 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7f518a6-7622-4e6d-a72b-8d65c9d5dfa0 None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:08:18.073761 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7f518a6-7622-4e6d-a72b-8d65c9d5dfa0 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:08:18.074177 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7f518a6-7622-4e6d-a72b-8d65c9d5dfa0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:08:18.087404 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7f518a6-7622-4e6d-a72b-8d65c9d5dfa0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:08:18.089328 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7f518a6-7622-4e6d-a72b-8d65c9d5dfa0 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:08:18.106633 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application [None req-b7f518a6-7622-4e6d-a72b-8d65c9d5dfa0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 03:08:18.106633 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 03:08:18.106633 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 03:08:18.106633 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 03:08:18.106633 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 03:08:18.106633 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 03:08:18.106633 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 03:08:18.106633 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 03:08:18.106633 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 03:08:18.106633 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 03:08:18.106633 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 03:08:18.106633 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 03:08:18.106633 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 13 03:08:18.106633 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 13 03:08:18.106633 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 13 03:08:18.106633 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 13 03:08:18.107856 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 03:08:18.107856 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 03:08:18.107856 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 03:08:18.107856 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 03:08:18.107856 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 03:08:18.107856 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 03:08:18.107856 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 03:08:18.107856 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application with Lock( Sep 13 03:08:18.107856 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 03:08:18.107856 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._enter() Sep 13 03:08:18.107856 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 03:08:18.107856 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 03:08:18.107856 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 03:08:18.107856 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.creator() Sep 13 03:08:18.107856 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 03:08:18.107856 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application created_value = creator( Sep 13 03:08:18.107856 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 13 03:08:18.109173 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 13 03:08:18.109173 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 03:08:18.109173 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Sep 13 03:08:18.112701 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 317/645] 10.222.0.18 () {56 vars in 1108 bytes} [Tue Sep 13 03:08:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 03:08:18.113692 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7f518a6-7622-4e6d-a72b-8d65c9d5dfa0 None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:08:18.140394 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:08:18.149793 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26681528-6d5e-4752-ae8c-074284ad60bc None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:08:18.185577 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7e5c29e-a605-45f4-acbf-7edae6f10910 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:08:18.186173 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e5c29e-a605-45f4-acbf-7edae6f10910 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:08:18.186995 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e5c29e-a605-45f4-acbf-7edae6f10910 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:08:18.187278 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e5c29e-a605-45f4-acbf-7edae6f10910 None admin] PATH_INFO: `/v3/roles` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:08:18.187931 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7e5c29e-a605-45f4-acbf-7edae6f10910 None admin] RBAC: Adding query filter params (name=service) {{(pid=81400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 03:08:18.188230 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7e5c29e-a605-45f4-acbf-7edae6f10910 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:08:18.189595 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7e5c29e-a605-45f4-acbf-7edae6f10910 None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:08:18.199791 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 329/646] 10.222.0.18 () {56 vars in 1103 bytes} [Tue Sep 13 03:08:18 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 03:08:18.205627 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7e5c29e-a605-45f4-acbf-7edae6f10910 None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:08:18.206469 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:08:18.222791 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7f518a6-7622-4e6d-a72b-8d65c9d5dfa0 None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:08:18.258875 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ad7b9d8-4803-4a1e-bf7e-55eab244b015 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:08:18.259594 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad7b9d8-4803-4a1e-bf7e-55eab244b015 None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:08:18.259977 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad7b9d8-4803-4a1e-bf7e-55eab244b015 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:08:18.260255 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad7b9d8-4803-4a1e-bf7e-55eab244b015 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:08:18.267971 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ad7b9d8-4803-4a1e-bf7e-55eab244b015 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:08:18.269512 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ad7b9d8-4803-4a1e-bf7e-55eab244b015 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:08:18.282003 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application [None req-6ad7b9d8-4803-4a1e-bf7e-55eab244b015 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 13 03:08:18.282003 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 03:08:18.282003 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 03:08:18.282003 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 03:08:18.282003 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 03:08:18.282003 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 03:08:18.282003 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 03:08:18.282003 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 03:08:18.282003 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 03:08:18.282003 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 03:08:18.282003 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 03:08:18.282003 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 03:08:18.282003 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 13 03:08:18.282003 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 13 03:08:18.282003 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 13 03:08:18.283113 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 13 03:08:18.283113 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 03:08:18.283113 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 03:08:18.283113 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 03:08:18.283113 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 03:08:18.283113 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 03:08:18.283113 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 03:08:18.283113 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 03:08:18.283113 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application with Lock( Sep 13 03:08:18.283113 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 03:08:18.283113 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._enter() Sep 13 03:08:18.283113 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 03:08:18.283113 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 03:08:18.283113 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 03:08:18.283113 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.creator() Sep 13 03:08:18.283113 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 03:08:18.283113 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application created_value = creator( Sep 13 03:08:18.284246 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 13 03:08:18.284246 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 13 03:08:18.284246 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 13 03:08:18.284246 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 13 03:08:18.284246 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 13 03:08:18.284246 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 13 03:08:18.284246 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 13 03:08:18.284246 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Sep 13 03:08:18.288084 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 318/647] 10.222.0.18 () {56 vars in 1111 bytes} [Tue Sep 13 03:08:18 2022] GET /identity/v3/projects/service => generated 88 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 03:08:18.288914 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ad7b9d8-4803-4a1e-bf7e-55eab244b015 None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:08:18.311536 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7e5c29e-a605-45f4-acbf-7edae6f10910 None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:08:18.346170 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b07fd16c-8efe-4a88-a644-2735712ca09b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:08:18.346773 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b07fd16c-8efe-4a88-a644-2735712ca09b None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:08:18.347050 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b07fd16c-8efe-4a88-a644-2735712ca09b None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:08:18.347585 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b07fd16c-8efe-4a88-a644-2735712ca09b None admin] PATH_INFO: `/v3/users/glance` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:08:18.354801 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b07fd16c-8efe-4a88-a644-2735712ca09b None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:08:18.356263 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b07fd16c-8efe-4a88-a644-2735712ca09b None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:08:18.368138 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application [None req-b07fd16c-8efe-4a88-a644-2735712ca09b None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Sep 13 03:08:18.368138 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 03:08:18.368138 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 03:08:18.368138 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 03:08:18.368138 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 03:08:18.368138 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 03:08:18.368138 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 03:08:18.368138 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 03:08:18.368138 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 03:08:18.368138 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 03:08:18.368138 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 03:08:18.368138 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 03:08:18.368138 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 13 03:08:18.368138 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 13 03:08:18.368138 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 13 03:08:18.368138 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 13 03:08:18.369106 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 03:08:18.369106 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 03:08:18.369106 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 13 03:08:18.369106 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 13 03:08:18.369106 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 13 03:08:18.369106 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 13 03:08:18.369106 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 03:08:18.369106 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 03:08:18.369106 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 03:08:18.369106 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 03:08:18.369106 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 03:08:18.369106 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application with Lock( Sep 13 03:08:18.369106 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 03:08:18.369106 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._enter() Sep 13 03:08:18.369106 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 03:08:18.369106 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 03:08:18.369106 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 03:08:18.369106 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.creator() Sep 13 03:08:18.370109 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 03:08:18.370109 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application created_value = creator( Sep 13 03:08:18.370109 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 13 03:08:18.370109 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 13 03:08:18.370109 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 13 03:08:18.370109 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 13 03:08:18.370109 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 13 03:08:18.370109 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 13 03:08:18.370109 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Sep 13 03:08:18.370109 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Sep 13 03:08:18.370109 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 330/648] 10.222.0.18 () {56 vars in 1099 bytes} [Tue Sep 13 03:08:18 2022] GET /identity/v3/users/glance => generated 84 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 03:08:18.371660 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b07fd16c-8efe-4a88-a644-2735712ca09b None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:08:18.387840 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:08:18.397793 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ad7b9d8-4803-4a1e-bf7e-55eab244b015 None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:08:18.433866 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34ec129e-740a-4e07-b275-905a7a6b7359 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:08:18.434477 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ec129e-740a-4e07-b275-905a7a6b7359 None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:08:18.434800 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ec129e-740a-4e07-b275-905a7a6b7359 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:08:18.435059 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ec129e-740a-4e07-b275-905a7a6b7359 None admin] PATH_INFO: `/v3/domains` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:08:18.435727 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34ec129e-740a-4e07-b275-905a7a6b7359 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 03:08:18.436026 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34ec129e-740a-4e07-b275-905a7a6b7359 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:08:18.437450 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34ec129e-740a-4e07-b275-905a7a6b7359 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:08:18.450357 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 319/649] 10.222.0.18 () {56 vars in 1109 bytes} [Tue Sep 13 03:08:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 03:08:18.451032 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34ec129e-740a-4e07-b275-905a7a6b7359 None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:08:18.460111 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:08:18.483169 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b07fd16c-8efe-4a88-a644-2735712ca09b None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:08:18.517790 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dcf0b5d-612a-4cd6-8552-9b01c520fc0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:08:18.518417 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dcf0b5d-612a-4cd6-8552-9b01c520fc0c None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:08:18.518720 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dcf0b5d-612a-4cd6-8552-9b01c520fc0c None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:08:18.518991 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dcf0b5d-612a-4cd6-8552-9b01c520fc0c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:08:18.529577 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dcf0b5d-612a-4cd6-8552-9b01c520fc0c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:08:18.531622 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dcf0b5d-612a-4cd6-8552-9b01c520fc0c None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:08:18.546374 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application [None req-4dcf0b5d-612a-4cd6-8552-9b01c520fc0c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 03:08:18.546374 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 03:08:18.546374 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 03:08:18.546374 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 03:08:18.546374 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 03:08:18.546374 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 03:08:18.546374 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 03:08:18.546374 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 03:08:18.546374 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 03:08:18.546374 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 03:08:18.546374 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 03:08:18.546374 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 03:08:18.546374 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 13 03:08:18.546374 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 13 03:08:18.546374 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 13 03:08:18.546374 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 13 03:08:18.547548 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 03:08:18.547548 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 03:08:18.547548 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 03:08:18.547548 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 03:08:18.547548 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 03:08:18.547548 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 03:08:18.547548 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 03:08:18.547548 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application with Lock( Sep 13 03:08:18.547548 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 03:08:18.547548 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._enter() Sep 13 03:08:18.547548 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 03:08:18.547548 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 03:08:18.547548 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 03:08:18.547548 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.creator() Sep 13 03:08:18.547548 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 03:08:18.547548 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application created_value = creator( Sep 13 03:08:18.547548 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 13 03:08:18.548631 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 13 03:08:18.548631 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 03:08:18.548631 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Sep 13 03:08:18.548631 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 331/650] 10.222.0.18 () {56 vars in 1108 bytes} [Tue Sep 13 03:08:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 03:08:18.548631 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dcf0b5d-612a-4cd6-8552-9b01c520fc0c None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:08:18.560756 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34ec129e-740a-4e07-b275-905a7a6b7359 None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:08:18.599113 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50f65976-11f5-41ee-98d4-8682e4340473 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:08:18.599635 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50f65976-11f5-41ee-98d4-8682e4340473 None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:08:18.600235 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50f65976-11f5-41ee-98d4-8682e4340473 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:08:18.600512 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50f65976-11f5-41ee-98d4-8682e4340473 None admin] PATH_INFO: `/v3/projects` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:08:18.601346 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50f65976-11f5-41ee-98d4-8682e4340473 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 03:08:18.601652 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50f65976-11f5-41ee-98d4-8682e4340473 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:08:18.603431 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50f65976-11f5-41ee-98d4-8682e4340473 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:08:18.615848 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 320/651] 10.222.0.18 () {56 vars in 1148 bytes} [Tue Sep 13 03:08:18 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 03:08:18.616695 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50f65976-11f5-41ee-98d4-8682e4340473 None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:08:18.626228 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:08:18.665956 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dcf0b5d-612a-4cd6-8552-9b01c520fc0c None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:08:18.684936 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:08:18.707695 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a39ccddc-9404-4b09-b49a-11d3722dccb3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:08:18.708297 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a39ccddc-9404-4b09-b49a-11d3722dccb3 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:08:18.708570 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a39ccddc-9404-4b09-b49a-11d3722dccb3 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:08:18.708837 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a39ccddc-9404-4b09-b49a-11d3722dccb3 None admin] PATH_INFO: `/v3/users` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:08:18.709558 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a39ccddc-9404-4b09-b49a-11d3722dccb3 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=81400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 03:08:18.709861 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a39ccddc-9404-4b09-b49a-11d3722dccb3 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:08:18.711340 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a39ccddc-9404-4b09-b49a-11d3722dccb3 None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:08:18.722396 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50f65976-11f5-41ee-98d4-8682e4340473 None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:08:18.729994 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 332/652] 10.222.0.18 () {56 vars in 1137 bytes} [Tue Sep 13 03:08:18 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 03:08:18.731060 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a39ccddc-9404-4b09-b49a-11d3722dccb3 None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:08:18.759668 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-984e1b30-b4a2-4fcd-8f7a-11b67d441fb3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:08:18.760303 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-984e1b30-b4a2-4fcd-8f7a-11b67d441fb3 None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:08:18.760585 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-984e1b30-b4a2-4fcd-8f7a-11b67d441fb3 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:08:18.760857 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-984e1b30-b4a2-4fcd-8f7a-11b67d441fb3 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:08:18.769445 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-984e1b30-b4a2-4fcd-8f7a-11b67d441fb3 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:08:18.770912 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-984e1b30-b4a2-4fcd-8f7a-11b67d441fb3 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:08:18.782644 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application [None req-984e1b30-b4a2-4fcd-8f7a-11b67d441fb3 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Sep 13 03:08:18.782644 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 03:08:18.782644 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 03:08:18.782644 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 03:08:18.782644 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 03:08:18.782644 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 03:08:18.782644 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 03:08:18.782644 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 03:08:18.782644 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 03:08:18.782644 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 03:08:18.782644 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 03:08:18.782644 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 03:08:18.782644 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 13 03:08:18.782644 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 13 03:08:18.782644 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 13 03:08:18.782644 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 13 03:08:18.783710 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 03:08:18.783710 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 03:08:18.783710 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 13 03:08:18.783710 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 13 03:08:18.783710 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 13 03:08:18.783710 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 13 03:08:18.783710 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 03:08:18.783710 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 03:08:18.783710 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 03:08:18.783710 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 03:08:18.783710 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 03:08:18.783710 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application with Lock( Sep 13 03:08:18.783710 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 03:08:18.783710 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._enter() Sep 13 03:08:18.783710 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 03:08:18.783710 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 03:08:18.783710 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 03:08:18.783710 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.creator() Sep 13 03:08:18.784683 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 03:08:18.784683 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application created_value = creator( Sep 13 03:08:18.784683 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 13 03:08:18.784683 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 13 03:08:18.784683 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 13 03:08:18.784683 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 13 03:08:18.784683 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 13 03:08:18.784683 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 13 03:08:18.784683 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Sep 13 03:08:18.784683 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Sep 13 03:08:18.784683 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 321/653] 10.222.0.18 () {56 vars in 1099 bytes} [Tue Sep 13 03:08:18 2022] GET /identity/v3/users/cinder => generated 84 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 03:08:18.784683 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-984e1b30-b4a2-4fcd-8f7a-11b67d441fb3 None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:08:18.841634 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a39ccddc-9404-4b09-b49a-11d3722dccb3 None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:08:18.866793 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:08:18.878515 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-530399ca-e312-4bbe-a715-1c2ac2a9f693 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:08:18.879152 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-530399ca-e312-4bbe-a715-1c2ac2a9f693 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:08:18.879425 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-530399ca-e312-4bbe-a715-1c2ac2a9f693 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:08:18.879691 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-530399ca-e312-4bbe-a715-1c2ac2a9f693 None admin] PATH_INFO: `/v3/domains` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:08:18.880319 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-530399ca-e312-4bbe-a715-1c2ac2a9f693 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 03:08:18.880620 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-530399ca-e312-4bbe-a715-1c2ac2a9f693 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:08:18.881978 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-530399ca-e312-4bbe-a715-1c2ac2a9f693 None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:08:18.894545 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-984e1b30-b4a2-4fcd-8f7a-11b67d441fb3 None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:08:18.894851 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 333/654] 10.222.0.18 () {56 vars in 1109 bytes} [Tue Sep 13 03:08:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 03:08:18.899545 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-530399ca-e312-4bbe-a715-1c2ac2a9f693 None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:08:18.922577 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:08:18.933318 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e8aa7bb-99d1-490a-baca-7b2393b1b4b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:08:18.933921 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e8aa7bb-99d1-490a-baca-7b2393b1b4b5 None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:08:18.934200 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e8aa7bb-99d1-490a-baca-7b2393b1b4b5 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:08:18.934463 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e8aa7bb-99d1-490a-baca-7b2393b1b4b5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:08:18.935201 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e8aa7bb-99d1-490a-baca-7b2393b1b4b5 None admin] RBAC: Adding query filter params (role.id=817d6837cd044890b41cf16b303a092b, scope.project.id=b9d94f6a68f7470d9494f112175620dc, user.id=c7377f5bee4a49959182e1bd34bc97a4) {{(pid=81399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 03:08:18.935502 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e8aa7bb-99d1-490a-baca-7b2393b1b4b5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:08:18.937597 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e8aa7bb-99d1-490a-baca-7b2393b1b4b5 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:08:18.954646 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 322/655] 10.222.0.18 () {56 vars in 1374 bytes} [Tue Sep 13 03:08:18 2022] GET /identity/v3/role_assignments?user.id=c7377f5bee4a49959182e1bd34bc97a4&scope.project.id=b9d94f6a68f7470d9494f112175620dc&role.id=817d6837cd044890b41cf16b303a092b => generated 608 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 03:08:18.954646 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e8aa7bb-99d1-490a-baca-7b2393b1b4b5 None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:08:19.019223 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-530399ca-e312-4bbe-a715-1c2ac2a9f693 None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:08:19.064156 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7282ddf5-4054-46e8-934d-b99ee8d91e26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:08:19.064891 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7282ddf5-4054-46e8-934d-b99ee8d91e26 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:08:19.065291 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7282ddf5-4054-46e8-934d-b99ee8d91e26 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:08:19.068526 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7282ddf5-4054-46e8-934d-b99ee8d91e26 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:08:19.086465 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e8aa7bb-99d1-490a-baca-7b2393b1b4b5 None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:08:19.128415 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a95bc63-a542-4a6d-ab04-9cda8699a9ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:08:19.129176 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a95bc63-a542-4a6d-ab04-9cda8699a9ef None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:08:19.129511 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a95bc63-a542-4a6d-ab04-9cda8699a9ef None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:08:19.129879 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a95bc63-a542-4a6d-ab04-9cda8699a9ef None admin] PATH_INFO: `/v3/users` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:08:19.130631 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a95bc63-a542-4a6d-ab04-9cda8699a9ef None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=81399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 03:08:19.130940 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a95bc63-a542-4a6d-ab04-9cda8699a9ef None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:08:19.132472 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a95bc63-a542-4a6d-ab04-9cda8699a9ef None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:08:19.132920 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:08:19.153378 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7282ddf5-4054-46e8-934d-b99ee8d91e26 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:08:19.154789 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 323/656] 10.222.0.18 () {56 vars in 1137 bytes} [Tue Sep 13 03:08:18 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 03:08:19.154905 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7282ddf5-4054-46e8-934d-b99ee8d91e26 None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:08:19.155837 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a95bc63-a542-4a6d-ab04-9cda8699a9ef None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:08:19.213113 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:08:19.282805 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 334/657] 10.222.0.18 () {58 vars in 1282 bytes} [Tue Sep 13 03:08:18 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 388 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 13 03:08:19.283940 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a95bc63-a542-4a6d-ab04-9cda8699a9ef None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:08:19.284266 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7282ddf5-4054-46e8-934d-b99ee8d91e26 None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:08:19.328322 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7144bdb7-d5bf-413d-8c97-4c67098e01dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:08:19.329225 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7144bdb7-d5bf-413d-8c97-4c67098e01dc None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:08:19.329504 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7144bdb7-d5bf-413d-8c97-4c67098e01dc None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:08:19.329771 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7144bdb7-d5bf-413d-8c97-4c67098e01dc None admin] PATH_INFO: `/v3/users/neutron` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:08:19.337272 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7144bdb7-d5bf-413d-8c97-4c67098e01dc None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:08:19.338767 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7144bdb7-d5bf-413d-8c97-4c67098e01dc None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:08:19.352365 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application [None req-7144bdb7-d5bf-413d-8c97-4c67098e01dc None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Sep 13 03:08:19.352365 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 03:08:19.352365 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 03:08:19.352365 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 03:08:19.352365 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 03:08:19.352365 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 03:08:19.352365 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 03:08:19.352365 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 03:08:19.352365 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 03:08:19.352365 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 03:08:19.352365 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 03:08:19.352365 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 03:08:19.352365 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 13 03:08:19.352365 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 13 03:08:19.352365 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 13 03:08:19.352365 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 13 03:08:19.353381 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 03:08:19.353381 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 03:08:19.353381 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 13 03:08:19.353381 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 13 03:08:19.353381 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 13 03:08:19.353381 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 13 03:08:19.353381 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 03:08:19.353381 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 03:08:19.353381 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 03:08:19.353381 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 03:08:19.353381 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 03:08:19.353381 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application with Lock( Sep 13 03:08:19.353381 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 03:08:19.353381 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._enter() Sep 13 03:08:19.353381 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 03:08:19.353381 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 03:08:19.353381 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 03:08:19.353381 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.creator() Sep 13 03:08:19.354353 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 03:08:19.354353 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application created_value = creator( Sep 13 03:08:19.354353 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 13 03:08:19.354353 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 13 03:08:19.354353 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 13 03:08:19.354353 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 13 03:08:19.354353 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 13 03:08:19.354353 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 13 03:08:19.354353 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Sep 13 03:08:19.354353 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Sep 13 03:08:19.358155 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 324/658] 10.222.0.18 () {56 vars in 1102 bytes} [Tue Sep 13 03:08:19 2022] GET /identity/v3/users/neutron => generated 85 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 03:08:19.359737 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7144bdb7-d5bf-413d-8c97-4c67098e01dc None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:08:19.395232 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:08:19.414720 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7282ddf5-4054-46e8-934d-b99ee8d91e26 None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:08:19.450095 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d38329a5-5d7a-47df-984f-94b07bfed44b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:08:19.450731 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d38329a5-5d7a-47df-984f-94b07bfed44b None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:08:19.451012 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d38329a5-5d7a-47df-984f-94b07bfed44b None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:08:19.451285 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d38329a5-5d7a-47df-984f-94b07bfed44b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:08:19.470562 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7144bdb7-d5bf-413d-8c97-4c67098e01dc None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:08:19.480278 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:08:19.508751 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fbbac38-1041-4284-b5ea-e510b9e3b68a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:08:19.509389 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fbbac38-1041-4284-b5ea-e510b9e3b68a None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:08:19.509662 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fbbac38-1041-4284-b5ea-e510b9e3b68a None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:08:19.509925 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fbbac38-1041-4284-b5ea-e510b9e3b68a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:08:19.520238 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fbbac38-1041-4284-b5ea-e510b9e3b68a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:08:19.521647 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fbbac38-1041-4284-b5ea-e510b9e3b68a None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:08:19.524445 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d38329a5-5d7a-47df-984f-94b07bfed44b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:08:19.526062 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d38329a5-5d7a-47df-984f-94b07bfed44b None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:08:19.535611 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application [None req-4fbbac38-1041-4284-b5ea-e510b9e3b68a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 03:08:19.535611 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 03:08:19.535611 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 03:08:19.535611 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 03:08:19.535611 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 03:08:19.535611 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 03:08:19.535611 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 03:08:19.535611 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 03:08:19.535611 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 03:08:19.535611 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 03:08:19.535611 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 03:08:19.535611 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 03:08:19.535611 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 13 03:08:19.535611 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 13 03:08:19.535611 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 13 03:08:19.535611 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 13 03:08:19.536753 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 03:08:19.536753 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 03:08:19.536753 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 03:08:19.536753 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 03:08:19.536753 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 03:08:19.536753 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 03:08:19.536753 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 03:08:19.536753 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application with Lock( Sep 13 03:08:19.536753 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 03:08:19.536753 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._enter() Sep 13 03:08:19.536753 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 03:08:19.536753 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 03:08:19.536753 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 03:08:19.536753 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.creator() Sep 13 03:08:19.536753 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 03:08:19.536753 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application created_value = creator( Sep 13 03:08:19.536753 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 13 03:08:19.537683 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 13 03:08:19.537683 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 03:08:19.537683 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Sep 13 03:08:19.537683 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 325/659] 10.222.0.18 () {56 vars in 1108 bytes} [Tue Sep 13 03:08:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 03:08:19.537683 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fbbac38-1041-4284-b5ea-e510b9e3b68a None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:08:19.631635 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:08:19.647116 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fbbac38-1041-4284-b5ea-e510b9e3b68a None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:08:19.653950 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 335/660] 10.222.0.18 () {58 vars in 1282 bytes} [Tue Sep 13 03:08:19 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 370 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 13 03:08:19.655833 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba27eecd-aea2-4092-a83e-eadbb5e43ff3 None None] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:08:19.656238 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba27eecd-aea2-4092-a83e-eadbb5e43ff3 None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:08:19.656613 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba27eecd-aea2-4092-a83e-eadbb5e43ff3 None None] PATH_INFO: `/` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:08:19.658659 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 336/661] 10.222.0.18 () {54 vars in 927 bytes} [Tue Sep 13 03:08:19 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 03:08:19.658807 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba27eecd-aea2-4092-a83e-eadbb5e43ff3 None None] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:08:19.691379 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b45a57ed-67ed-455e-89e6-e5363fb99765 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:08:19.692530 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b45a57ed-67ed-455e-89e6-e5363fb99765 None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:08:19.692955 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b45a57ed-67ed-455e-89e6-e5363fb99765 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:08:19.693384 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b45a57ed-67ed-455e-89e6-e5363fb99765 None admin] PATH_INFO: `/v3/users` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:08:19.694262 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b45a57ed-67ed-455e-89e6-e5363fb99765 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=81399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 03:08:19.694743 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b45a57ed-67ed-455e-89e6-e5363fb99765 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:08:19.697405 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b45a57ed-67ed-455e-89e6-e5363fb99765 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:08:19.723004 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 326/662] 10.222.0.18 () {56 vars in 1139 bytes} [Tue Sep 13 03:08:19 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 03:08:19.723861 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b45a57ed-67ed-455e-89e6-e5363fb99765 None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:08:19.743243 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:08:19.768804 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba27eecd-aea2-4092-a83e-eadbb5e43ff3 None None] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:08:19.805420 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d55feadd-9ba2-424c-9c7f-1475a34d0e06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:08:19.806022 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d55feadd-9ba2-424c-9c7f-1475a34d0e06 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:08:19.806294 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d55feadd-9ba2-424c-9c7f-1475a34d0e06 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:08:19.806560 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d55feadd-9ba2-424c-9c7f-1475a34d0e06 None admin] PATH_INFO: `/v3/domains` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:08:19.807217 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d55feadd-9ba2-424c-9c7f-1475a34d0e06 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 03:08:19.807514 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d55feadd-9ba2-424c-9c7f-1475a34d0e06 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:08:19.808866 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d55feadd-9ba2-424c-9c7f-1475a34d0e06 None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:08:19.820022 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 337/663] 10.222.0.18 () {56 vars in 1109 bytes} [Tue Sep 13 03:08:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 03:08:19.821199 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f29ac73e-8979-4e3e-928a-c5b748e69d77 None None] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:08:19.821473 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f29ac73e-8979-4e3e-928a-c5b748e69d77 None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:08:19.821740 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f29ac73e-8979-4e3e-928a-c5b748e69d77 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:08:19.835594 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b45a57ed-67ed-455e-89e6-e5363fb99765 None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:08:19.841326 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.auth.core [None req-f29ac73e-8979-4e3e-928a-c5b748e69d77 None None] MFA Rules not processed for user `49f280ab086b4610855c4ffa378517d9`. Rule list: `[]` (Enabled: `True`). {{(pid=81400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 03:08:19.842512 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-f29ac73e-8979-4e3e-928a-c5b748e69d77 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:08:19.875112 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb7c378c-8935-4757-929a-178948a402ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:08:19.876078 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb7c378c-8935-4757-929a-178948a402ab None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:08:19.876549 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb7c378c-8935-4757-929a-178948a402ab None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:08:19.876824 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb7c378c-8935-4757-929a-178948a402ab None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:08:19.884668 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:08:19.887945 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb7c378c-8935-4757-929a-178948a402ab None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:08:19.889394 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb7c378c-8935-4757-929a-178948a402ab None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:08:19.904854 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application [None req-bb7c378c-8935-4757-929a-178948a402ab None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 03:08:19.904854 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 03:08:19.904854 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 03:08:19.904854 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 03:08:19.904854 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 03:08:19.904854 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 03:08:19.904854 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 03:08:19.904854 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 03:08:19.904854 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 03:08:19.904854 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 03:08:19.904854 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 03:08:19.904854 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 03:08:19.904854 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 13 03:08:19.904854 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 13 03:08:19.904854 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 13 03:08:19.904854 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 13 03:08:19.906340 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 03:08:19.906340 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 03:08:19.906340 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 03:08:19.906340 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 03:08:19.906340 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 03:08:19.906340 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 03:08:19.906340 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 03:08:19.906340 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application with Lock( Sep 13 03:08:19.906340 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 03:08:19.906340 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._enter() Sep 13 03:08:19.906340 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 03:08:19.906340 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 03:08:19.906340 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 03:08:19.906340 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.creator() Sep 13 03:08:19.906340 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 03:08:19.906340 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application created_value = creator( Sep 13 03:08:19.906340 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 13 03:08:19.907757 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 13 03:08:19.907757 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 03:08:19.907757 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Sep 13 03:08:19.907757 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 327/664] 10.222.0.18 () {56 vars in 1108 bytes} [Tue Sep 13 03:08:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 03:08:19.907757 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb7c378c-8935-4757-929a-178948a402ab None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:08:19.920554 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 338/665] 10.222.0.18 () {58 vars in 1024 bytes} [Tue Sep 13 03:08:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 13 03:08:19.922050 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f29ac73e-8979-4e3e-928a-c5b748e69d77 None None] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:08:20.007538 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:08:20.026416 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f29ac73e-8979-4e3e-928a-c5b748e69d77 None None] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:08:20.052860 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb7c378c-8935-4757-929a-178948a402ab None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:08:20.067920 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47308379-ce40-49b6-a6fd-ec9a5c8a3027 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:08:20.069100 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47308379-ce40-49b6-a6fd-ec9a5c8a3027 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:08:20.070001 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47308379-ce40-49b6-a6fd-ec9a5c8a3027 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:08:20.070453 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47308379-ce40-49b6-a6fd-ec9a5c8a3027 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:08:20.079837 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47308379-ce40-49b6-a6fd-ec9a5c8a3027 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:08:20.081955 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47308379-ce40-49b6-a6fd-ec9a5c8a3027 None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:08:20.094171 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8f6e0b5-874a-457a-9ca1-26759cdaeba8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:08:20.095022 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8f6e0b5-874a-457a-9ca1-26759cdaeba8 None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:08:20.095531 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8f6e0b5-874a-457a-9ca1-26759cdaeba8 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:08:20.095993 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8f6e0b5-874a-457a-9ca1-26759cdaeba8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:08:20.097526 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application [None req-47308379-ce40-49b6-a6fd-ec9a5c8a3027 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 13 03:08:20.097526 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 03:08:20.097526 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 03:08:20.097526 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 03:08:20.097526 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 03:08:20.097526 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 03:08:20.097526 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 03:08:20.097526 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 03:08:20.097526 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 03:08:20.097526 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 03:08:20.097526 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 03:08:20.097526 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 03:08:20.097526 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 13 03:08:20.097526 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 13 03:08:20.097526 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 13 03:08:20.098546 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 13 03:08:20.098546 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 03:08:20.098546 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 03:08:20.098546 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 03:08:20.098546 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 03:08:20.098546 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 03:08:20.098546 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 03:08:20.098546 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 03:08:20.098546 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application with Lock( Sep 13 03:08:20.098546 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 03:08:20.098546 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._enter() Sep 13 03:08:20.098546 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 03:08:20.098546 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 03:08:20.098546 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 03:08:20.098546 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.creator() Sep 13 03:08:20.098546 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 03:08:20.098546 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application created_value = creator( Sep 13 03:08:20.099686 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 13 03:08:20.099686 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 13 03:08:20.099686 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 13 03:08:20.099686 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 13 03:08:20.099686 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 13 03:08:20.099686 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 13 03:08:20.099686 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 13 03:08:20.099686 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Sep 13 03:08:20.102741 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 339/666] 10.222.0.18 () {56 vars in 1111 bytes} [Tue Sep 13 03:08:19 2022] GET /identity/v3/projects/service => generated 88 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 03:08:20.103644 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47308379-ce40-49b6-a6fd-ec9a5c8a3027 None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:08:20.149574 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:08:20.175654 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8f6e0b5-874a-457a-9ca1-26759cdaeba8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:08:20.177292 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8f6e0b5-874a-457a-9ca1-26759cdaeba8 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:08:20.230371 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47308379-ce40-49b6-a6fd-ec9a5c8a3027 None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:08:20.249152 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:08:20.266348 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a9dbb43-5096-4b27-a2f0-8eebabf5c60f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:08:20.266980 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a9dbb43-5096-4b27-a2f0-8eebabf5c60f None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:08:20.267268 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a9dbb43-5096-4b27-a2f0-8eebabf5c60f None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:08:20.267539 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a9dbb43-5096-4b27-a2f0-8eebabf5c60f None admin] PATH_INFO: `/v3/domains` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:08:20.268175 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a9dbb43-5096-4b27-a2f0-8eebabf5c60f None admin] RBAC: Adding query filter params (name=Default) {{(pid=81400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 03:08:20.268480 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a9dbb43-5096-4b27-a2f0-8eebabf5c60f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:08:20.269848 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a9dbb43-5096-4b27-a2f0-8eebabf5c60f None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:08:20.281726 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 340/667] 10.222.0.18 () {56 vars in 1109 bytes} [Tue Sep 13 03:08:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 03:08:20.289561 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 328/668] 10.222.0.18 () {58 vars in 1282 bytes} [Tue Sep 13 03:08:19 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 378 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 13 03:08:20.289561 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8f6e0b5-874a-457a-9ca1-26759cdaeba8 None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:08:20.289809 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a9dbb43-5096-4b27-a2f0-8eebabf5c60f None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:08:20.383511 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:08:20.391232 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a9dbb43-5096-4b27-a2f0-8eebabf5c60f None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:08:20.395086 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8f6e0b5-874a-457a-9ca1-26759cdaeba8 None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:08:20.425338 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d20a9005-7a81-4b3c-8a9c-37ea12498989 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:08:20.426033 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d20a9005-7a81-4b3c-8a9c-37ea12498989 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:08:20.426253 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d20a9005-7a81-4b3c-8a9c-37ea12498989 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:08:20.426531 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d20a9005-7a81-4b3c-8a9c-37ea12498989 None admin] PATH_INFO: `/v3/projects` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:08:20.427241 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d20a9005-7a81-4b3c-8a9c-37ea12498989 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 03:08:20.427546 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d20a9005-7a81-4b3c-8a9c-37ea12498989 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:08:20.428973 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d20a9005-7a81-4b3c-8a9c-37ea12498989 None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:08:20.431700 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1180fa74-9904-4798-b6dd-80880c952372 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:08:20.432328 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1180fa74-9904-4798-b6dd-80880c952372 None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:08:20.432601 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1180fa74-9904-4798-b6dd-80880c952372 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:08:20.432866 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1180fa74-9904-4798-b6dd-80880c952372 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:08:20.446703 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 341/669] 10.222.0.18 () {56 vars in 1148 bytes} [Tue Sep 13 03:08:20 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 03:08:20.446703 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d20a9005-7a81-4b3c-8a9c-37ea12498989 None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:08:20.448159 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1180fa74-9904-4798-b6dd-80880c952372 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 03:08:20.453596 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application [None req-1180fa74-9904-4798-b6dd-80880c952372 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 13 03:08:20.453596 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 03:08:20.453596 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 03:08:20.453596 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 03:08:20.453596 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 03:08:20.453596 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 03:08:20.453596 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 03:08:20.453596 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 03:08:20.453596 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 03:08:20.453596 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 03:08:20.453596 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 03:08:20.453596 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 03:08:20.453596 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 03:08:20.453596 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 03:08:20.453596 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 03:08:20.453596 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 03:08:20.454547 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 03:08:20.454547 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 03:08:20.454547 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 03:08:20.454547 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 03:08:20.454547 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 03:08:20.454547 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 03:08:20.454547 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 03:08:20.454547 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 03:08:20.454547 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 03:08:20.454547 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 03:08:20.454547 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 03:08:20.454547 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application with Lock( Sep 13 03:08:20.454547 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 03:08:20.454547 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._enter() Sep 13 03:08:20.454547 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 03:08:20.454547 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 03:08:20.459760 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 03:08:20.459760 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.creator() Sep 13 03:08:20.459760 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 03:08:20.459760 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application created_value = creator( Sep 13 03:08:20.459760 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 03:08:20.459760 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 03:08:20.459760 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 03:08:20.459760 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 03:08:20.459760 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 03:08:20.459760 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 03:08:20.459760 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 13 03:08:20.459760 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Sep 13 03:08:20.459760 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 329/670] 10.222.0.18 () {56 vars in 1102 bytes} [Tue Sep 13 03:08:20 2022] GET /identity/v3/roles/service => generated 85 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 03:08:20.459760 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1180fa74-9904-4798-b6dd-80880c952372 None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:08:20.489228 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:08:20.550962 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d20a9005-7a81-4b3c-8a9c-37ea12498989 None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:08:20.560356 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1180fa74-9904-4798-b6dd-80880c952372 None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:08:20.585392 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a702b387-f198-4a6c-b59a-3868d1cedca7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:08:20.585997 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a702b387-f198-4a6c-b59a-3868d1cedca7 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:08:20.586271 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a702b387-f198-4a6c-b59a-3868d1cedca7 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:08:20.586537 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a702b387-f198-4a6c-b59a-3868d1cedca7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:08:20.594902 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72480df9-ef03-4bd1-81f6-23e0c3a5726d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:08:20.595530 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72480df9-ef03-4bd1-81f6-23e0c3a5726d None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:08:20.595800 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72480df9-ef03-4bd1-81f6-23e0c3a5726d None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:08:20.596078 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72480df9-ef03-4bd1-81f6-23e0c3a5726d None admin] PATH_INFO: `/v3/projects/service` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:08:20.596674 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a702b387-f198-4a6c-b59a-3868d1cedca7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:08:20.598092 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a702b387-f198-4a6c-b59a-3868d1cedca7 None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:08:20.602785 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72480df9-ef03-4bd1-81f6-23e0c3a5726d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:08:20.604201 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72480df9-ef03-4bd1-81f6-23e0c3a5726d None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:08:20.612452 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application [None req-a702b387-f198-4a6c-b59a-3868d1cedca7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 03:08:20.612452 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 03:08:20.612452 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 03:08:20.612452 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 03:08:20.612452 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 03:08:20.612452 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 03:08:20.612452 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 03:08:20.612452 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 03:08:20.612452 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 03:08:20.612452 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 03:08:20.612452 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 03:08:20.612452 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 03:08:20.612452 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 13 03:08:20.612452 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 13 03:08:20.612452 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 13 03:08:20.612452 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 13 03:08:20.613415 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 03:08:20.613415 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 03:08:20.613415 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 03:08:20.613415 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 03:08:20.613415 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 03:08:20.613415 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 03:08:20.613415 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 03:08:20.613415 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application with Lock( Sep 13 03:08:20.613415 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 03:08:20.613415 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._enter() Sep 13 03:08:20.613415 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 03:08:20.613415 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 03:08:20.613415 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 03:08:20.613415 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.creator() Sep 13 03:08:20.613415 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 03:08:20.613415 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application created_value = creator( Sep 13 03:08:20.613415 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 13 03:08:20.614328 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 13 03:08:20.614328 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 03:08:20.614328 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Sep 13 03:08:20.615342 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application [None req-72480df9-ef03-4bd1-81f6-23e0c3a5726d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 13 03:08:20.615342 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 03:08:20.615342 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 03:08:20.615342 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 03:08:20.615342 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 03:08:20.615342 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 03:08:20.615342 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 03:08:20.615342 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 03:08:20.615342 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 03:08:20.615342 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 03:08:20.615342 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 03:08:20.615342 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 03:08:20.615342 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 13 03:08:20.615342 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 13 03:08:20.615342 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 13 03:08:20.616272 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 13 03:08:20.616272 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 03:08:20.616272 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 03:08:20.616272 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 03:08:20.616272 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 03:08:20.616272 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 03:08:20.616272 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 03:08:20.616272 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 03:08:20.616272 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application with Lock( Sep 13 03:08:20.616272 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 03:08:20.616272 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._enter() Sep 13 03:08:20.616272 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 03:08:20.616272 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 03:08:20.616272 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 03:08:20.616272 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.creator() Sep 13 03:08:20.616272 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 03:08:20.616272 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application created_value = creator( Sep 13 03:08:20.622370 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 13 03:08:20.622370 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 13 03:08:20.622370 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 13 03:08:20.622370 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 13 03:08:20.622370 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 13 03:08:20.622370 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 13 03:08:20.622370 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 13 03:08:20.622370 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Sep 13 03:08:20.622370 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 330/671] 10.222.0.18 () {56 vars in 1111 bytes} [Tue Sep 13 03:08:20 2022] GET /identity/v3/projects/service => generated 88 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 03:08:20.622370 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72480df9-ef03-4bd1-81f6-23e0c3a5726d None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:08:20.622965 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 342/672] 10.222.0.18 () {56 vars in 1108 bytes} [Tue Sep 13 03:08:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 03:08:20.622965 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a702b387-f198-4a6c-b59a-3868d1cedca7 None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:08:20.630889 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:08:20.720524 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:08:20.727824 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a702b387-f198-4a6c-b59a-3868d1cedca7 None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:08:20.731356 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72480df9-ef03-4bd1-81f6-23e0c3a5726d None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:08:20.763070 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96631379-ab4c-404c-910d-8bd4f6556d2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:08:20.763668 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96631379-ab4c-404c-910d-8bd4f6556d2c None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:08:20.763942 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96631379-ab4c-404c-910d-8bd4f6556d2c None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:08:20.764212 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96631379-ab4c-404c-910d-8bd4f6556d2c None admin] PATH_INFO: `/v3/roles` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:08:20.765412 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96631379-ab4c-404c-910d-8bd4f6556d2c None admin] RBAC: Adding query filter params (name=service) {{(pid=81400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 03:08:20.765707 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96631379-ab4c-404c-910d-8bd4f6556d2c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:08:20.767123 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96631379-ab4c-404c-910d-8bd4f6556d2c None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:08:20.769802 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-124700e8-f3c8-44ae-83df-38508648937f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:08:20.770639 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-124700e8-f3c8-44ae-83df-38508648937f None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:08:20.771100 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-124700e8-f3c8-44ae-83df-38508648937f None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:08:20.771550 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-124700e8-f3c8-44ae-83df-38508648937f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:08:20.772453 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-124700e8-f3c8-44ae-83df-38508648937f None admin] RBAC: Adding query filter params (role.id=817d6837cd044890b41cf16b303a092b, scope.project.id=b9d94f6a68f7470d9494f112175620dc, user.id=b5fdc2ff46a14638b409a13e73ba5500) {{(pid=81399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 03:08:20.772833 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-124700e8-f3c8-44ae-83df-38508648937f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:08:20.774258 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-124700e8-f3c8-44ae-83df-38508648937f None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:08:20.775534 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 343/673] 10.222.0.18 () {56 vars in 1103 bytes} [Tue Sep 13 03:08:20 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 03:08:20.776150 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96631379-ab4c-404c-910d-8bd4f6556d2c None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:08:20.794640 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 331/674] 10.222.0.18 () {56 vars in 1374 bytes} [Tue Sep 13 03:08:20 2022] GET /identity/v3/role_assignments?user.id=b5fdc2ff46a14638b409a13e73ba5500&scope.project.id=b9d94f6a68f7470d9494f112175620dc&role.id=817d6837cd044890b41cf16b303a092b => generated 608 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 03:08:20.794640 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-124700e8-f3c8-44ae-83df-38508648937f None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:08:20.883600 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:08:20.932942 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-124700e8-f3c8-44ae-83df-38508648937f None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:08:20.938168 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96631379-ab4c-404c-910d-8bd4f6556d2c None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:08:20.978760 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-388462a9-c7bb-4a46-8fcb-f51e8b4347f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:08:20.979299 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-388462a9-c7bb-4a46-8fcb-f51e8b4347f5 None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:08:20.979574 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-388462a9-c7bb-4a46-8fcb-f51e8b4347f5 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:08:20.979841 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-388462a9-c7bb-4a46-8fcb-f51e8b4347f5 None admin] PATH_INFO: `/v3/domains` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:08:20.980471 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-388462a9-c7bb-4a46-8fcb-f51e8b4347f5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 03:08:20.980771 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-388462a9-c7bb-4a46-8fcb-f51e8b4347f5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:08:20.982124 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-388462a9-c7bb-4a46-8fcb-f51e8b4347f5 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:08:20.983116 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bb53d9f-63b4-4e72-ab11-eff950f29bab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:08:20.984324 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb53d9f-63b4-4e72-ab11-eff950f29bab None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:08:20.984610 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb53d9f-63b4-4e72-ab11-eff950f29bab None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:08:20.984875 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb53d9f-63b4-4e72-ab11-eff950f29bab None admin] PATH_INFO: `/v3/projects` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:08:20.985571 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bb53d9f-63b4-4e72-ab11-eff950f29bab None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 03:08:20.985876 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bb53d9f-63b4-4e72-ab11-eff950f29bab None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:08:20.987432 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bb53d9f-63b4-4e72-ab11-eff950f29bab None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:08:20.995440 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 332/675] 10.222.0.18 () {56 vars in 1109 bytes} [Tue Sep 13 03:08:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 03:08:20.996389 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-388462a9-c7bb-4a46-8fcb-f51e8b4347f5 None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:08:21.002723 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 344/676] 10.222.0.18 () {56 vars in 1148 bytes} [Tue Sep 13 03:08:20 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 03:08:21.003627 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bb53d9f-63b4-4e72-ab11-eff950f29bab None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:08:21.021076 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:08:21.129181 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-388462a9-c7bb-4a46-8fcb-f51e8b4347f5 None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:08:21.142191 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bb53d9f-63b4-4e72-ab11-eff950f29bab None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:08:21.168961 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c00f20c-91aa-4ac9-acf5-1047f8d33979 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:08:21.169896 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c00f20c-91aa-4ac9-acf5-1047f8d33979 None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:08:21.170357 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c00f20c-91aa-4ac9-acf5-1047f8d33979 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:08:21.170874 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c00f20c-91aa-4ac9-acf5-1047f8d33979 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:08:21.176508 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:08:21.188825 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29c7d032-2d61-41a4-a93c-8423c47ff984 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:08:21.189350 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c00f20c-91aa-4ac9-acf5-1047f8d33979 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:08:21.189651 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29c7d032-2d61-41a4-a93c-8423c47ff984 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:08:21.189998 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29c7d032-2d61-41a4-a93c-8423c47ff984 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:08:21.190340 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29c7d032-2d61-41a4-a93c-8423c47ff984 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:08:21.191914 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c00f20c-91aa-4ac9-acf5-1047f8d33979 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:08:21.199216 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29c7d032-2d61-41a4-a93c-8423c47ff984 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:08:21.201511 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29c7d032-2d61-41a4-a93c-8423c47ff984 None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:08:21.211397 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application [None req-4c00f20c-91aa-4ac9-acf5-1047f8d33979 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 03:08:21.211397 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 03:08:21.211397 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 03:08:21.211397 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 03:08:21.211397 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 03:08:21.211397 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 03:08:21.211397 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 03:08:21.211397 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 03:08:21.211397 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 03:08:21.211397 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 03:08:21.211397 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 03:08:21.211397 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 03:08:21.211397 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 13 03:08:21.211397 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 13 03:08:21.211397 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 13 03:08:21.211397 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 13 03:08:21.212435 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 03:08:21.212435 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 03:08:21.212435 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 03:08:21.212435 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 03:08:21.212435 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 03:08:21.212435 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 03:08:21.212435 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 03:08:21.212435 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application with Lock( Sep 13 03:08:21.212435 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 03:08:21.212435 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._enter() Sep 13 03:08:21.212435 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 03:08:21.212435 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 03:08:21.212435 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 03:08:21.212435 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.creator() Sep 13 03:08:21.212435 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 03:08:21.212435 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application created_value = creator( Sep 13 03:08:21.212435 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 13 03:08:21.213369 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 13 03:08:21.213369 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 03:08:21.213369 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Sep 13 03:08:21.220029 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application [None req-29c7d032-2d61-41a4-a93c-8423c47ff984 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 13 03:08:21.220029 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 03:08:21.220029 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 03:08:21.220029 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 03:08:21.220029 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 03:08:21.220029 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 03:08:21.220029 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 03:08:21.220029 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 03:08:21.220029 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 03:08:21.220029 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 03:08:21.220029 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 03:08:21.220029 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 03:08:21.220029 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 13 03:08:21.220029 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 13 03:08:21.220029 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 13 03:08:21.220963 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 13 03:08:21.220963 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 03:08:21.220963 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 03:08:21.220963 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 03:08:21.220963 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 03:08:21.220963 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 03:08:21.220963 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 03:08:21.220963 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 03:08:21.220963 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application with Lock( Sep 13 03:08:21.220963 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 03:08:21.220963 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._enter() Sep 13 03:08:21.220963 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 03:08:21.220963 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 03:08:21.220963 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 03:08:21.220963 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.creator() Sep 13 03:08:21.220963 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 03:08:21.220963 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application created_value = creator( Sep 13 03:08:21.222141 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 13 03:08:21.222141 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 13 03:08:21.222141 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 13 03:08:21.222141 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 13 03:08:21.222141 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 13 03:08:21.222141 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 13 03:08:21.222141 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 13 03:08:21.222141 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Sep 13 03:08:21.222141 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 345/678] 10.222.0.18 () {56 vars in 1111 bytes} [Tue Sep 13 03:08:21 2022] GET /identity/v3/projects/service => generated 88 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 03:08:21.222141 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29c7d032-2d61-41a4-a93c-8423c47ff984 None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:08:21.223025 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 333/678] 10.222.0.18 () {56 vars in 1108 bytes} [Tue Sep 13 03:08:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 03:08:21.224268 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c00f20c-91aa-4ac9-acf5-1047f8d33979 None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:08:21.302185 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:08:21.342489 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c00f20c-91aa-4ac9-acf5-1047f8d33979 None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:08:21.348893 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29c7d032-2d61-41a4-a93c-8423c47ff984 None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:08:21.377738 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6888e53d-f842-4901-809f-fe09a00c9f91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:08:21.378337 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6888e53d-f842-4901-809f-fe09a00c9f91 None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:08:21.378643 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6888e53d-f842-4901-809f-fe09a00c9f91 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:08:21.378926 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6888e53d-f842-4901-809f-fe09a00c9f91 None admin] PATH_INFO: `/v3/projects` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:08:21.379608 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6888e53d-f842-4901-809f-fe09a00c9f91 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 03:08:21.379911 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6888e53d-f842-4901-809f-fe09a00c9f91 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:08:21.381341 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6888e53d-f842-4901-809f-fe09a00c9f91 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:08:21.384430 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4259da5-eb89-42f8-a8c6-36b92a1a2e01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:08:21.385036 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4259da5-eb89-42f8-a8c6-36b92a1a2e01 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:08:21.385316 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4259da5-eb89-42f8-a8c6-36b92a1a2e01 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:08:21.385588 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4259da5-eb89-42f8-a8c6-36b92a1a2e01 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:08:21.386304 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4259da5-eb89-42f8-a8c6-36b92a1a2e01 None admin] RBAC: Adding query filter params (role.id=817d6837cd044890b41cf16b303a092b, scope.project.id=b9d94f6a68f7470d9494f112175620dc, user.id=924ee3f608d34a18b0ddc4695f102a56) {{(pid=81400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 03:08:21.386632 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4259da5-eb89-42f8-a8c6-36b92a1a2e01 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:08:21.388044 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4259da5-eb89-42f8-a8c6-36b92a1a2e01 None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:08:21.394063 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 334/679] 10.222.0.18 () {56 vars in 1148 bytes} [Tue Sep 13 03:08:21 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 03:08:21.402179 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 346/680] 10.222.0.18 () {56 vars in 1374 bytes} [Tue Sep 13 03:08:21 2022] GET /identity/v3/role_assignments?user.id=924ee3f608d34a18b0ddc4695f102a56&scope.project.id=b9d94f6a68f7470d9494f112175620dc&role.id=817d6837cd044890b41cf16b303a092b => generated 608 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 03:08:21.402361 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6888e53d-f842-4901-809f-fe09a00c9f91 None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:08:21.402462 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a425fc4-c9fe-45ec-a82e-5bcd78d15f81 None None] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:08:21.402462 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a425fc4-c9fe-45ec-a82e-5bcd78d15f81 None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:08:21.402462 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a425fc4-c9fe-45ec-a82e-5bcd78d15f81 None None] PATH_INFO: `/` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:08:21.402462 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 347/681] 10.222.0.18 () {54 vars in 927 bytes} [Tue Sep 13 03:08:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 03:08:21.402759 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a425fc4-c9fe-45ec-a82e-5bcd78d15f81 None None] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:08:21.476185 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:08:21.522137 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6888e53d-f842-4901-809f-fe09a00c9f91 None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:08:21.546786 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a425fc4-c9fe-45ec-a82e-5bcd78d15f81 None None] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:08:21.551133 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:08:21.562494 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56203843-3c17-4128-9c56-8a88a3003520 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:08:21.563058 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56203843-3c17-4128-9c56-8a88a3003520 None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:08:21.563330 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56203843-3c17-4128-9c56-8a88a3003520 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:08:21.563594 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56203843-3c17-4128-9c56-8a88a3003520 None admin] PATH_INFO: `/v3/domains` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:08:21.564229 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56203843-3c17-4128-9c56-8a88a3003520 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 03:08:21.564525 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56203843-3c17-4128-9c56-8a88a3003520 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:08:21.565909 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56203843-3c17-4128-9c56-8a88a3003520 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:08:21.578477 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 335/682] 10.222.0.18 () {56 vars in 1109 bytes} [Tue Sep 13 03:08:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 03:08:21.579653 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f8547f3-118d-4b5b-b1a3-e070882e0f02 None None] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:08:21.579947 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f8547f3-118d-4b5b-b1a3-e070882e0f02 None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:08:21.580217 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f8547f3-118d-4b5b-b1a3-e070882e0f02 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:08:21.595360 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d697f78-0638-4ef4-b660-9b16e5ca993a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:08:21.596299 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d697f78-0638-4ef4-b660-9b16e5ca993a None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:08:21.596766 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d697f78-0638-4ef4-b660-9b16e5ca993a None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:08:21.597213 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d697f78-0638-4ef4-b660-9b16e5ca993a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:08:21.598146 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d697f78-0638-4ef4-b660-9b16e5ca993a None admin] RBAC: Adding query filter params (role.id=817d6837cd044890b41cf16b303a092b, scope.project.id=b9d94f6a68f7470d9494f112175620dc, user.id=cdffbc851086441ebd01022d1e4c2609) {{(pid=81400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 03:08:21.598675 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d697f78-0638-4ef4-b660-9b16e5ca993a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:08:21.599194 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.auth.core [None req-6f8547f3-118d-4b5b-b1a3-e070882e0f02 None None] MFA Rules not processed for user `49f280ab086b4610855c4ffa378517d9`. Rule list: `[]` (Enabled: `True`). {{(pid=81399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 03:08:21.600365 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-6f8547f3-118d-4b5b-b1a3-e070882e0f02 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:08:21.600638 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d697f78-0638-4ef4-b660-9b16e5ca993a None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:08:21.607708 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 348/683] 10.222.0.18 () {56 vars in 1374 bytes} [Tue Sep 13 03:08:21 2022] GET /identity/v3/role_assignments?user.id=cdffbc851086441ebd01022d1e4c2609&scope.project.id=b9d94f6a68f7470d9494f112175620dc&role.id=817d6837cd044890b41cf16b303a092b => generated 608 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 03:08:21.609127 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d697f78-0638-4ef4-b660-9b16e5ca993a None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:08:21.696610 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 336/684] 10.222.0.18 () {58 vars in 1024 bytes} [Tue Sep 13 03:08:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 13 03:08:21.785212 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d697f78-0638-4ef4-b660-9b16e5ca993a None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:08:21.811007 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:08:21.833591 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc5e2308-a079-4584-b4f3-43c0a0459fef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:08:21.834285 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc5e2308-a079-4584-b4f3-43c0a0459fef None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:08:21.834721 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc5e2308-a079-4584-b4f3-43c0a0459fef None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:08:21.835088 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc5e2308-a079-4584-b4f3-43c0a0459fef None admin] PATH_INFO: `/v3/users/swift` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:08:21.844830 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc5e2308-a079-4584-b4f3-43c0a0459fef None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:08:21.846411 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc5e2308-a079-4584-b4f3-43c0a0459fef None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:08:21.859736 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application [None req-fc5e2308-a079-4584-b4f3-43c0a0459fef None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Sep 13 03:08:21.859736 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 03:08:21.859736 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 03:08:21.859736 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 03:08:21.859736 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 03:08:21.859736 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 03:08:21.859736 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 03:08:21.859736 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 03:08:21.859736 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 03:08:21.859736 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 03:08:21.859736 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 03:08:21.859736 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 03:08:21.859736 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 13 03:08:21.859736 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 13 03:08:21.859736 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 13 03:08:21.859736 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 13 03:08:21.860906 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 03:08:21.860906 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 03:08:21.860906 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 13 03:08:21.860906 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 13 03:08:21.860906 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 13 03:08:21.860906 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 13 03:08:21.860906 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 03:08:21.860906 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 03:08:21.860906 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 03:08:21.860906 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 03:08:21.860906 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 03:08:21.860906 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application with Lock( Sep 13 03:08:21.860906 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 03:08:21.860906 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._enter() Sep 13 03:08:21.860906 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 03:08:21.860906 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 03:08:21.860906 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 03:08:21.860906 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.creator() Sep 13 03:08:21.862079 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 03:08:21.862079 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application created_value = creator( Sep 13 03:08:21.862079 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 13 03:08:21.862079 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 13 03:08:21.862079 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 13 03:08:21.862079 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 13 03:08:21.862079 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 13 03:08:21.862079 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 13 03:08:21.862079 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Sep 13 03:08:21.862079 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Sep 13 03:08:21.863865 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 349/685] 10.222.0.18 () {56 vars in 1096 bytes} [Tue Sep 13 03:08:21 2022] GET /identity/v3/users/swift => generated 83 bytes in 255 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 03:08:21.869541 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f8547f3-118d-4b5b-b1a3-e070882e0f02 None None] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:08:21.980574 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f8547f3-118d-4b5b-b1a3-e070882e0f02 None None] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:08:21.998121 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:08:22.022774 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ea36165-1350-42c0-b6b6-0ce0c0566ea8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:08:22.024146 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea36165-1350-42c0-b6b6-0ce0c0566ea8 None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:08:22.024626 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea36165-1350-42c0-b6b6-0ce0c0566ea8 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:08:22.025118 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea36165-1350-42c0-b6b6-0ce0c0566ea8 None admin] PATH_INFO: `/v3/users` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:08:22.026049 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ea36165-1350-42c0-b6b6-0ce0c0566ea8 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 03:08:22.026551 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ea36165-1350-42c0-b6b6-0ce0c0566ea8 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:08:22.028213 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ea36165-1350-42c0-b6b6-0ce0c0566ea8 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:08:22.044832 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc5e2308-a079-4584-b4f3-43c0a0459fef None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:08:22.054216 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 337/686] 10.222.0.18 () {56 vars in 1135 bytes} [Tue Sep 13 03:08:21 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 03:08:22.056155 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ea36165-1350-42c0-b6b6-0ce0c0566ea8 None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:08:22.198314 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ea36165-1350-42c0-b6b6-0ce0c0566ea8 None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:08:22.205245 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc5e2308-a079-4584-b4f3-43c0a0459fef None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:08:22.247099 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57e84352-b0fc-4a45-a6ef-d8717fb3119e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:08:22.248965 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e84352-b0fc-4a45-a6ef-d8717fb3119e None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:08:22.250066 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e84352-b0fc-4a45-a6ef-d8717fb3119e None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:08:22.250884 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e84352-b0fc-4a45-a6ef-d8717fb3119e None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:08:22.260087 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efaeb8b8-2e6c-4241-a211-d1559727e76e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:08:22.260895 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efaeb8b8-2e6c-4241-a211-d1559727e76e None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:08:22.261270 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efaeb8b8-2e6c-4241-a211-d1559727e76e None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:08:22.261634 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efaeb8b8-2e6c-4241-a211-d1559727e76e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:08:22.271005 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57e84352-b0fc-4a45-a6ef-d8717fb3119e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 03:08:22.281651 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application [None req-57e84352-b0fc-4a45-a6ef-d8717fb3119e None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 13 03:08:22.281651 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 03:08:22.281651 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 03:08:22.281651 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 03:08:22.281651 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 03:08:22.281651 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 03:08:22.281651 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 03:08:22.281651 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 03:08:22.281651 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 03:08:22.281651 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 03:08:22.281651 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 03:08:22.281651 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 03:08:22.281651 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 03:08:22.281651 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 03:08:22.281651 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 03:08:22.281651 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 03:08:22.282860 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 03:08:22.282860 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 03:08:22.282860 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 03:08:22.282860 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 03:08:22.282860 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 03:08:22.282860 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 03:08:22.282860 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 03:08:22.282860 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 03:08:22.282860 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 03:08:22.282860 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 03:08:22.282860 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 03:08:22.282860 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application with Lock( Sep 13 03:08:22.282860 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 03:08:22.282860 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._enter() Sep 13 03:08:22.282860 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 03:08:22.282860 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 03:08:22.283977 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 03:08:22.283977 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.creator() Sep 13 03:08:22.283977 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 03:08:22.283977 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application created_value = creator( Sep 13 03:08:22.283977 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 03:08:22.283977 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 03:08:22.283977 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 03:08:22.283977 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 03:08:22.283977 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 03:08:22.283977 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 03:08:22.283977 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 13 03:08:22.283977 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Sep 13 03:08:22.287819 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 350/687] 10.222.0.18 () {56 vars in 1096 bytes} [Tue Sep 13 03:08:22 2022] GET /identity/v3/roles/admin => generated 83 bytes in 243 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 03:08:22.293373 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57e84352-b0fc-4a45-a6ef-d8717fb3119e None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:08:22.342159 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:08:22.371013 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:08:22.392943 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efaeb8b8-2e6c-4241-a211-d1559727e76e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:08:22.394784 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efaeb8b8-2e6c-4241-a211-d1559727e76e None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:08:22.467037 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57e84352-b0fc-4a45-a6ef-d8717fb3119e None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:08:22.527367 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f65de5e2-6f11-4064-b474-94e734402459 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:08:22.528332 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f65de5e2-6f11-4064-b474-94e734402459 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:08:22.528816 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f65de5e2-6f11-4064-b474-94e734402459 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:08:22.529315 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f65de5e2-6f11-4064-b474-94e734402459 None admin] PATH_INFO: `/v3/roles` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:08:22.530772 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f65de5e2-6f11-4064-b474-94e734402459 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 03:08:22.531296 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f65de5e2-6f11-4064-b474-94e734402459 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:08:22.533025 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f65de5e2-6f11-4064-b474-94e734402459 None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:08:22.553022 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 351/688] 10.222.0.18 () {56 vars in 1099 bytes} [Tue Sep 13 03:08:22 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 260 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 03:08:22.557192 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f65de5e2-6f11-4064-b474-94e734402459 None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:08:22.590492 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 338/689] 10.222.0.18 () {58 vars in 1282 bytes} [Tue Sep 13 03:08:22 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 530 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 13 03:08:22.592037 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efaeb8b8-2e6c-4241-a211-d1559727e76e None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:08:22.720578 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:08:22.721983 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:08:22.729818 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f65de5e2-6f11-4064-b474-94e734402459 None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:08:22.750243 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efaeb8b8-2e6c-4241-a211-d1559727e76e None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:08:22.789076 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c7536d6-42c6-4ca3-8d91-a98723ff8da4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:08:22.790066 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c7536d6-42c6-4ca3-8d91-a98723ff8da4 None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:08:22.790575 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c7536d6-42c6-4ca3-8d91-a98723ff8da4 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:08:22.791196 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c7536d6-42c6-4ca3-8d91-a98723ff8da4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:08:22.803190 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c7536d6-42c6-4ca3-8d91-a98723ff8da4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:08:22.805162 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c7536d6-42c6-4ca3-8d91-a98723ff8da4 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:08:22.816263 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-691e9fb6-4cc3-4a8a-a502-cc28a4ff5eb3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:08:22.816913 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-691e9fb6-4cc3-4a8a-a502-cc28a4ff5eb3 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:08:22.817191 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-691e9fb6-4cc3-4a8a-a502-cc28a4ff5eb3 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:08:22.817464 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-691e9fb6-4cc3-4a8a-a502-cc28a4ff5eb3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:08:22.822526 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application [None req-8c7536d6-42c6-4ca3-8d91-a98723ff8da4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 03:08:22.822526 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 03:08:22.822526 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 03:08:22.822526 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 03:08:22.822526 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 03:08:22.822526 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 03:08:22.822526 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 03:08:22.822526 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 03:08:22.822526 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 03:08:22.822526 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 03:08:22.822526 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 03:08:22.822526 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 03:08:22.822526 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 13 03:08:22.822526 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 13 03:08:22.822526 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 13 03:08:22.822526 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 13 03:08:22.823617 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 03:08:22.823617 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 03:08:22.823617 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 03:08:22.823617 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 03:08:22.823617 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 03:08:22.823617 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 03:08:22.823617 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 03:08:22.823617 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application with Lock( Sep 13 03:08:22.823617 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 03:08:22.823617 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._enter() Sep 13 03:08:22.823617 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 03:08:22.823617 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 03:08:22.823617 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 03:08:22.823617 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.creator() Sep 13 03:08:22.823617 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 03:08:22.823617 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application created_value = creator( Sep 13 03:08:22.823617 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 13 03:08:22.824643 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 13 03:08:22.824643 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 03:08:22.824643 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Sep 13 03:08:22.826375 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 339/690] 10.222.0.18 () {56 vars in 1108 bytes} [Tue Sep 13 03:08:22 2022] GET /identity/v3/domains/Default => generated 87 bytes in 235 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 13 03:08:22.832159 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-691e9fb6-4cc3-4a8a-a502-cc28a4ff5eb3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:08:22.832159 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-691e9fb6-4cc3-4a8a-a502-cc28a4ff5eb3 None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:08:22.833306 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c7536d6-42c6-4ca3-8d91-a98723ff8da4 None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:08:22.856983 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application [None req-691e9fb6-4cc3-4a8a-a502-cc28a4ff5eb3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 03:08:22.856983 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 03:08:22.856983 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 03:08:22.856983 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 03:08:22.856983 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 03:08:22.856983 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 03:08:22.856983 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 03:08:22.856983 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 03:08:22.856983 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 03:08:22.856983 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 03:08:22.856983 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 03:08:22.856983 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 03:08:22.856983 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 13 03:08:22.856983 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 13 03:08:22.856983 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 13 03:08:22.856983 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 13 03:08:22.858213 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 03:08:22.858213 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 03:08:22.858213 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 03:08:22.858213 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 03:08:22.858213 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 03:08:22.858213 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 03:08:22.858213 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 03:08:22.858213 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application with Lock( Sep 13 03:08:22.858213 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 03:08:22.858213 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._enter() Sep 13 03:08:22.858213 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 03:08:22.858213 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 03:08:22.858213 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 03:08:22.858213 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.creator() Sep 13 03:08:22.858213 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 03:08:22.858213 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application created_value = creator( Sep 13 03:08:22.858213 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 13 03:08:22.859236 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 13 03:08:22.859236 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 03:08:22.859236 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Sep 13 03:08:22.862658 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 352/691] 10.222.0.18 () {56 vars in 1108 bytes} [Tue Sep 13 03:08:22 2022] GET /identity/v3/domains/Default => generated 87 bytes in 306 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 03:08:22.874677 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-691e9fb6-4cc3-4a8a-a502-cc28a4ff5eb3 None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:08:23.006476 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c7536d6-42c6-4ca3-8d91-a98723ff8da4 None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:08:23.059964 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:08:23.079890 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49c8ce5f-ac68-4681-a01b-d319d4ab2644 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:08:23.079890 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49c8ce5f-ac68-4681-a01b-d319d4ab2644 None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:08:23.079890 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49c8ce5f-ac68-4681-a01b-d319d4ab2644 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:08:23.079890 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49c8ce5f-ac68-4681-a01b-d319d4ab2644 None admin] PATH_INFO: `/v3/domains` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:08:23.080244 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49c8ce5f-ac68-4681-a01b-d319d4ab2644 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 03:08:23.080674 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49c8ce5f-ac68-4681-a01b-d319d4ab2644 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:08:23.082662 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49c8ce5f-ac68-4681-a01b-d319d4ab2644 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:08:23.083729 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-691e9fb6-4cc3-4a8a-a502-cc28a4ff5eb3 None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:08:23.097318 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 340/692] 10.222.0.18 () {56 vars in 1109 bytes} [Tue Sep 13 03:08:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 264 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 03:08:23.102710 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49c8ce5f-ac68-4681-a01b-d319d4ab2644 None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:08:23.148678 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae3de8dd-55ad-47a2-9e07-2781f4f28ed5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:08:23.150711 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae3de8dd-55ad-47a2-9e07-2781f4f28ed5 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:08:23.150998 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae3de8dd-55ad-47a2-9e07-2781f4f28ed5 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:08:23.151270 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae3de8dd-55ad-47a2-9e07-2781f4f28ed5 None admin] PATH_INFO: `/v3/domains` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:08:23.151913 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae3de8dd-55ad-47a2-9e07-2781f4f28ed5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 03:08:23.152214 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae3de8dd-55ad-47a2-9e07-2781f4f28ed5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:08:23.153807 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae3de8dd-55ad-47a2-9e07-2781f4f28ed5 None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:08:23.181947 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 353/693] 10.222.0.18 () {56 vars in 1109 bytes} [Tue Sep 13 03:08:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 313 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 03:08:23.187450 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae3de8dd-55ad-47a2-9e07-2781f4f28ed5 None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:08:23.207021 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:08:23.276572 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49c8ce5f-ac68-4681-a01b-d319d4ab2644 None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:08:23.317695 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce3782f3-e71c-45f0-a86f-0a8d92f1a8b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:08:23.318916 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce3782f3-e71c-45f0-a86f-0a8d92f1a8b4 None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:08:23.319202 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce3782f3-e71c-45f0-a86f-0a8d92f1a8b4 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:08:23.319476 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce3782f3-e71c-45f0-a86f-0a8d92f1a8b4 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:08:23.327469 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce3782f3-e71c-45f0-a86f-0a8d92f1a8b4 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:08:23.329209 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce3782f3-e71c-45f0-a86f-0a8d92f1a8b4 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:08:23.344368 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application [None req-ce3782f3-e71c-45f0-a86f-0a8d92f1a8b4 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 13 03:08:23.344368 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 03:08:23.344368 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 03:08:23.344368 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 03:08:23.344368 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 03:08:23.344368 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 03:08:23.344368 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 03:08:23.344368 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 03:08:23.344368 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 03:08:23.344368 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 03:08:23.344368 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 03:08:23.344368 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 03:08:23.344368 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 13 03:08:23.344368 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 13 03:08:23.344368 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 13 03:08:23.345343 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 13 03:08:23.345343 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 03:08:23.345343 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 03:08:23.345343 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 03:08:23.345343 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 03:08:23.345343 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 03:08:23.345343 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 03:08:23.345343 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 03:08:23.345343 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application with Lock( Sep 13 03:08:23.345343 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 03:08:23.345343 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._enter() Sep 13 03:08:23.345343 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 03:08:23.345343 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 03:08:23.345343 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 03:08:23.345343 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.creator() Sep 13 03:08:23.345343 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 03:08:23.345343 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application created_value = creator( Sep 13 03:08:23.346307 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 13 03:08:23.346307 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 13 03:08:23.346307 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 13 03:08:23.346307 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 13 03:08:23.346307 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 13 03:08:23.346307 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 13 03:08:23.346307 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 13 03:08:23.346307 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Sep 13 03:08:23.351416 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 341/694] 10.222.0.18 () {56 vars in 1111 bytes} [Tue Sep 13 03:08:23 2022] GET /identity/v3/projects/service => generated 88 bytes in 249 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 03:08:23.355946 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce3782f3-e71c-45f0-a86f-0a8d92f1a8b4 None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:08:23.401433 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae3de8dd-55ad-47a2-9e07-2781f4f28ed5 None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:08:23.439311 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:08:23.462280 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-853b39f8-e588-4604-87e8-65c15fda67e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:08:23.463377 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-853b39f8-e588-4604-87e8-65c15fda67e7 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:08:23.464380 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-853b39f8-e588-4604-87e8-65c15fda67e7 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:08:23.464865 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-853b39f8-e588-4604-87e8-65c15fda67e7 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:08:23.474024 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-853b39f8-e588-4604-87e8-65c15fda67e7 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:08:23.476130 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-853b39f8-e588-4604-87e8-65c15fda67e7 None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:08:23.492796 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application [None req-853b39f8-e588-4604-87e8-65c15fda67e7 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 13 03:08:23.492796 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 03:08:23.492796 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 03:08:23.492796 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 03:08:23.492796 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 03:08:23.492796 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 03:08:23.492796 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 03:08:23.492796 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 03:08:23.492796 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 03:08:23.492796 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 03:08:23.492796 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 03:08:23.492796 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 03:08:23.492796 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 13 03:08:23.492796 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 13 03:08:23.492796 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 13 03:08:23.492796 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 13 03:08:23.493785 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 03:08:23.493785 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 03:08:23.493785 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 13 03:08:23.493785 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 13 03:08:23.493785 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 13 03:08:23.493785 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 13 03:08:23.493785 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 03:08:23.493785 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 03:08:23.493785 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 03:08:23.493785 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 03:08:23.493785 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 03:08:23.493785 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application with Lock( Sep 13 03:08:23.493785 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 03:08:23.493785 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._enter() Sep 13 03:08:23.493785 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 03:08:23.493785 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 03:08:23.493785 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 03:08:23.493785 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.creator() Sep 13 03:08:23.494769 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 03:08:23.494769 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application created_value = creator( Sep 13 03:08:23.494769 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 13 03:08:23.494769 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 13 03:08:23.494769 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 13 03:08:23.494769 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 13 03:08:23.494769 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 13 03:08:23.494769 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 13 03:08:23.494769 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 13 03:08:23.494769 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Sep 13 03:08:23.501299 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 354/695] 10.222.0.18 () {56 vars in 1093 bytes} [Tue Sep 13 03:08:23 2022] GET /identity/v3/users/nova => generated 82 bytes in 309 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 03:08:23.501299 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57baf824-1bb3-4d94-820d-93083b504c42 None None] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:08:23.501299 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57baf824-1bb3-4d94-820d-93083b504c42 None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:08:23.501299 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57baf824-1bb3-4d94-820d-93083b504c42 None None] PATH_INFO: `/` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:08:23.501299 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 355/696] 10.222.0.18 () {54 vars in 927 bytes} [Tue Sep 13 03:08:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 03:08:23.503020 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57baf824-1bb3-4d94-820d-93083b504c42 None None] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:08:23.509552 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce3782f3-e71c-45f0-a86f-0a8d92f1a8b4 None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:08:23.549124 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd7c734a-bb0c-4c03-b309-9b80189965bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:08:23.549767 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd7c734a-bb0c-4c03-b309-9b80189965bd None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:08:23.550051 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd7c734a-bb0c-4c03-b309-9b80189965bd None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:08:23.550322 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd7c734a-bb0c-4c03-b309-9b80189965bd None admin] PATH_INFO: `/v3/projects` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:08:23.551163 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd7c734a-bb0c-4c03-b309-9b80189965bd None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 03:08:23.551471 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd7c734a-bb0c-4c03-b309-9b80189965bd None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:08:23.553108 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd7c734a-bb0c-4c03-b309-9b80189965bd None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:08:23.571183 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 342/697] 10.222.0.18 () {56 vars in 1148 bytes} [Tue Sep 13 03:08:23 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 03:08:23.571183 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b60f0c0c-b4ae-440d-9f74-94c6dc5a6d66 None None] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:08:23.571183 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b60f0c0c-b4ae-440d-9f74-94c6dc5a6d66 None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:08:23.571183 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b60f0c0c-b4ae-440d-9f74-94c6dc5a6d66 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:08:23.589182 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.auth.core [None req-b60f0c0c-b4ae-440d-9f74-94c6dc5a6d66 None None] MFA Rules not processed for user `49f280ab086b4610855c4ffa378517d9`. Rule list: `[]` (Enabled: `True`). {{(pid=81399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 03:08:23.590284 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-b60f0c0c-b4ae-440d-9f74-94c6dc5a6d66 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:08:23.605167 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:08:23.642944 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57baf824-1bb3-4d94-820d-93083b504c42 None None] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:08:23.674255 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 343/698] 10.222.0.18 () {58 vars in 1024 bytes} [Tue Sep 13 03:08:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 13 03:08:23.674255 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b60f0c0c-b4ae-440d-9f74-94c6dc5a6d66 None None] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:08:23.699961 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95ae764b-dd74-4e15-a01b-0b7d1695f62c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:08:23.700670 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ae764b-dd74-4e15-a01b-0b7d1695f62c None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:08:23.700947 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ae764b-dd74-4e15-a01b-0b7d1695f62c None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:08:23.701218 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ae764b-dd74-4e15-a01b-0b7d1695f62c None admin] PATH_INFO: `/v3/users` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:08:23.702657 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95ae764b-dd74-4e15-a01b-0b7d1695f62c None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 03:08:23.702971 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95ae764b-dd74-4e15-a01b-0b7d1695f62c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:08:23.704626 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95ae764b-dd74-4e15-a01b-0b7d1695f62c None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:08:23.740023 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:08:23.740670 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 356/699] 10.222.0.18 () {56 vars in 1133 bytes} [Tue Sep 13 03:08:23 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 239 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 03:08:23.748160 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95ae764b-dd74-4e15-a01b-0b7d1695f62c None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:08:23.889010 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b60f0c0c-b4ae-440d-9f74-94c6dc5a6d66 None None] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:08:23.890157 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95ae764b-dd74-4e15-a01b-0b7d1695f62c None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:08:23.926398 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:08:23.929192 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-546bbe4e-7b1f-4ee5-a633-7a23284ed074 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:08:23.930588 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-546bbe4e-7b1f-4ee5-a633-7a23284ed074 None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:08:23.930931 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-546bbe4e-7b1f-4ee5-a633-7a23284ed074 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:08:23.931225 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-546bbe4e-7b1f-4ee5-a633-7a23284ed074 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:08:23.932208 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-546bbe4e-7b1f-4ee5-a633-7a23284ed074 None admin] RBAC: Adding query filter params (role.id=817d6837cd044890b41cf16b303a092b, scope.project.id=b9d94f6a68f7470d9494f112175620dc, user.id=893b0c0d85964b7dbc9705c0a091682c) {{(pid=81399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 03:08:23.932520 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-546bbe4e-7b1f-4ee5-a633-7a23284ed074 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:08:23.934431 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-546bbe4e-7b1f-4ee5-a633-7a23284ed074 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:08:23.937709 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a00311ef-8c1d-438a-83aa-238ce9e7579d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:08:23.939059 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a00311ef-8c1d-438a-83aa-238ce9e7579d None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:08:23.939785 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a00311ef-8c1d-438a-83aa-238ce9e7579d None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:08:23.940256 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a00311ef-8c1d-438a-83aa-238ce9e7579d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:08:23.941604 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 344/700] 10.222.0.18 () {56 vars in 1374 bytes} [Tue Sep 13 03:08:23 2022] GET /identity/v3/role_assignments?user.id=893b0c0d85964b7dbc9705c0a091682c&scope.project.id=b9d94f6a68f7470d9494f112175620dc&role.id=817d6837cd044890b41cf16b303a092b => generated 608 bytes in 270 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 03:08:23.950661 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf632450-d31b-46cc-8813-1a0f0ec099d4 None None] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:08:23.950661 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf632450-d31b-46cc-8813-1a0f0ec099d4 None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:08:23.950661 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf632450-d31b-46cc-8813-1a0f0ec099d4 None None] PATH_INFO: `/` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:08:23.950661 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 345/701] 10.222.0.18 () {54 vars in 927 bytes} [Tue Sep 13 03:08:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 03:08:23.950661 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b8bda0f-cadd-4b78-a015-c1b70d53d244 None None] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:08:23.951111 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b8bda0f-cadd-4b78-a015-c1b70d53d244 None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:08:23.951111 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b8bda0f-cadd-4b78-a015-c1b70d53d244 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:08:23.978957 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.auth.core [None req-0b8bda0f-cadd-4b78-a015-c1b70d53d244 None None] MFA Rules not processed for user `49f280ab086b4610855c4ffa378517d9`. Rule list: `[]` (Enabled: `True`). {{(pid=81399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 03:08:23.980549 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-0b8bda0f-cadd-4b78-a015-c1b70d53d244 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:08:24.065306 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 346/702] 10.222.0.18 () {58 vars in 1024 bytes} [Tue Sep 13 03:08:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 13 03:08:24.066288 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b8bda0f-cadd-4b78-a015-c1b70d53d244 None None] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:08:24.067291 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a00311ef-8c1d-438a-83aa-238ce9e7579d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:08:24.069389 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a00311ef-8c1d-438a-83aa-238ce9e7579d None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:08:24.097557 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:08:24.179152 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b8bda0f-cadd-4b78-a015-c1b70d53d244 None None] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:08:24.187674 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 357/703] 10.222.0.18 () {58 vars in 1282 bytes} [Tue Sep 13 03:08:23 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 440 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 13 03:08:24.193648 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a00311ef-8c1d-438a-83aa-238ce9e7579d None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:08:24.216624 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b0911cc-89d0-4c81-b95c-98cfaf96c362 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:08:24.217267 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b0911cc-89d0-4c81-b95c-98cfaf96c362 None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:08:24.218080 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b0911cc-89d0-4c81-b95c-98cfaf96c362 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:08:24.218355 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b0911cc-89d0-4c81-b95c-98cfaf96c362 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:08:24.224661 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:08:24.234507 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b0911cc-89d0-4c81-b95c-98cfaf96c362 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:08:24.237154 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b0911cc-89d0-4c81-b95c-98cfaf96c362 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:08:24.258351 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application [None req-4b0911cc-89d0-4c81-b95c-98cfaf96c362 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 03:08:24.258351 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 03:08:24.258351 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 03:08:24.258351 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 03:08:24.258351 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 03:08:24.258351 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 03:08:24.258351 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 03:08:24.258351 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 03:08:24.258351 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 03:08:24.258351 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 03:08:24.258351 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 03:08:24.258351 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 03:08:24.258351 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 13 03:08:24.258351 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 13 03:08:24.258351 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 13 03:08:24.258351 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 13 03:08:24.259523 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 03:08:24.259523 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 03:08:24.259523 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 03:08:24.259523 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 03:08:24.259523 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 03:08:24.259523 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 03:08:24.259523 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 03:08:24.259523 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application with Lock( Sep 13 03:08:24.259523 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 03:08:24.259523 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._enter() Sep 13 03:08:24.259523 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 03:08:24.259523 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 03:08:24.259523 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 03:08:24.259523 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.creator() Sep 13 03:08:24.259523 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 03:08:24.259523 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application created_value = creator( Sep 13 03:08:24.259523 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 13 03:08:24.260438 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 13 03:08:24.260438 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 03:08:24.260438 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Sep 13 03:08:24.265436 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 347/704] 10.222.0.18 () {56 vars in 1108 bytes} [Tue Sep 13 03:08:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 03:08:24.267111 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b0911cc-89d0-4c81-b95c-98cfaf96c362 None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:08:24.319079 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a00311ef-8c1d-438a-83aa-238ce9e7579d None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:08:24.356295 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0d943cf-bda5-41af-9386-46a48a8f28eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:08:24.357372 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d943cf-bda5-41af-9386-46a48a8f28eb None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:08:24.357849 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d943cf-bda5-41af-9386-46a48a8f28eb None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:08:24.358911 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d943cf-bda5-41af-9386-46a48a8f28eb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:08:24.364850 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:08:24.370924 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0d943cf-bda5-41af-9386-46a48a8f28eb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:08:24.372403 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0d943cf-bda5-41af-9386-46a48a8f28eb None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:08:24.387375 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b0911cc-89d0-4c81-b95c-98cfaf96c362 None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:08:24.390409 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application [None req-d0d943cf-bda5-41af-9386-46a48a8f28eb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 03:08:24.390409 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 03:08:24.390409 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 03:08:24.390409 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 03:08:24.390409 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 03:08:24.390409 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 03:08:24.390409 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 03:08:24.390409 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 03:08:24.390409 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 03:08:24.390409 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 03:08:24.390409 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 03:08:24.390409 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 03:08:24.390409 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 13 03:08:24.390409 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 13 03:08:24.390409 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 13 03:08:24.390409 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 13 03:08:24.391403 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 03:08:24.391403 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 03:08:24.391403 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 03:08:24.391403 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 03:08:24.391403 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 03:08:24.391403 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 03:08:24.391403 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 03:08:24.391403 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application with Lock( Sep 13 03:08:24.391403 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 03:08:24.391403 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._enter() Sep 13 03:08:24.391403 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 03:08:24.391403 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 03:08:24.391403 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 03:08:24.391403 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.creator() Sep 13 03:08:24.391403 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 03:08:24.391403 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application created_value = creator( Sep 13 03:08:24.391403 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 13 03:08:24.392367 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 13 03:08:24.392367 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 03:08:24.392367 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Sep 13 03:08:24.392367 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 358/705] 10.222.0.18 () {56 vars in 1108 bytes} [Tue Sep 13 03:08:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 13 03:08:24.395723 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25199dc1-9b06-45d3-a05f-fdcaa31a1efb None None] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:08:24.396216 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25199dc1-9b06-45d3-a05f-fdcaa31a1efb None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:08:24.396688 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25199dc1-9b06-45d3-a05f-fdcaa31a1efb None None] PATH_INFO: `/` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:08:24.399281 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 359/706] 10.222.0.18 () {54 vars in 927 bytes} [Tue Sep 13 03:08:24 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 03:08:24.400671 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25199dc1-9b06-45d3-a05f-fdcaa31a1efb None None] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:08:24.437265 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6341109-612b-4a30-b8f1-5903f1de81bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:08:24.437788 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6341109-612b-4a30-b8f1-5903f1de81bf None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:08:24.438058 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6341109-612b-4a30-b8f1-5903f1de81bf None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:08:24.438327 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6341109-612b-4a30-b8f1-5903f1de81bf None admin] PATH_INFO: `/v3/domains` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:08:24.439019 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6341109-612b-4a30-b8f1-5903f1de81bf None admin] RBAC: Adding query filter params (name=Default) {{(pid=81399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 03:08:24.439319 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6341109-612b-4a30-b8f1-5903f1de81bf None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:08:24.440700 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6341109-612b-4a30-b8f1-5903f1de81bf None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:08:24.453269 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 348/707] 10.222.0.18 () {56 vars in 1109 bytes} [Tue Sep 13 03:08:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 03:08:24.454274 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f96f1752-258c-4194-9936-4ed504fef222 None None] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:08:24.454552 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f96f1752-258c-4194-9936-4ed504fef222 None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:08:24.454849 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f96f1752-258c-4194-9936-4ed504fef222 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:08:24.472596 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.auth.core [None req-f96f1752-258c-4194-9936-4ed504fef222 None None] MFA Rules not processed for user `49f280ab086b4610855c4ffa378517d9`. Rule list: `[]` (Enabled: `True`). {{(pid=81399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 03:08:24.473755 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-f96f1752-258c-4194-9936-4ed504fef222 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:08:24.497852 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:08:24.519694 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25199dc1-9b06-45d3-a05f-fdcaa31a1efb None None] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:08:24.548267 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 349/708] 10.222.0.18 () {58 vars in 1024 bytes} [Tue Sep 13 03:08:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 13 03:08:24.549405 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f96f1752-258c-4194-9936-4ed504fef222 None None] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:08:24.560739 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-344ef04b-32d2-46b3-80b6-b5e9ddecaca3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:08:24.561504 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-344ef04b-32d2-46b3-80b6-b5e9ddecaca3 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:08:24.561786 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-344ef04b-32d2-46b3-80b6-b5e9ddecaca3 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:08:24.562057 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-344ef04b-32d2-46b3-80b6-b5e9ddecaca3 None admin] PATH_INFO: `/v3/domains` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:08:24.562731 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-344ef04b-32d2-46b3-80b6-b5e9ddecaca3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 03:08:24.563030 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-344ef04b-32d2-46b3-80b6-b5e9ddecaca3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:08:24.564499 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-344ef04b-32d2-46b3-80b6-b5e9ddecaca3 None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:08:24.578275 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 360/709] 10.222.0.18 () {56 vars in 1109 bytes} [Tue Sep 13 03:08:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 03:08:24.579257 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-344ef04b-32d2-46b3-80b6-b5e9ddecaca3 None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:08:24.632922 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:08:24.664916 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f96f1752-258c-4194-9936-4ed504fef222 None None] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:08:24.701099 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b318a4e-3ea6-4923-b9a5-9b595c0b02d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:08:24.701648 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b318a4e-3ea6-4923-b9a5-9b595c0b02d7 None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:08:24.702444 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b318a4e-3ea6-4923-b9a5-9b595c0b02d7 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:08:24.702783 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b318a4e-3ea6-4923-b9a5-9b595c0b02d7 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:08:24.703350 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b318a4e-3ea6-4923-b9a5-9b595c0b02d7 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:08:24.704821 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b318a4e-3ea6-4923-b9a5-9b595c0b02d7 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:08:24.714405 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-344ef04b-32d2-46b3-80b6-b5e9ddecaca3 None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:08:24.715934 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application [None req-7b318a4e-3ea6-4923-b9a5-9b595c0b02d7 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 13 03:08:24.715934 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 03:08:24.715934 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 13 03:08:24.715934 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 13 03:08:24.715934 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 13 03:08:24.715934 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 13 03:08:24.715934 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 13 03:08:24.715934 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 13 03:08:24.715934 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 13 03:08:24.715934 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Sep 13 03:08:24.715934 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 13 03:08:24.715934 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Sep 13 03:08:24.715934 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 03:08:24.715934 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 03:08:24.715934 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 03:08:24.715934 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 03:08:24.715934 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 03:08:24.717006 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 03:08:24.717006 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 03:08:24.717006 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 03:08:24.717006 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 03:08:24.717006 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 03:08:24.717006 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 03:08:24.717006 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 13 03:08:24.717006 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 13 03:08:24.717006 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 13 03:08:24.717006 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 13 03:08:24.717006 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 03:08:24.717006 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 03:08:24.717006 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 03:08:24.717006 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 03:08:24.717006 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 03:08:24.717006 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 03:08:24.719399 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 03:08:24.719399 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application with Lock( Sep 13 03:08:24.719399 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 03:08:24.719399 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._enter() Sep 13 03:08:24.719399 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 03:08:24.719399 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 03:08:24.719399 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 03:08:24.719399 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.creator() Sep 13 03:08:24.719399 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 03:08:24.719399 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application created_value = creator( Sep 13 03:08:24.719399 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 13 03:08:24.719399 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 13 03:08:24.719399 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 13 03:08:24.719399 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Sep 13 03:08:24.719399 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 350/710] 10.222.0.18 () {56 vars in 1129 bytes} [Tue Sep 13 03:08:24 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 03:08:24.720258 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b318a4e-3ea6-4923-b9a5-9b595c0b02d7 None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:08:24.767039 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-398dec6e-c3a7-45a0-b6a2-b80d939baa4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:08:24.768306 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-398dec6e-c3a7-45a0-b6a2-b80d939baa4d None admin] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:08:24.769293 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-398dec6e-c3a7-45a0-b6a2-b80d939baa4d None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:08:24.770042 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-398dec6e-c3a7-45a0-b6a2-b80d939baa4d None admin] PATH_INFO: `/v3/users` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:08:24.770957 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-398dec6e-c3a7-45a0-b6a2-b80d939baa4d None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:08:24.773167 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-398dec6e-c3a7-45a0-b6a2-b80d939baa4d None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:08:24.789647 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:08:24.805661 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 361/711] 10.222.0.18 () {60 vars in 1132 bytes} [Tue Sep 13 03:08:24 2022] POST /identity/v3/users => generated 254 bytes in 227 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 03:08:24.806947 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-398dec6e-c3a7-45a0-b6a2-b80d939baa4d None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:08:24.857201 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b318a4e-3ea6-4923-b9a5-9b595c0b02d7 None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:08:24.902010 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bb1a7c0-6f5a-4e6f-9a82-7868f3c74b0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:08:24.903105 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb1a7c0-6f5a-4e6f-9a82-7868f3c74b0d None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:08:24.903618 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb1a7c0-6f5a-4e6f-9a82-7868f3c74b0d None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:08:24.904262 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb1a7c0-6f5a-4e6f-9a82-7868f3c74b0d None admin] PATH_INFO: `/v3/projects/service` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:08:24.909305 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:08:24.915365 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bb1a7c0-6f5a-4e6f-9a82-7868f3c74b0d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:08:24.917528 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bb1a7c0-6f5a-4e6f-9a82-7868f3c74b0d None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:08:24.933811 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application [None req-0bb1a7c0-6f5a-4e6f-9a82-7868f3c74b0d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 13 03:08:24.933811 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 03:08:24.933811 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 03:08:24.933811 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 03:08:24.933811 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 03:08:24.933811 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 03:08:24.933811 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 03:08:24.933811 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 03:08:24.933811 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 03:08:24.933811 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 03:08:24.933811 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 03:08:24.933811 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 03:08:24.933811 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 13 03:08:24.933811 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 13 03:08:24.933811 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 13 03:08:24.934811 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 13 03:08:24.934811 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 03:08:24.934811 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 03:08:24.934811 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 03:08:24.934811 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 03:08:24.934811 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 03:08:24.934811 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 03:08:24.934811 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 03:08:24.934811 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application with Lock( Sep 13 03:08:24.934811 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 03:08:24.934811 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._enter() Sep 13 03:08:24.934811 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 03:08:24.934811 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 03:08:24.934811 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 03:08:24.934811 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.creator() Sep 13 03:08:24.934811 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 03:08:24.934811 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application created_value = creator( Sep 13 03:08:24.935732 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 13 03:08:24.935732 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 13 03:08:24.935732 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 13 03:08:24.935732 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 13 03:08:24.935732 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 13 03:08:24.935732 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 13 03:08:24.935732 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 13 03:08:24.935732 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Sep 13 03:08:24.939605 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 351/712] 10.222.0.18 () {56 vars in 1111 bytes} [Tue Sep 13 03:08:24 2022] GET /identity/v3/projects/service => generated 88 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 03:08:24.940717 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bb1a7c0-6f5a-4e6f-9a82-7868f3c74b0d None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:08:24.962184 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-398dec6e-c3a7-45a0-b6a2-b80d939baa4d None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:08:25.005676 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ad4d77f-c078-43ea-9b53-db1495fc661e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:08:25.006256 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad4d77f-c078-43ea-9b53-db1495fc661e None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:08:25.006534 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad4d77f-c078-43ea-9b53-db1495fc661e None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:08:25.006849 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad4d77f-c078-43ea-9b53-db1495fc661e None admin] PATH_INFO: `/v3/services` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:08:25.007479 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ad4d77f-c078-43ea-9b53-db1495fc661e None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=81400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 03:08:25.007776 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ad4d77f-c078-43ea-9b53-db1495fc661e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:08:25.009135 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ad4d77f-c078-43ea-9b53-db1495fc661e None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:08:25.016472 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 362/713] 10.222.0.18 () {56 vars in 1124 bytes} [Tue Sep 13 03:08:24 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 03:08:25.016472 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ad4d77f-c078-43ea-9b53-db1495fc661e None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:08:25.074089 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bb1a7c0-6f5a-4e6f-9a82-7868f3c74b0d None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:08:25.097697 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:08:25.117053 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cab90ff-28fe-4396-8695-26d90f4f3270 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:08:25.117693 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cab90ff-28fe-4396-8695-26d90f4f3270 None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:08:25.117975 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cab90ff-28fe-4396-8695-26d90f4f3270 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:08:25.118250 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cab90ff-28fe-4396-8695-26d90f4f3270 None admin] PATH_INFO: `/v3/services/network` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:08:25.120892 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cab90ff-28fe-4396-8695-26d90f4f3270 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:08:25.120892 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cab90ff-28fe-4396-8695-26d90f4f3270 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:08:25.132289 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application [None req-2cab90ff-28fe-4396-8695-26d90f4f3270 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Sep 13 03:08:25.132289 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 03:08:25.132289 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 13 03:08:25.132289 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 13 03:08:25.132289 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 13 03:08:25.132289 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 13 03:08:25.132289 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 13 03:08:25.132289 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 13 03:08:25.132289 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Sep 13 03:08:25.132289 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Sep 13 03:08:25.132289 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 13 03:08:25.132289 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Sep 13 03:08:25.132289 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 03:08:25.132289 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 03:08:25.132289 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 03:08:25.132289 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 03:08:25.132289 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 03:08:25.133519 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 03:08:25.133519 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 03:08:25.133519 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 03:08:25.133519 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 03:08:25.133519 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 03:08:25.133519 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 03:08:25.133519 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 13 03:08:25.133519 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 13 03:08:25.133519 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 13 03:08:25.133519 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 13 03:08:25.133519 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 03:08:25.133519 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 03:08:25.133519 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 03:08:25.133519 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 03:08:25.133519 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 03:08:25.133519 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 03:08:25.134368 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 03:08:25.134368 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application with Lock( Sep 13 03:08:25.134368 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 03:08:25.134368 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._enter() Sep 13 03:08:25.134368 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 03:08:25.134368 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 03:08:25.134368 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 03:08:25.134368 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.creator() Sep 13 03:08:25.134368 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 03:08:25.134368 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application created_value = creator( Sep 13 03:08:25.134368 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 13 03:08:25.134368 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 13 03:08:25.134368 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Sep 13 03:08:25.134368 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Sep 13 03:08:25.134368 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 352/714] 10.222.0.18 () {56 vars in 1111 bytes} [Tue Sep 13 03:08:24 2022] GET /identity/v3/services/network => generated 88 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 03:08:25.135203 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cab90ff-28fe-4396-8695-26d90f4f3270 None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:08:25.153979 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ad4d77f-c078-43ea-9b53-db1495fc661e None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:08:25.197836 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f58452a5-bda8-4a76-9782-f037e938bb69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:08:25.198862 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f58452a5-bda8-4a76-9782-f037e938bb69 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:08:25.199283 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f58452a5-bda8-4a76-9782-f037e938bb69 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:08:25.199745 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f58452a5-bda8-4a76-9782-f037e938bb69 None admin] PATH_INFO: `/v3/projects` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:08:25.200568 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f58452a5-bda8-4a76-9782-f037e938bb69 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 03:08:25.201016 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f58452a5-bda8-4a76-9782-f037e938bb69 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:08:25.203288 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f58452a5-bda8-4a76-9782-f037e938bb69 None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:08:25.209746 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:08:25.223619 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 363/715] 10.222.0.18 () {56 vars in 1148 bytes} [Tue Sep 13 03:08:25 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 03:08:25.224697 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f58452a5-bda8-4a76-9782-f037e938bb69 None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:08:25.255232 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cab90ff-28fe-4396-8695-26d90f4f3270 None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:08:25.291741 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f3bd46a-8d26-46b5-9bed-a494e21de261 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:08:25.292401 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f3bd46a-8d26-46b5-9bed-a494e21de261 None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:08:25.292633 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f3bd46a-8d26-46b5-9bed-a494e21de261 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:08:25.292901 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f3bd46a-8d26-46b5-9bed-a494e21de261 None admin] PATH_INFO: `/v3/services` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:08:25.293529 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f3bd46a-8d26-46b5-9bed-a494e21de261 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=81399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 03:08:25.293834 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f3bd46a-8d26-46b5-9bed-a494e21de261 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:08:25.295287 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f3bd46a-8d26-46b5-9bed-a494e21de261 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:08:25.307178 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 353/716] 10.222.0.18 () {56 vars in 1124 bytes} [Tue Sep 13 03:08:25 2022] GET /identity/v3/services?type=block-storage => generated 130 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 03:08:25.308278 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f3bd46a-8d26-46b5-9bed-a494e21de261 None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:08:25.347666 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f58452a5-bda8-4a76-9782-f037e938bb69 None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:08:25.398160 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:08:25.433225 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c030cb1c-6312-4702-bce8-39c1be09d4ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:08:25.434022 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c030cb1c-6312-4702-bce8-39c1be09d4ee None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:08:25.434402 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c030cb1c-6312-4702-bce8-39c1be09d4ee None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:08:25.434816 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c030cb1c-6312-4702-bce8-39c1be09d4ee None admin] PATH_INFO: `/v3/services` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:08:25.435558 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c030cb1c-6312-4702-bce8-39c1be09d4ee None admin] RBAC: Adding query filter params (name=network) {{(pid=81400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 03:08:25.435961 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c030cb1c-6312-4702-bce8-39c1be09d4ee None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:08:25.437964 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c030cb1c-6312-4702-bce8-39c1be09d4ee None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:08:25.459122 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 364/717] 10.222.0.18 () {56 vars in 1112 bytes} [Tue Sep 13 03:08:25 2022] GET /identity/v3/services?name=network => generated 124 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 03:08:25.460395 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f3bd46a-8d26-46b5-9bed-a494e21de261 None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:08:25.464907 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c030cb1c-6312-4702-bce8-39c1be09d4ee None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:08:25.489906 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:08:25.501795 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a2a35f8-8a36-4466-ac44-00baf853173f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:08:25.502432 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a2a35f8-8a36-4466-ac44-00baf853173f None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:08:25.502742 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a2a35f8-8a36-4466-ac44-00baf853173f None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:08:25.503021 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a2a35f8-8a36-4466-ac44-00baf853173f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:08:25.503755 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a2a35f8-8a36-4466-ac44-00baf853173f None admin] RBAC: Adding query filter params (role.id=425d4b481c7846e0bc07e7e805d5b15f, scope.project.id=b9d94f6a68f7470d9494f112175620dc, user.id=c7377f5bee4a49959182e1bd34bc97a4) {{(pid=81399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 03:08:25.504065 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a2a35f8-8a36-4466-ac44-00baf853173f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:08:25.505520 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a2a35f8-8a36-4466-ac44-00baf853173f None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:08:25.521405 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 354/718] 10.222.0.18 () {56 vars in 1374 bytes} [Tue Sep 13 03:08:25 2022] GET /identity/v3/role_assignments?user.id=c7377f5bee4a49959182e1bd34bc97a4&scope.project.id=b9d94f6a68f7470d9494f112175620dc&role.id=425d4b481c7846e0bc07e7e805d5b15f => generated 259 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 03:08:25.610440 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c030cb1c-6312-4702-bce8-39c1be09d4ee None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:08:25.649250 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7552885d-4e7b-4b5b-a3a0-2862406c4b6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:08:25.649876 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7552885d-4e7b-4b5b-a3a0-2862406c4b6d None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:08:25.650154 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7552885d-4e7b-4b5b-a3a0-2862406c4b6d None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:08:25.650432 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7552885d-4e7b-4b5b-a3a0-2862406c4b6d None admin] PATH_INFO: `/v3/services` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:08:25.651202 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7552885d-4e7b-4b5b-a3a0-2862406c4b6d None admin] RBAC: Adding query filter params (type=network) {{(pid=81400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 03:08:25.651509 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7552885d-4e7b-4b5b-a3a0-2862406c4b6d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:08:25.652977 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7552885d-4e7b-4b5b-a3a0-2862406c4b6d None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:08:25.660536 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 365/719] 10.222.0.18 () {56 vars in 1112 bytes} [Tue Sep 13 03:08:25 2022] GET /identity/v3/services?type=network => generated 124 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 03:08:26.487937 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-352841bb-2a27-4e58-a66f-ce57b6c391da None None] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:08:26.489042 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-352841bb-2a27-4e58-a66f-ce57b6c391da None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:08:26.489556 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-352841bb-2a27-4e58-a66f-ce57b6c391da None None] PATH_INFO: `/` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:08:26.491453 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 355/720] 10.222.0.18 () {54 vars in 927 bytes} [Tue Sep 13 03:08:26 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 03:08:26.496495 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f0871c-2872-4a2e-aa84-014438645786 None None] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:08:26.497328 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f0871c-2872-4a2e-aa84-014438645786 None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:08:26.497877 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f0871c-2872-4a2e-aa84-014438645786 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:08:26.522862 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.auth.core [None req-d9f0871c-2872-4a2e-aa84-014438645786 None None] MFA Rules not processed for user `49f280ab086b4610855c4ffa378517d9`. Rule list: `[]` (Enabled: `True`). {{(pid=81400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 03:08:26.524050 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-d9f0871c-2872-4a2e-aa84-014438645786 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:08:26.580313 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:08:26.612678 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 366/721] 10.222.0.18 () {58 vars in 1024 bytes} [Tue Sep 13 03:08:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 13 03:08:26.916758 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-352841bb-2a27-4e58-a66f-ce57b6c391da None None] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:08:27.031512 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-352841bb-2a27-4e58-a66f-ce57b6c391da None None] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:08:27.079497 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1561a57-7db8-4fe8-8dff-b1916eaf91f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:08:27.080476 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1561a57-7db8-4fe8-8dff-b1916eaf91f0 None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:08:27.080967 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1561a57-7db8-4fe8-8dff-b1916eaf91f0 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:08:27.081474 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1561a57-7db8-4fe8-8dff-b1916eaf91f0 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:08:27.092679 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1561a57-7db8-4fe8-8dff-b1916eaf91f0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 03:08:27.099181 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application [None req-c1561a57-7db8-4fe8-8dff-b1916eaf91f0 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 13 03:08:27.099181 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 03:08:27.099181 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 03:08:27.099181 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 03:08:27.099181 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 03:08:27.099181 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 03:08:27.099181 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 03:08:27.099181 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 03:08:27.099181 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 03:08:27.099181 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 03:08:27.099181 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 03:08:27.099181 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 03:08:27.099181 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 03:08:27.099181 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 03:08:27.099181 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 03:08:27.099181 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 03:08:27.100160 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 03:08:27.100160 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 03:08:27.100160 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 03:08:27.100160 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 03:08:27.100160 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 03:08:27.100160 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 03:08:27.100160 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 03:08:27.100160 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 03:08:27.100160 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 03:08:27.100160 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 03:08:27.100160 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 03:08:27.100160 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application with Lock( Sep 13 03:08:27.100160 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 03:08:27.100160 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._enter() Sep 13 03:08:27.100160 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 03:08:27.100160 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 03:08:27.101044 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 03:08:27.101044 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.creator() Sep 13 03:08:27.101044 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 03:08:27.101044 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application created_value = creator( Sep 13 03:08:27.101044 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 03:08:27.101044 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 03:08:27.101044 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 03:08:27.101044 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 03:08:27.101044 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 03:08:27.101044 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 03:08:27.101044 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 13 03:08:27.101044 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Sep 13 03:08:27.104771 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 356/722] 10.222.0.18 () {56 vars in 1096 bytes} [Tue Sep 13 03:08:26 2022] GET /identity/v3/roles/admin => generated 83 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 03:08:27.111496 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9f0871c-2872-4a2e-aa84-014438645786 None None] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:08:27.244448 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9f0871c-2872-4a2e-aa84-014438645786 None None] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:08:27.287913 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f6d53d4-3b4c-4446-b5a0-2707a60f8841 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:08:27.288874 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f6d53d4-3b4c-4446-b5a0-2707a60f8841 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:08:27.289383 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f6d53d4-3b4c-4446-b5a0-2707a60f8841 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:08:27.289883 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f6d53d4-3b4c-4446-b5a0-2707a60f8841 None admin] PATH_INFO: `/v3/roles` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:08:27.290783 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f6d53d4-3b4c-4446-b5a0-2707a60f8841 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 03:08:27.291331 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f6d53d4-3b4c-4446-b5a0-2707a60f8841 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:08:27.292905 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f6d53d4-3b4c-4446-b5a0-2707a60f8841 None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:08:27.302794 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 367/723] 10.222.0.18 () {56 vars in 1099 bytes} [Tue Sep 13 03:08:27 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 03:08:27.312120 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1561a57-7db8-4fe8-8dff-b1916eaf91f0 None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:08:27.375780 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:08:27.431655 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1561a57-7db8-4fe8-8dff-b1916eaf91f0 None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:08:27.468918 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5dac090-cce3-45d7-9d09-8371efb0b7f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:08:27.469579 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5dac090-cce3-45d7-9d09-8371efb0b7f0 None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:08:27.469859 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5dac090-cce3-45d7-9d09-8371efb0b7f0 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:08:27.470132 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5dac090-cce3-45d7-9d09-8371efb0b7f0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:08:27.482497 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5dac090-cce3-45d7-9d09-8371efb0b7f0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:08:27.484700 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5dac090-cce3-45d7-9d09-8371efb0b7f0 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:08:27.499918 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application [None req-e5dac090-cce3-45d7-9d09-8371efb0b7f0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 03:08:27.499918 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 03:08:27.499918 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 03:08:27.499918 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 03:08:27.499918 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 03:08:27.499918 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 03:08:27.499918 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 03:08:27.499918 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 03:08:27.499918 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 03:08:27.499918 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 03:08:27.499918 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 03:08:27.499918 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 03:08:27.499918 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 13 03:08:27.499918 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 13 03:08:27.499918 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 13 03:08:27.499918 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 13 03:08:27.501308 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 03:08:27.501308 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 03:08:27.501308 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 03:08:27.501308 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 03:08:27.501308 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 03:08:27.501308 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 03:08:27.501308 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 03:08:27.501308 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application with Lock( Sep 13 03:08:27.501308 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 03:08:27.501308 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._enter() Sep 13 03:08:27.501308 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 03:08:27.501308 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 03:08:27.501308 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 03:08:27.501308 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.creator() Sep 13 03:08:27.501308 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 03:08:27.501308 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application created_value = creator( Sep 13 03:08:27.501308 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 13 03:08:27.502287 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 13 03:08:27.502287 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 03:08:27.502287 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Sep 13 03:08:27.502287 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 357/724] 10.222.0.18 () {56 vars in 1108 bytes} [Tue Sep 13 03:08:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 03:08:27.508459 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f6d53d4-3b4c-4446-b5a0-2707a60f8841 None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:08:27.562585 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:08:27.615999 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3de57030-654f-4900-95f1-ff112a171aaa None None] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:08:27.616343 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3de57030-654f-4900-95f1-ff112a171aaa None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:08:27.616543 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3de57030-654f-4900-95f1-ff112a171aaa None None] PATH_INFO: `/` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:08:27.618708 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 358/725] 10.222.0.18 () {54 vars in 927 bytes} [Tue Sep 13 03:08:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 03:08:27.623253 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40599501-2a72-4bdb-822a-d1a9300c3de3 None None] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:08:27.623543 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40599501-2a72-4bdb-822a-d1a9300c3de3 None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:08:27.623870 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40599501-2a72-4bdb-822a-d1a9300c3de3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:08:27.631840 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f6d53d4-3b4c-4446-b5a0-2707a60f8841 None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:08:27.656417 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.auth.core [None req-40599501-2a72-4bdb-822a-d1a9300c3de3 None None] MFA Rules not processed for user `49f280ab086b4610855c4ffa378517d9`. Rule list: `[]` (Enabled: `True`). {{(pid=81399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 03:08:27.657627 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-40599501-2a72-4bdb-822a-d1a9300c3de3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:08:27.686690 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f31d83c2-cdb7-43ba-97c0-bb591d1b73f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:08:27.688701 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f31d83c2-cdb7-43ba-97c0-bb591d1b73f5 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:08:27.689375 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f31d83c2-cdb7-43ba-97c0-bb591d1b73f5 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:08:27.690535 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f31d83c2-cdb7-43ba-97c0-bb591d1b73f5 None admin] PATH_INFO: `/v3/domains` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:08:27.691422 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f31d83c2-cdb7-43ba-97c0-bb591d1b73f5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 03:08:27.691996 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f31d83c2-cdb7-43ba-97c0-bb591d1b73f5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:08:27.696179 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f31d83c2-cdb7-43ba-97c0-bb591d1b73f5 None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:08:27.719619 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 368/726] 10.222.0.18 () {56 vars in 1109 bytes} [Tue Sep 13 03:08:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 03:08:27.727825 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f31d83c2-cdb7-43ba-97c0-bb591d1b73f5 None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:08:27.813333 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 359/727] 10.222.0.18 () {58 vars in 1024 bytes} [Tue Sep 13 03:08:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 13 03:08:27.853490 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:08:27.857382 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f31d83c2-cdb7-43ba-97c0-bb591d1b73f5 None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:08:27.894321 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c59991c-76f3-408c-ab8f-47b3379a097d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:08:27.895326 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c59991c-76f3-408c-ab8f-47b3379a097d None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:08:27.895748 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c59991c-76f3-408c-ab8f-47b3379a097d None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:08:27.896162 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c59991c-76f3-408c-ab8f-47b3379a097d None admin] PATH_INFO: `/v3/users/swift` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:08:27.903883 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c59991c-76f3-408c-ab8f-47b3379a097d None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:08:27.906069 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c59991c-76f3-408c-ab8f-47b3379a097d None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:08:27.918072 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application [None req-1c59991c-76f3-408c-ab8f-47b3379a097d None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Sep 13 03:08:27.918072 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 03:08:27.918072 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 03:08:27.918072 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 03:08:27.918072 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 03:08:27.918072 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 03:08:27.918072 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 03:08:27.918072 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 03:08:27.918072 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 03:08:27.918072 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 03:08:27.918072 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 03:08:27.918072 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 03:08:27.918072 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 13 03:08:27.918072 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 13 03:08:27.918072 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 13 03:08:27.918072 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 13 03:08:27.919243 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 03:08:27.919243 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 03:08:27.919243 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 13 03:08:27.919243 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 13 03:08:27.919243 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 13 03:08:27.919243 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 13 03:08:27.919243 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 03:08:27.919243 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 03:08:27.919243 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 03:08:27.919243 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 03:08:27.919243 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 03:08:27.919243 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application with Lock( Sep 13 03:08:27.919243 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 03:08:27.919243 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._enter() Sep 13 03:08:27.919243 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 03:08:27.919243 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 03:08:27.919243 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 03:08:27.919243 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.creator() Sep 13 03:08:27.920407 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 03:08:27.920407 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application created_value = creator( Sep 13 03:08:27.920407 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 13 03:08:27.920407 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 13 03:08:27.920407 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 13 03:08:27.920407 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 13 03:08:27.920407 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 13 03:08:27.920407 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 13 03:08:27.920407 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Sep 13 03:08:27.920407 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Sep 13 03:08:27.922545 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 369/728] 10.222.0.18 () {56 vars in 1096 bytes} [Tue Sep 13 03:08:27 2022] GET /identity/v3/users/swift => generated 83 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 03:08:27.928298 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40599501-2a72-4bdb-822a-d1a9300c3de3 None None] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:08:27.961849 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:08:28.041405 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40599501-2a72-4bdb-822a-d1a9300c3de3 None None] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:08:28.079523 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d7d5ccd-d9df-499e-833e-0a1c7d58df1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:08:28.080141 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d7d5ccd-d9df-499e-833e-0a1c7d58df1d None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:08:28.080419 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d7d5ccd-d9df-499e-833e-0a1c7d58df1d None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:08:28.080690 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d7d5ccd-d9df-499e-833e-0a1c7d58df1d None admin] PATH_INFO: `/v3/users` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:08:28.081418 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d7d5ccd-d9df-499e-833e-0a1c7d58df1d None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 03:08:28.081719 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d7d5ccd-d9df-499e-833e-0a1c7d58df1d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:08:28.083187 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d7d5ccd-d9df-499e-833e-0a1c7d58df1d None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:08:28.101840 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 360/729] 10.222.0.18 () {56 vars in 1135 bytes} [Tue Sep 13 03:08:27 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 03:08:28.107354 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c59991c-76f3-408c-ab8f-47b3379a097d None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:08:28.149287 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b742533-43b8-459c-9adf-e0910a4a63e5 None None] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:08:28.149613 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b742533-43b8-459c-9adf-e0910a4a63e5 None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:08:28.149836 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b742533-43b8-459c-9adf-e0910a4a63e5 None None] PATH_INFO: `/` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:08:28.151194 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 361/730] 10.222.0.18 () {54 vars in 927 bytes} [Tue Sep 13 03:08:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 13 03:08:28.156518 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f75dee26-2016-4f81-8475-a0a0b0a84ddf None None] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:08:28.156796 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f75dee26-2016-4f81-8475-a0a0b0a84ddf None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:08:28.157069 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f75dee26-2016-4f81-8475-a0a0b0a84ddf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:08:28.176986 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.auth.core [None req-f75dee26-2016-4f81-8475-a0a0b0a84ddf None None] MFA Rules not processed for user `49f280ab086b4610855c4ffa378517d9`. Rule list: `[]` (Enabled: `True`). {{(pid=81399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 03:08:28.179869 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-f75dee26-2016-4f81-8475-a0a0b0a84ddf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:08:28.228196 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c59991c-76f3-408c-ab8f-47b3379a097d None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:08:28.261758 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 362/731] 10.222.0.18 () {58 vars in 1024 bytes} [Tue Sep 13 03:08:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 13 03:08:28.262605 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f75dee26-2016-4f81-8475-a0a0b0a84ddf None None] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:08:28.263698 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c56d03d4-d28a-4ce2-883b-a12915e04e87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:08:28.264357 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c56d03d4-d28a-4ce2-883b-a12915e04e87 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:08:28.265125 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c56d03d4-d28a-4ce2-883b-a12915e04e87 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:08:28.265394 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c56d03d4-d28a-4ce2-883b-a12915e04e87 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:08:28.283115 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:08:28.294193 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:08:28.337721 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c56d03d4-d28a-4ce2-883b-a12915e04e87 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:08:28.339146 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c56d03d4-d28a-4ce2-883b-a12915e04e87 None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:08:28.392880 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f75dee26-2016-4f81-8475-a0a0b0a84ddf None None] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:08:28.454034 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a511caf-8fc9-4cab-86f5-d5addc850c25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:08:28.454695 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a511caf-8fc9-4cab-86f5-d5addc850c25 None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:08:28.454974 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a511caf-8fc9-4cab-86f5-d5addc850c25 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:08:28.455247 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a511caf-8fc9-4cab-86f5-d5addc850c25 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:08:28.473486 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 370/732] 10.222.0.18 () {58 vars in 1282 bytes} [Tue Sep 13 03:08:28 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 364 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 13 03:08:28.474317 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a511caf-8fc9-4cab-86f5-d5addc850c25 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 03:08:28.476343 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c56d03d4-d28a-4ce2-883b-a12915e04e87 None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:08:28.488781 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application [None req-8a511caf-8fc9-4cab-86f5-d5addc850c25 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 13 03:08:28.488781 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 03:08:28.488781 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 03:08:28.488781 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 03:08:28.488781 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 03:08:28.488781 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 03:08:28.488781 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 03:08:28.488781 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 03:08:28.488781 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 03:08:28.488781 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 03:08:28.488781 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 03:08:28.488781 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 03:08:28.488781 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 03:08:28.488781 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 03:08:28.488781 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 03:08:28.488781 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 03:08:28.489812 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 03:08:28.489812 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 03:08:28.489812 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 03:08:28.489812 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 03:08:28.489812 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 03:08:28.489812 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 03:08:28.489812 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 03:08:28.489812 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 03:08:28.489812 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 03:08:28.489812 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 03:08:28.489812 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 03:08:28.489812 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application with Lock( Sep 13 03:08:28.489812 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 03:08:28.489812 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._enter() Sep 13 03:08:28.489812 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 03:08:28.489812 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 03:08:28.490728 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 03:08:28.490728 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.creator() Sep 13 03:08:28.490728 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 03:08:28.490728 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application created_value = creator( Sep 13 03:08:28.490728 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 03:08:28.490728 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 03:08:28.490728 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 03:08:28.490728 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 03:08:28.490728 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 03:08:28.490728 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 03:08:28.490728 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 13 03:08:28.490728 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Sep 13 03:08:28.490728 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 363/733] 10.222.0.18 () {56 vars in 1102 bytes} [Tue Sep 13 03:08:28 2022] GET /identity/v3/roles/service => generated 85 bytes in 228 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 03:08:28.494918 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f170f675-436a-4f9d-a765-6dfff363ba9d None None] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:08:28.495195 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f170f675-436a-4f9d-a765-6dfff363ba9d None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:08:28.495468 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f170f675-436a-4f9d-a765-6dfff363ba9d None None] PATH_INFO: `/` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:08:28.496836 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 364/734] 10.222.0.18 () {54 vars in 927 bytes} [Tue Sep 13 03:08:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 03:08:28.499482 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f170f675-436a-4f9d-a765-6dfff363ba9d None None] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:08:28.548792 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:08:28.589373 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:08:28.594036 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c56d03d4-d28a-4ce2-883b-a12915e04e87 None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:08:28.614971 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f170f675-436a-4f9d-a765-6dfff363ba9d None None] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:08:28.629113 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dee74e4-52cf-410b-93a0-193f8eab2ea4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:08:28.629734 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dee74e4-52cf-410b-93a0-193f8eab2ea4 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:08:28.630005 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dee74e4-52cf-410b-93a0-193f8eab2ea4 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:08:28.630272 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dee74e4-52cf-410b-93a0-193f8eab2ea4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:08:28.639910 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dee74e4-52cf-410b-93a0-193f8eab2ea4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:08:28.641329 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dee74e4-52cf-410b-93a0-193f8eab2ea4 None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:08:28.648635 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd4ba78f-578c-4bb1-b190-5f2a270974b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:08:28.649233 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd4ba78f-578c-4bb1-b190-5f2a270974b7 None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:08:28.649514 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd4ba78f-578c-4bb1-b190-5f2a270974b7 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:08:28.649783 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd4ba78f-578c-4bb1-b190-5f2a270974b7 None admin] PATH_INFO: `/v3/roles` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:08:28.650430 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd4ba78f-578c-4bb1-b190-5f2a270974b7 None admin] RBAC: Adding query filter params (name=service) {{(pid=81399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 03:08:28.650771 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd4ba78f-578c-4bb1-b190-5f2a270974b7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:08:28.652110 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd4ba78f-578c-4bb1-b190-5f2a270974b7 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:08:28.655805 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application [None req-1dee74e4-52cf-410b-93a0-193f8eab2ea4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 03:08:28.655805 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 03:08:28.655805 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 03:08:28.655805 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 03:08:28.655805 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 03:08:28.655805 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 03:08:28.655805 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 03:08:28.655805 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 03:08:28.655805 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 03:08:28.655805 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 03:08:28.655805 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 03:08:28.655805 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 03:08:28.655805 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 13 03:08:28.655805 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 13 03:08:28.655805 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 13 03:08:28.655805 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 13 03:08:28.656848 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 03:08:28.656848 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 03:08:28.656848 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 03:08:28.656848 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 03:08:28.656848 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 03:08:28.656848 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 03:08:28.656848 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 03:08:28.656848 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application with Lock( Sep 13 03:08:28.656848 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 03:08:28.656848 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._enter() Sep 13 03:08:28.656848 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 03:08:28.656848 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 03:08:28.656848 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 03:08:28.656848 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.creator() Sep 13 03:08:28.656848 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 03:08:28.656848 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application created_value = creator( Sep 13 03:08:28.656848 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 13 03:08:28.657734 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 13 03:08:28.657734 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 03:08:28.657734 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Sep 13 03:08:28.657734 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 371/735] 10.222.0.18 () {56 vars in 1108 bytes} [Tue Sep 13 03:08:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 03:08:28.658043 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e749cc89-27cb-4478-bafd-1ec634b1328b None None] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:08:28.658285 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e749cc89-27cb-4478-bafd-1ec634b1328b None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:08:28.658573 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e749cc89-27cb-4478-bafd-1ec634b1328b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:08:28.668592 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 365/736] 10.222.0.18 () {56 vars in 1103 bytes} [Tue Sep 13 03:08:28 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 03:08:28.668592 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dfd98bd-6927-4a6f-96cf-6cc8735fc87c None None] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:08:28.668592 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dfd98bd-6927-4a6f-96cf-6cc8735fc87c None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:08:28.668592 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dfd98bd-6927-4a6f-96cf-6cc8735fc87c None None] PATH_INFO: `/` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:08:28.668592 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 366/737] 10.222.0.18 () {54 vars in 927 bytes} [Tue Sep 13 03:08:28 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 03:08:28.668592 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dfd98bd-6927-4a6f-96cf-6cc8735fc87c None None] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:08:28.677570 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.auth.core [None req-e749cc89-27cb-4478-bafd-1ec634b1328b None None] MFA Rules not processed for user `49f280ab086b4610855c4ffa378517d9`. Rule list: `[]` (Enabled: `True`). {{(pid=81400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 03:08:28.678805 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-e749cc89-27cb-4478-bafd-1ec634b1328b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:08:28.744514 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 372/738] 10.222.0.18 () {58 vars in 1024 bytes} [Tue Sep 13 03:08:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 13 03:08:28.745335 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e749cc89-27cb-4478-bafd-1ec634b1328b None None] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:08:28.772576 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dfd98bd-6927-4a6f-96cf-6cc8735fc87c None None] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:08:28.787212 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:08:28.811491 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cf29783-ce50-497b-85f7-58a2f2c7aa2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:08:28.812614 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cf29783-ce50-497b-85f7-58a2f2c7aa2d None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:08:28.812898 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cf29783-ce50-497b-85f7-58a2f2c7aa2d None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:08:28.813170 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cf29783-ce50-497b-85f7-58a2f2c7aa2d None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:08:28.822888 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:08:28.827837 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cf29783-ce50-497b-85f7-58a2f2c7aa2d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 03:08:28.832729 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application [None req-4cf29783-ce50-497b-85f7-58a2f2c7aa2d None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 13 03:08:28.832729 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 03:08:28.832729 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 03:08:28.832729 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 03:08:28.832729 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 03:08:28.832729 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 03:08:28.832729 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 03:08:28.832729 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 03:08:28.832729 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 03:08:28.832729 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 03:08:28.832729 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 03:08:28.832729 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 03:08:28.832729 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 03:08:28.832729 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 03:08:28.832729 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 03:08:28.832729 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 03:08:28.833644 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 03:08:28.833644 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 03:08:28.833644 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 03:08:28.833644 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 03:08:28.833644 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 03:08:28.833644 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 03:08:28.833644 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 03:08:28.833644 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 03:08:28.833644 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 03:08:28.833644 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 03:08:28.833644 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 03:08:28.833644 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application with Lock( Sep 13 03:08:28.833644 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 03:08:28.833644 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._enter() Sep 13 03:08:28.833644 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 03:08:28.833644 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 03:08:28.834521 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 03:08:28.834521 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.creator() Sep 13 03:08:28.834521 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 03:08:28.834521 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application created_value = creator( Sep 13 03:08:28.834521 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 03:08:28.834521 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 03:08:28.834521 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 03:08:28.834521 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 03:08:28.834521 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 03:08:28.834521 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 03:08:28.834521 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 13 03:08:28.834521 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Sep 13 03:08:28.834521 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 367/739] 10.222.0.18 () {56 vars in 1096 bytes} [Tue Sep 13 03:08:28 2022] GET /identity/v3/roles/admin => generated 83 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 03:08:28.835265 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cf29783-ce50-497b-85f7-58a2f2c7aa2d None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:08:28.852452 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e749cc89-27cb-4478-bafd-1ec634b1328b None None] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:08:28.890021 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed6557f8-932a-4fee-a604-a4209dc96f1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:08:28.890631 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed6557f8-932a-4fee-a604-a4209dc96f1c None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:08:28.890929 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed6557f8-932a-4fee-a604-a4209dc96f1c None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:08:28.891204 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed6557f8-932a-4fee-a604-a4209dc96f1c None admin] PATH_INFO: `/v3/domains` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:08:28.891843 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed6557f8-932a-4fee-a604-a4209dc96f1c None admin] RBAC: Adding query filter params (name=Default) {{(pid=81400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 03:08:28.892144 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed6557f8-932a-4fee-a604-a4209dc96f1c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:08:28.893537 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed6557f8-932a-4fee-a604-a4209dc96f1c None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:08:28.906912 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 373/740] 10.222.0.18 () {56 vars in 1109 bytes} [Tue Sep 13 03:08:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 03:08:28.908273 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c84e1271-f0de-49f5-8606-d23d1bd20f5e None None] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:08:28.908555 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c84e1271-f0de-49f5-8606-d23d1bd20f5e None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:08:28.908828 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c84e1271-f0de-49f5-8606-d23d1bd20f5e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:08:28.927728 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.auth.core [None req-c84e1271-f0de-49f5-8606-d23d1bd20f5e None None] MFA Rules not processed for user `49f280ab086b4610855c4ffa378517d9`. Rule list: `[]` (Enabled: `True`). {{(pid=81400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 03:08:28.928907 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-c84e1271-f0de-49f5-8606-d23d1bd20f5e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:08:28.952780 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cf29783-ce50-497b-85f7-58a2f2c7aa2d None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:08:28.987314 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13727856-5894-4fe1-8732-0b5179a9567a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:08:28.988349 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13727856-5894-4fe1-8732-0b5179a9567a None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:08:28.988627 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13727856-5894-4fe1-8732-0b5179a9567a None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:08:28.988979 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13727856-5894-4fe1-8732-0b5179a9567a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:08:28.999388 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13727856-5894-4fe1-8732-0b5179a9567a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:08:29.000797 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13727856-5894-4fe1-8732-0b5179a9567a None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:08:29.007350 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 374/741] 10.222.0.18 () {58 vars in 1024 bytes} [Tue Sep 13 03:08:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 13 03:08:29.008522 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c84e1271-f0de-49f5-8606-d23d1bd20f5e None None] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:08:29.015635 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application [None req-13727856-5894-4fe1-8732-0b5179a9567a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 03:08:29.015635 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 03:08:29.015635 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 03:08:29.015635 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 03:08:29.015635 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 03:08:29.015635 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 03:08:29.015635 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 03:08:29.015635 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 03:08:29.015635 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 03:08:29.015635 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 03:08:29.015635 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 03:08:29.015635 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 03:08:29.015635 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 13 03:08:29.015635 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 13 03:08:29.015635 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 13 03:08:29.015635 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 13 03:08:29.016571 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 03:08:29.016571 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 03:08:29.016571 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 03:08:29.016571 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 03:08:29.016571 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 03:08:29.016571 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 03:08:29.016571 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 03:08:29.016571 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application with Lock( Sep 13 03:08:29.016571 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 03:08:29.016571 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._enter() Sep 13 03:08:29.016571 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 03:08:29.016571 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 03:08:29.016571 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 03:08:29.016571 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.creator() Sep 13 03:08:29.016571 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 03:08:29.016571 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application created_value = creator( Sep 13 03:08:29.016571 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 13 03:08:29.017480 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 13 03:08:29.017480 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 03:08:29.017480 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Sep 13 03:08:29.017480 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 368/742] 10.222.0.18 () {56 vars in 1108 bytes} [Tue Sep 13 03:08:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 03:08:29.017718 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13727856-5894-4fe1-8732-0b5179a9567a None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:08:29.042320 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:08:29.078935 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:08:29.116194 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c84e1271-f0de-49f5-8606-d23d1bd20f5e None None] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:08:29.132602 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13727856-5894-4fe1-8732-0b5179a9567a None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:08:29.150584 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f4f5463-5ada-41d6-ad56-8b7c31aa83c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:08:29.151383 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f4f5463-5ada-41d6-ad56-8b7c31aa83c1 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:08:29.151822 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f4f5463-5ada-41d6-ad56-8b7c31aa83c1 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:08:29.152248 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f4f5463-5ada-41d6-ad56-8b7c31aa83c1 None admin] PATH_INFO: `/v3/roles` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:08:29.153069 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f4f5463-5ada-41d6-ad56-8b7c31aa83c1 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 03:08:29.153515 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f4f5463-5ada-41d6-ad56-8b7c31aa83c1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:08:29.155595 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f4f5463-5ada-41d6-ad56-8b7c31aa83c1 None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:08:29.168054 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0355b95-d6a6-4683-b57b-6d10ffbb11ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:08:29.168054 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0355b95-d6a6-4683-b57b-6d10ffbb11ed None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:08:29.168054 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0355b95-d6a6-4683-b57b-6d10ffbb11ed None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:08:29.168054 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0355b95-d6a6-4683-b57b-6d10ffbb11ed None admin] PATH_INFO: `/v3/projects/service` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:08:29.168994 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 375/743] 10.222.0.18 () {56 vars in 1099 bytes} [Tue Sep 13 03:08:29 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 03:08:29.169930 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f4f5463-5ada-41d6-ad56-8b7c31aa83c1 None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:08:29.174908 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0355b95-d6a6-4683-b57b-6d10ffbb11ed None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:08:29.176335 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0355b95-d6a6-4683-b57b-6d10ffbb11ed None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:08:29.188204 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application [None req-f0355b95-d6a6-4683-b57b-6d10ffbb11ed None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 13 03:08:29.188204 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 03:08:29.188204 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 03:08:29.188204 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 03:08:29.188204 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 03:08:29.188204 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 03:08:29.188204 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 03:08:29.188204 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 03:08:29.188204 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 03:08:29.188204 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 03:08:29.188204 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 03:08:29.188204 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 03:08:29.188204 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 13 03:08:29.188204 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 13 03:08:29.188204 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 13 03:08:29.189274 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 13 03:08:29.189274 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 03:08:29.189274 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 03:08:29.189274 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 03:08:29.189274 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 03:08:29.189274 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 03:08:29.189274 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 03:08:29.189274 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 03:08:29.189274 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application with Lock( Sep 13 03:08:29.189274 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 03:08:29.189274 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._enter() Sep 13 03:08:29.189274 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 03:08:29.189274 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 03:08:29.189274 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 03:08:29.189274 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.creator() Sep 13 03:08:29.189274 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 03:08:29.189274 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application created_value = creator( Sep 13 03:08:29.190375 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 13 03:08:29.190375 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 13 03:08:29.190375 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 13 03:08:29.190375 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 13 03:08:29.190375 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 13 03:08:29.190375 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 13 03:08:29.190375 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 13 03:08:29.190375 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Sep 13 03:08:29.196615 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 369/744] 10.222.0.18 () {56 vars in 1111 bytes} [Tue Sep 13 03:08:29 2022] GET /identity/v3/projects/service => generated 88 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 03:08:29.196615 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0355b95-d6a6-4683-b57b-6d10ffbb11ed None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:08:29.265821 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:08:29.273609 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f4f5463-5ada-41d6-ad56-8b7c31aa83c1 None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:08:29.299850 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0355b95-d6a6-4683-b57b-6d10ffbb11ed None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:08:29.306921 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3132c409-07b9-4bdb-bd79-eccd0dc45a22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:08:29.307523 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3132c409-07b9-4bdb-bd79-eccd0dc45a22 None admin] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:08:29.307803 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3132c409-07b9-4bdb-bd79-eccd0dc45a22 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:08:29.308073 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3132c409-07b9-4bdb-bd79-eccd0dc45a22 None admin] PATH_INFO: `/v3/services` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:08:29.308718 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3132c409-07b9-4bdb-bd79-eccd0dc45a22 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:08:29.310085 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3132c409-07b9-4bdb-bd79-eccd0dc45a22 None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:08:29.312227 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:08:29.326083 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 376/745] 10.222.0.18 () {60 vars in 1141 bytes} [Tue Sep 13 03:08:29 2022] POST /identity/v3/services => generated 240 bytes in 156 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 03:08:29.327421 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3132c409-07b9-4bdb-bd79-eccd0dc45a22 None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:08:29.349337 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb0cf1e5-558a-4594-b7d4-c7fd1c315833 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:08:29.349972 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb0cf1e5-558a-4594-b7d4-c7fd1c315833 None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:08:29.350239 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb0cf1e5-558a-4594-b7d4-c7fd1c315833 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:08:29.350504 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb0cf1e5-558a-4594-b7d4-c7fd1c315833 None admin] PATH_INFO: `/v3/domains` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:08:29.351301 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb0cf1e5-558a-4594-b7d4-c7fd1c315833 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 03:08:29.351628 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb0cf1e5-558a-4594-b7d4-c7fd1c315833 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:08:29.353208 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb0cf1e5-558a-4594-b7d4-c7fd1c315833 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:08:29.371546 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 370/746] 10.222.0.18 () {56 vars in 1109 bytes} [Tue Sep 13 03:08:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 03:08:29.371546 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb0cf1e5-558a-4594-b7d4-c7fd1c315833 None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:08:29.446170 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3132c409-07b9-4bdb-bd79-eccd0dc45a22 None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:08:29.484003 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69f96257-842c-4408-a867-590ce7e82228 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:08:29.485024 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f96257-842c-4408-a867-590ce7e82228 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:08:29.485133 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb0cf1e5-558a-4594-b7d4-c7fd1c315833 None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:08:29.485628 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f96257-842c-4408-a867-590ce7e82228 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:08:29.486099 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f96257-842c-4408-a867-590ce7e82228 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:08:29.498146 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69f96257-842c-4408-a867-590ce7e82228 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:08:29.500222 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69f96257-842c-4408-a867-590ce7e82228 None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:08:29.516839 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application [None req-69f96257-842c-4408-a867-590ce7e82228 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 03:08:29.516839 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 03:08:29.516839 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 03:08:29.516839 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 03:08:29.516839 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 03:08:29.516839 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 03:08:29.516839 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 03:08:29.516839 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 03:08:29.516839 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 03:08:29.516839 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 03:08:29.516839 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 03:08:29.516839 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 03:08:29.516839 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 13 03:08:29.516839 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 13 03:08:29.516839 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 13 03:08:29.516839 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 13 03:08:29.517891 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 03:08:29.517891 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 03:08:29.517891 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 03:08:29.517891 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 03:08:29.517891 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 03:08:29.517891 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 03:08:29.517891 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 03:08:29.517891 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application with Lock( Sep 13 03:08:29.517891 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 03:08:29.517891 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._enter() Sep 13 03:08:29.517891 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 03:08:29.517891 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 03:08:29.517891 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 03:08:29.517891 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.creator() Sep 13 03:08:29.517891 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 03:08:29.517891 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application created_value = creator( Sep 13 03:08:29.517891 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 13 03:08:29.518816 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 13 03:08:29.518816 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 03:08:29.518816 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Sep 13 03:08:29.518816 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 377/747] 10.222.0.18 () {56 vars in 1108 bytes} [Tue Sep 13 03:08:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 03:08:29.518816 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69f96257-842c-4408-a867-590ce7e82228 None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:08:29.528517 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:08:29.529059 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-041a0cfb-5b8c-4094-86ce-b33b887e7c95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:08:29.529679 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-041a0cfb-5b8c-4094-86ce-b33b887e7c95 None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:08:29.529965 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-041a0cfb-5b8c-4094-86ce-b33b887e7c95 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:08:29.530246 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-041a0cfb-5b8c-4094-86ce-b33b887e7c95 None admin] PATH_INFO: `/v3/projects` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:08:29.531214 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-041a0cfb-5b8c-4094-86ce-b33b887e7c95 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 03:08:29.531646 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-041a0cfb-5b8c-4094-86ce-b33b887e7c95 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:08:29.533087 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-041a0cfb-5b8c-4094-86ce-b33b887e7c95 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:08:29.547663 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 371/748] 10.222.0.18 () {56 vars in 1148 bytes} [Tue Sep 13 03:08:29 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 03:08:29.548535 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-041a0cfb-5b8c-4094-86ce-b33b887e7c95 None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:08:29.583211 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:08:29.631162 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69f96257-842c-4408-a867-590ce7e82228 None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:08:29.661363 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-041a0cfb-5b8c-4094-86ce-b33b887e7c95 None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:08:29.667574 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-109da78c-f8e0-44c8-8ec8-bdd89ae91feb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:08:29.668180 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-109da78c-f8e0-44c8-8ec8-bdd89ae91feb None admin] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:08:29.668456 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-109da78c-f8e0-44c8-8ec8-bdd89ae91feb None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:08:29.668723 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-109da78c-f8e0-44c8-8ec8-bdd89ae91feb None admin] PATH_INFO: `/v3/services` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:08:29.669369 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-109da78c-f8e0-44c8-8ec8-bdd89ae91feb None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:08:29.670751 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-109da78c-f8e0-44c8-8ec8-bdd89ae91feb None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:08:29.686895 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 378/749] 10.222.0.18 () {60 vars in 1141 bytes} [Tue Sep 13 03:08:29 2022] POST /identity/v3/services => generated 251 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 03:08:29.687536 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-109da78c-f8e0-44c8-8ec8-bdd89ae91feb None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:08:29.714276 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50608e6a-ddc3-4975-9c11-03ee9167794c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:08:29.715137 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50608e6a-ddc3-4975-9c11-03ee9167794c None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:08:29.715573 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50608e6a-ddc3-4975-9c11-03ee9167794c None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:08:29.716006 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50608e6a-ddc3-4975-9c11-03ee9167794c None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:08:29.726008 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50608e6a-ddc3-4975-9c11-03ee9167794c None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:08:29.727764 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50608e6a-ddc3-4975-9c11-03ee9167794c None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:08:29.741085 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application [None req-50608e6a-ddc3-4975-9c11-03ee9167794c None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 13 03:08:29.741085 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 03:08:29.741085 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 03:08:29.741085 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 03:08:29.741085 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 03:08:29.741085 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 03:08:29.741085 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 03:08:29.741085 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 03:08:29.741085 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 03:08:29.741085 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 03:08:29.741085 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 03:08:29.741085 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 03:08:29.741085 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 13 03:08:29.741085 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 13 03:08:29.741085 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 13 03:08:29.741085 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 13 03:08:29.742331 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 03:08:29.742331 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 03:08:29.742331 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 13 03:08:29.742331 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 13 03:08:29.742331 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 13 03:08:29.742331 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 13 03:08:29.742331 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 03:08:29.742331 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 03:08:29.742331 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 03:08:29.742331 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 03:08:29.742331 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 03:08:29.742331 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application with Lock( Sep 13 03:08:29.742331 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 03:08:29.742331 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._enter() Sep 13 03:08:29.742331 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 03:08:29.742331 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 03:08:29.742331 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 03:08:29.742331 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.creator() Sep 13 03:08:29.743601 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 03:08:29.743601 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application created_value = creator( Sep 13 03:08:29.743601 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 13 03:08:29.743601 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 13 03:08:29.743601 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 13 03:08:29.743601 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 13 03:08:29.743601 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 13 03:08:29.743601 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 13 03:08:29.743601 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 13 03:08:29.743601 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Sep 13 03:08:29.750597 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 372/750] 10.222.0.18 () {56 vars in 1117 bytes} [Tue Sep 13 03:08:29 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 03:08:29.751722 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50608e6a-ddc3-4975-9c11-03ee9167794c None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:08:29.780180 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:08:29.826978 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-109da78c-f8e0-44c8-8ec8-bdd89ae91feb None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:08:29.870676 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ad59a81-e018-4710-8480-c0bcf492ff48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:08:29.871553 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ad59a81-e018-4710-8480-c0bcf492ff48 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:08:29.872022 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ad59a81-e018-4710-8480-c0bcf492ff48 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:08:29.872711 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:08:29.873200 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ad59a81-e018-4710-8480-c0bcf492ff48 None admin] PATH_INFO: `/v3/domains` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:08:29.874699 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ad59a81-e018-4710-8480-c0bcf492ff48 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 03:08:29.875165 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ad59a81-e018-4710-8480-c0bcf492ff48 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:08:29.876838 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ad59a81-e018-4710-8480-c0bcf492ff48 None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:08:29.893067 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 379/751] 10.222.0.18 () {56 vars in 1109 bytes} [Tue Sep 13 03:08:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 03:08:29.893666 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ad59a81-e018-4710-8480-c0bcf492ff48 None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:08:29.893845 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50608e6a-ddc3-4975-9c11-03ee9167794c None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:08:29.941022 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2f251f2-e21d-4bd3-8f07-b11d21efe030 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:08:29.941989 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2f251f2-e21d-4bd3-8f07-b11d21efe030 None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:08:29.942266 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2f251f2-e21d-4bd3-8f07-b11d21efe030 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:08:29.942531 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2f251f2-e21d-4bd3-8f07-b11d21efe030 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:08:29.943633 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2f251f2-e21d-4bd3-8f07-b11d21efe030 None admin] RBAC: Adding query filter params (role.id=425d4b481c7846e0bc07e7e805d5b15f, scope.project.id=b9d94f6a68f7470d9494f112175620dc, user.id=893b0c0d85964b7dbc9705c0a091682c) {{(pid=81399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 03:08:29.944136 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2f251f2-e21d-4bd3-8f07-b11d21efe030 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:08:29.945847 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2f251f2-e21d-4bd3-8f07-b11d21efe030 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:08:29.953554 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 373/752] 10.222.0.18 () {56 vars in 1374 bytes} [Tue Sep 13 03:08:29 2022] GET /identity/v3/role_assignments?user.id=893b0c0d85964b7dbc9705c0a091682c&scope.project.id=b9d94f6a68f7470d9494f112175620dc&role.id=425d4b481c7846e0bc07e7e805d5b15f => generated 259 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 03:08:29.954461 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2f251f2-e21d-4bd3-8f07-b11d21efe030 None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:08:30.025288 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ad59a81-e018-4710-8480-c0bcf492ff48 None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:08:30.083351 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:08:30.101402 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69b5d958-bc26-46d4-a177-fecd1ae192a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:08:30.102030 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b5d958-bc26-46d4-a177-fecd1ae192a5 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:08:30.102304 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b5d958-bc26-46d4-a177-fecd1ae192a5 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:08:30.102573 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b5d958-bc26-46d4-a177-fecd1ae192a5 None admin] PATH_INFO: `/v3/users` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:08:30.103353 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69b5d958-bc26-46d4-a177-fecd1ae192a5 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 03:08:30.103654 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69b5d958-bc26-46d4-a177-fecd1ae192a5 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:08:30.105120 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69b5d958-bc26-46d4-a177-fecd1ae192a5 None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:08:30.141238 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 380/753] 10.222.0.18 () {56 vars in 1149 bytes} [Tue Sep 13 03:08:29 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 248 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 03:08:30.156632 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2f251f2-e21d-4bd3-8f07-b11d21efe030 None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:08:30.161493 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69b5d958-bc26-46d4-a177-fecd1ae192a5 None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:08:30.221466 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-691714e2-f90c-47b2-b027-0707b5a2a334 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:08:30.222105 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-691714e2-f90c-47b2-b027-0707b5a2a334 None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:08:30.222382 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-691714e2-f90c-47b2-b027-0707b5a2a334 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:08:30.222664 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-691714e2-f90c-47b2-b027-0707b5a2a334 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:08:30.241869 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-691714e2-f90c-47b2-b027-0707b5a2a334 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:08:30.243358 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-691714e2-f90c-47b2-b027-0707b5a2a334 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:08:30.264639 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application [None req-691714e2-f90c-47b2-b027-0707b5a2a334 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 13 03:08:30.264639 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 03:08:30.264639 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 03:08:30.264639 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 03:08:30.264639 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 03:08:30.264639 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 03:08:30.264639 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 03:08:30.264639 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 03:08:30.264639 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 03:08:30.264639 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 03:08:30.264639 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 03:08:30.264639 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 03:08:30.264639 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 13 03:08:30.264639 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 13 03:08:30.264639 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 13 03:08:30.264639 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 13 03:08:30.265659 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 03:08:30.265659 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 03:08:30.265659 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 13 03:08:30.265659 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 13 03:08:30.265659 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 13 03:08:30.265659 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 13 03:08:30.265659 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 03:08:30.265659 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 03:08:30.265659 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 03:08:30.265659 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 03:08:30.265659 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 03:08:30.265659 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application with Lock( Sep 13 03:08:30.265659 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 03:08:30.265659 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._enter() Sep 13 03:08:30.265659 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 03:08:30.265659 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 03:08:30.265659 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 03:08:30.265659 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.creator() Sep 13 03:08:30.267269 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 03:08:30.267269 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application created_value = creator( Sep 13 03:08:30.267269 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 13 03:08:30.267269 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 13 03:08:30.267269 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 13 03:08:30.267269 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 13 03:08:30.267269 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 13 03:08:30.267269 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 13 03:08:30.267269 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 13 03:08:30.267269 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Sep 13 03:08:30.267269 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 374/754] 10.222.0.18 () {56 vars in 1093 bytes} [Tue Sep 13 03:08:29 2022] GET /identity/v3/users/nova => generated 82 bytes in 312 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 03:08:30.271526 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-691714e2-f90c-47b2-b027-0707b5a2a334 None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:08:30.288994 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:08:30.340165 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69b5d958-bc26-46d4-a177-fecd1ae192a5 None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:08:30.417266 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-691714e2-f90c-47b2-b027-0707b5a2a334 None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:08:30.431858 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c09ee2f-cda5-4581-a4df-094893ee1305 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:08:30.432480 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c09ee2f-cda5-4581-a4df-094893ee1305 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:08:30.432762 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c09ee2f-cda5-4581-a4df-094893ee1305 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:08:30.433029 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c09ee2f-cda5-4581-a4df-094893ee1305 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:08:30.461427 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0da40df1-7d14-4c77-87df-0904dfaf2546 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:08:30.463694 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0da40df1-7d14-4c77-87df-0904dfaf2546 None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:08:30.464207 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0da40df1-7d14-4c77-87df-0904dfaf2546 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:08:30.466203 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0da40df1-7d14-4c77-87df-0904dfaf2546 None admin] PATH_INFO: `/v3/users` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:08:30.467169 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0da40df1-7d14-4c77-87df-0904dfaf2546 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 03:08:30.468185 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0da40df1-7d14-4c77-87df-0904dfaf2546 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:08:30.470847 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0da40df1-7d14-4c77-87df-0904dfaf2546 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:08:30.504897 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 375/755] 10.222.0.18 () {56 vars in 1133 bytes} [Tue Sep 13 03:08:30 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 03:08:30.510941 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0da40df1-7d14-4c77-87df-0904dfaf2546 None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:08:30.543037 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:08:30.574994 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c09ee2f-cda5-4581-a4df-094893ee1305 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:08:30.579537 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c09ee2f-cda5-4581-a4df-094893ee1305 None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:08:30.638994 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:08:30.710255 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0da40df1-7d14-4c77-87df-0904dfaf2546 None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:08:30.765929 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2bb2f1f-4125-468f-aa8c-b304ac3ddf53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:08:30.766573 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2bb2f1f-4125-468f-aa8c-b304ac3ddf53 None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:08:30.766873 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2bb2f1f-4125-468f-aa8c-b304ac3ddf53 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:08:30.767146 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2bb2f1f-4125-468f-aa8c-b304ac3ddf53 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:08:30.773408 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 381/756] 10.222.0.18 () {58 vars in 1282 bytes} [Tue Sep 13 03:08:30 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 612 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 13 03:08:30.777980 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2bb2f1f-4125-468f-aa8c-b304ac3ddf53 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:08:30.779662 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2bb2f1f-4125-468f-aa8c-b304ac3ddf53 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:08:30.783597 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c09ee2f-cda5-4581-a4df-094893ee1305 None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:08:30.796661 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application [None req-f2bb2f1f-4125-468f-aa8c-b304ac3ddf53 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 03:08:30.796661 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 03:08:30.796661 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 03:08:30.796661 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 03:08:30.796661 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 03:08:30.796661 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 03:08:30.796661 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 03:08:30.796661 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 03:08:30.796661 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 03:08:30.796661 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 03:08:30.796661 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 03:08:30.796661 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 03:08:30.796661 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 13 03:08:30.796661 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 13 03:08:30.796661 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 13 03:08:30.796661 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 13 03:08:30.797755 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 03:08:30.797755 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 03:08:30.797755 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 03:08:30.797755 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 03:08:30.797755 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 03:08:30.797755 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 03:08:30.797755 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 03:08:30.797755 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application with Lock( Sep 13 03:08:30.797755 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 03:08:30.797755 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._enter() Sep 13 03:08:30.797755 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 03:08:30.797755 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 03:08:30.797755 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 03:08:30.797755 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.creator() Sep 13 03:08:30.797755 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 03:08:30.797755 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application created_value = creator( Sep 13 03:08:30.797755 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 13 03:08:30.799295 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 13 03:08:30.799295 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 03:08:30.799295 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Sep 13 03:08:30.799295 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 376/757] 10.222.0.18 () {56 vars in 1108 bytes} [Tue Sep 13 03:08:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 287 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 13 03:08:30.804988 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2bb2f1f-4125-468f-aa8c-b304ac3ddf53 None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:08:30.929174 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:08:30.947167 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c09ee2f-cda5-4581-a4df-094893ee1305 None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:08:30.988506 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43f8e587-895d-4900-9bec-028b859f6568 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:08:30.989063 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f8e587-895d-4900-9bec-028b859f6568 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:08:30.989339 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f8e587-895d-4900-9bec-028b859f6568 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:08:30.989609 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f8e587-895d-4900-9bec-028b859f6568 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:08:30.998339 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2bb2f1f-4125-468f-aa8c-b304ac3ddf53 None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:08:31.017013 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43f8e587-895d-4900-9bec-028b859f6568 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:08:31.018501 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43f8e587-895d-4900-9bec-028b859f6568 None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:08:31.045051 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application [None req-43f8e587-895d-4900-9bec-028b859f6568 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 03:08:31.045051 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 03:08:31.045051 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 03:08:31.045051 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 03:08:31.045051 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 03:08:31.045051 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 03:08:31.045051 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 03:08:31.045051 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 03:08:31.045051 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 03:08:31.045051 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 03:08:31.045051 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 03:08:31.045051 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 03:08:31.045051 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 13 03:08:31.045051 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 13 03:08:31.045051 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 13 03:08:31.045051 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 13 03:08:31.046099 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 03:08:31.046099 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 03:08:31.046099 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 03:08:31.046099 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 03:08:31.046099 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 03:08:31.046099 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 03:08:31.046099 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 03:08:31.046099 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application with Lock( Sep 13 03:08:31.046099 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 03:08:31.046099 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._enter() Sep 13 03:08:31.046099 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 03:08:31.046099 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 03:08:31.046099 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 03:08:31.046099 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.creator() Sep 13 03:08:31.046099 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 03:08:31.046099 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application created_value = creator( Sep 13 03:08:31.046099 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 13 03:08:31.047038 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 13 03:08:31.047038 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 03:08:31.047038 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Sep 13 03:08:31.047231 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:08:31.055417 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 382/758] 10.222.0.18 () {56 vars in 1108 bytes} [Tue Sep 13 03:08:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 272 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 03:08:31.059233 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-277d5366-4c20-48e4-9c4b-5dea0d334754 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:08:31.059860 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-277d5366-4c20-48e4-9c4b-5dea0d334754 None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:08:31.060141 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-277d5366-4c20-48e4-9c4b-5dea0d334754 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:08:31.060414 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-277d5366-4c20-48e4-9c4b-5dea0d334754 None admin] PATH_INFO: `/v3/domains` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:08:31.061588 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-277d5366-4c20-48e4-9c4b-5dea0d334754 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 03:08:31.061894 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-277d5366-4c20-48e4-9c4b-5dea0d334754 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:08:31.063297 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-277d5366-4c20-48e4-9c4b-5dea0d334754 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:08:31.071834 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43f8e587-895d-4900-9bec-028b859f6568 None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:08:31.080370 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 377/759] 10.222.0.18 () {56 vars in 1109 bytes} [Tue Sep 13 03:08:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 274 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 03:08:31.084657 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-277d5366-4c20-48e4-9c4b-5dea0d334754 None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:08:31.204219 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-277d5366-4c20-48e4-9c4b-5dea0d334754 None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:08:31.241349 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e72f0264-d6c7-4fd3-bc5f-efd6ced09209 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:08:31.241907 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e72f0264-d6c7-4fd3-bc5f-efd6ced09209 None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:08:31.242185 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e72f0264-d6c7-4fd3-bc5f-efd6ced09209 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:08:31.242454 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e72f0264-d6c7-4fd3-bc5f-efd6ced09209 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:08:31.251154 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e72f0264-d6c7-4fd3-bc5f-efd6ced09209 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:08:31.252715 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e72f0264-d6c7-4fd3-bc5f-efd6ced09209 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:08:31.293025 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43f8e587-895d-4900-9bec-028b859f6568 None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:08:31.356762 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86a1d913-8ad3-4d19-b3a0-b6a119749c9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:08:31.357768 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a1d913-8ad3-4d19-b3a0-b6a119749c9e None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:08:31.358800 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a1d913-8ad3-4d19-b3a0-b6a119749c9e None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:08:31.359301 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a1d913-8ad3-4d19-b3a0-b6a119749c9e None admin] PATH_INFO: `/v3/domains` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:08:31.360187 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86a1d913-8ad3-4d19-b3a0-b6a119749c9e None admin] RBAC: Adding query filter params (name=Default) {{(pid=81400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 03:08:31.360696 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86a1d913-8ad3-4d19-b3a0-b6a119749c9e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:08:31.362653 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86a1d913-8ad3-4d19-b3a0-b6a119749c9e None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:08:31.374501 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:08:31.386168 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application [None req-e72f0264-d6c7-4fd3-bc5f-efd6ced09209 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 13 03:08:31.386168 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 03:08:31.386168 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 03:08:31.386168 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 03:08:31.386168 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 03:08:31.386168 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 03:08:31.386168 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 03:08:31.386168 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 03:08:31.386168 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 03:08:31.386168 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 03:08:31.386168 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 03:08:31.386168 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 03:08:31.386168 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 13 03:08:31.386168 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 13 03:08:31.386168 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 13 03:08:31.387641 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 13 03:08:31.387641 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 03:08:31.387641 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 03:08:31.387641 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 03:08:31.387641 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 03:08:31.387641 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 03:08:31.387641 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 03:08:31.387641 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 03:08:31.387641 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application with Lock( Sep 13 03:08:31.387641 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 03:08:31.387641 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._enter() Sep 13 03:08:31.387641 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 03:08:31.387641 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 03:08:31.387641 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 03:08:31.387641 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.creator() Sep 13 03:08:31.387641 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 03:08:31.387641 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application created_value = creator( Sep 13 03:08:31.388549 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 13 03:08:31.388549 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 13 03:08:31.388549 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 13 03:08:31.388549 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 13 03:08:31.388549 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 13 03:08:31.388549 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 13 03:08:31.388549 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 13 03:08:31.388549 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Sep 13 03:08:31.398634 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 383/760] 10.222.0.18 () {56 vars in 1109 bytes} [Tue Sep 13 03:08:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 321 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 03:08:31.399187 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 378/761] 10.222.0.18 () {56 vars in 1111 bytes} [Tue Sep 13 03:08:31 2022] GET /identity/v3/projects/service => generated 88 bytes in 315 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 03:08:31.400400 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86a1d913-8ad3-4d19-b3a0-b6a119749c9e None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:08:31.401687 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e72f0264-d6c7-4fd3-bc5f-efd6ced09209 None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:08:31.467419 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:08:31.548088 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e72f0264-d6c7-4fd3-bc5f-efd6ced09209 None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:08:31.599042 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d354a58-5c6d-4eab-8868-29719ee2e835 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:08:31.599712 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d354a58-5c6d-4eab-8868-29719ee2e835 None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:08:31.599996 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d354a58-5c6d-4eab-8868-29719ee2e835 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:08:31.600266 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d354a58-5c6d-4eab-8868-29719ee2e835 None admin] PATH_INFO: `/v3/projects` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:08:31.600961 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d354a58-5c6d-4eab-8868-29719ee2e835 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 03:08:31.601266 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d354a58-5c6d-4eab-8868-29719ee2e835 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:08:31.602734 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d354a58-5c6d-4eab-8868-29719ee2e835 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:08:31.621377 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86a1d913-8ad3-4d19-b3a0-b6a119749c9e None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:08:31.632170 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 379/762] 10.222.0.18 () {56 vars in 1148 bytes} [Tue Sep 13 03:08:31 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 03:08:31.637892 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d354a58-5c6d-4eab-8868-29719ee2e835 None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:08:31.695375 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a0c4214-5cd6-446b-8f23-343ad9a01c6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:08:31.696013 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a0c4214-5cd6-446b-8f23-343ad9a01c6c None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:08:31.696292 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a0c4214-5cd6-446b-8f23-343ad9a01c6c None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:08:31.696563 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a0c4214-5cd6-446b-8f23-343ad9a01c6c None admin] PATH_INFO: `/v3/projects/service` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:08:31.715684 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a0c4214-5cd6-446b-8f23-343ad9a01c6c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:08:31.717145 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a0c4214-5cd6-446b-8f23-343ad9a01c6c None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:08:31.736712 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application [None req-6a0c4214-5cd6-446b-8f23-343ad9a01c6c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 13 03:08:31.736712 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 03:08:31.736712 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 03:08:31.736712 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 03:08:31.736712 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 03:08:31.736712 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 03:08:31.736712 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 03:08:31.736712 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 03:08:31.736712 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 03:08:31.736712 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 03:08:31.736712 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 03:08:31.736712 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 03:08:31.736712 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 13 03:08:31.736712 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 13 03:08:31.736712 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 13 03:08:31.737662 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 13 03:08:31.737662 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 03:08:31.737662 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 03:08:31.737662 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 03:08:31.737662 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 03:08:31.737662 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 03:08:31.737662 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 03:08:31.737662 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 03:08:31.737662 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application with Lock( Sep 13 03:08:31.737662 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 03:08:31.737662 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._enter() Sep 13 03:08:31.737662 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 03:08:31.737662 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 03:08:31.737662 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 03:08:31.737662 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.creator() Sep 13 03:08:31.737662 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 03:08:31.737662 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application created_value = creator( Sep 13 03:08:31.738569 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 13 03:08:31.738569 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 13 03:08:31.738569 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 13 03:08:31.738569 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 13 03:08:31.738569 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 13 03:08:31.738569 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 13 03:08:31.738569 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 13 03:08:31.738569 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Sep 13 03:08:31.738569 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 384/763] 10.222.0.18 () {56 vars in 1111 bytes} [Tue Sep 13 03:08:31 2022] GET /identity/v3/projects/service => generated 88 bytes in 338 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 03:08:31.745090 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a0c4214-5cd6-446b-8f23-343ad9a01c6c None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:08:31.769047 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:08:31.773620 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d354a58-5c6d-4eab-8868-29719ee2e835 None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:08:31.815718 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cebe0b1b-f2fd-4227-af7d-5f55f1fbc703 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:08:31.816300 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cebe0b1b-f2fd-4227-af7d-5f55f1fbc703 None admin] REQUEST_METHOD: `PUT` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:08:31.816577 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cebe0b1b-f2fd-4227-af7d-5f55f1fbc703 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:08:31.816845 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cebe0b1b-f2fd-4227-af7d-5f55f1fbc703 None admin] PATH_INFO: `/v3/projects/b9d94f6a68f7470d9494f112175620dc/users/c7377f5bee4a49959182e1bd34bc97a4/roles/425d4b481c7846e0bc07e7e805d5b15f` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:08:31.819673 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cebe0b1b-f2fd-4227-af7d-5f55f1fbc703 None admin] RBAC: Authorizing `identity:create_grant(project_id=b9d94f6a68f7470d9494f112175620dc, user_id=c7377f5bee4a49959182e1bd34bc97a4, role_id=425d4b481c7846e0bc07e7e805d5b15f)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:08:31.821557 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:08:31.822363 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cebe0b1b-f2fd-4227-af7d-5f55f1fbc703 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:08:31.846745 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 380/764] 10.222.0.18 () {58 vars in 1439 bytes} [Tue Sep 13 03:08:31 2022] PUT /identity/v3/projects/b9d94f6a68f7470d9494f112175620dc/users/c7377f5bee4a49959182e1bd34bc97a4/roles/425d4b481c7846e0bc07e7e805d5b15f => generated 0 bytes in 203 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 03:08:31.848545 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4574dd30-6928-49e6-b2e2-2f768dacbc55 None None] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:08:31.849033 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4574dd30-6928-49e6-b2e2-2f768dacbc55 None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:08:31.849527 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4574dd30-6928-49e6-b2e2-2f768dacbc55 None None] PATH_INFO: `/` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:08:31.851377 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 381/765] 10.222.0.18 () {54 vars in 927 bytes} [Tue Sep 13 03:08:31 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 03:08:31.864480 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b41826-e36f-4f0b-ac7a-91384abf81af None None] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:08:31.865104 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b41826-e36f-4f0b-ac7a-91384abf81af None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:08:31.865575 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b41826-e36f-4f0b-ac7a-91384abf81af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:08:31.886644 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a0c4214-5cd6-446b-8f23-343ad9a01c6c None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:08:31.925355 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b9c07c8-fbb6-46f4-bd4c-43cd2b57496b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:08:31.925979 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b9c07c8-fbb6-46f4-bd4c-43cd2b57496b None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:08:31.926301 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b9c07c8-fbb6-46f4-bd4c-43cd2b57496b None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:08:31.926579 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b9c07c8-fbb6-46f4-bd4c-43cd2b57496b None admin] PATH_INFO: `/v3/projects` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:08:31.926763 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.auth.core [None req-f4b41826-e36f-4f0b-ac7a-91384abf81af None None] MFA Rules not processed for user `49f280ab086b4610855c4ffa378517d9`. Rule list: `[]` (Enabled: `True`). {{(pid=81399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 03:08:31.927305 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b9c07c8-fbb6-46f4-bd4c-43cd2b57496b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 03:08:31.927611 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b9c07c8-fbb6-46f4-bd4c-43cd2b57496b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:08:31.927966 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-f4b41826-e36f-4f0b-ac7a-91384abf81af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:08:31.929119 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b9c07c8-fbb6-46f4-bd4c-43cd2b57496b None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:08:31.946875 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 385/766] 10.222.0.18 () {56 vars in 1148 bytes} [Tue Sep 13 03:08:31 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 03:08:31.953304 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b9c07c8-fbb6-46f4-bd4c-43cd2b57496b None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:08:32.060305 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 382/767] 10.222.0.18 () {58 vars in 1024 bytes} [Tue Sep 13 03:08:31 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 197 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 13 03:08:32.132390 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b9c07c8-fbb6-46f4-bd4c-43cd2b57496b None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:08:32.149963 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:08:32.171279 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a3fa758-4382-48ec-89a7-40f95dd5a96b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:08:32.171946 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a3fa758-4382-48ec-89a7-40f95dd5a96b None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:08:32.172278 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a3fa758-4382-48ec-89a7-40f95dd5a96b None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:08:32.172547 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a3fa758-4382-48ec-89a7-40f95dd5a96b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:08:32.173273 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a3fa758-4382-48ec-89a7-40f95dd5a96b None admin] RBAC: Adding query filter params (role.id=817d6837cd044890b41cf16b303a092b, scope.project.id=b9d94f6a68f7470d9494f112175620dc, user.id=4c2d20fb54a54ecc99b47f237be4917c) {{(pid=81400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 03:08:32.173636 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a3fa758-4382-48ec-89a7-40f95dd5a96b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:08:32.175171 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a3fa758-4382-48ec-89a7-40f95dd5a96b None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:08:32.181729 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 386/768] 10.222.0.18 () {56 vars in 1374 bytes} [Tue Sep 13 03:08:31 2022] GET /identity/v3/role_assignments?user.id=4c2d20fb54a54ecc99b47f237be4917c&scope.project.id=b9d94f6a68f7470d9494f112175620dc&role.id=817d6837cd044890b41cf16b303a092b => generated 259 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 03:08:32.187627 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ebb218-f293-4c7c-b136-24e9de9cdfbb None None] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:08:32.187907 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ebb218-f293-4c7c-b136-24e9de9cdfbb None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:08:32.188175 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ebb218-f293-4c7c-b136-24e9de9cdfbb None None] PATH_INFO: `/` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:08:32.189928 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 383/769] 10.222.0.18 () {54 vars in 927 bytes} [Tue Sep 13 03:08:32 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 03:08:32.197129 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3620ee12-d058-4b8e-979b-dc5e62b93361 None None] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:08:32.197447 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3620ee12-d058-4b8e-979b-dc5e62b93361 None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:08:32.197692 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3620ee12-d058-4b8e-979b-dc5e62b93361 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:08:32.220757 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.auth.core [None req-3620ee12-d058-4b8e-979b-dc5e62b93361 None None] MFA Rules not processed for user `49f280ab086b4610855c4ffa378517d9`. Rule list: `[]` (Enabled: `True`). {{(pid=81400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 03:08:32.221968 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-3620ee12-d058-4b8e-979b-dc5e62b93361 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:08:32.297869 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 387/770] 10.222.0.18 () {58 vars in 1024 bytes} [Tue Sep 13 03:08:32 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 13 03:08:32.432310 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22ebb218-f293-4c7c-b136-24e9de9cdfbb None None] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:08:32.505277 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80ad77ce-4c44-4f7a-a7d3-285ed2a4b337 None None] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:08:32.505605 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80ad77ce-4c44-4f7a-a7d3-285ed2a4b337 None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:08:32.505821 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80ad77ce-4c44-4f7a-a7d3-285ed2a4b337 None None] PATH_INFO: `/` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:08:32.507455 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 388/771] 10.222.0.18 () {54 vars in 927 bytes} [Tue Sep 13 03:08:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 03:08:32.512216 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa9fa76a-76c1-4e96-8488-b09344d6938e None None] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:08:32.512495 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa9fa76a-76c1-4e96-8488-b09344d6938e None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:08:32.512768 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa9fa76a-76c1-4e96-8488-b09344d6938e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:08:32.531440 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.auth.core [None req-aa9fa76a-76c1-4e96-8488-b09344d6938e None None] MFA Rules not processed for user `49f280ab086b4610855c4ffa378517d9`. Rule list: `[]` (Enabled: `True`). {{(pid=81400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 03:08:32.532618 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-aa9fa76a-76c1-4e96-8488-b09344d6938e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:08:32.550881 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:08:32.580270 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22ebb218-f293-4c7c-b136-24e9de9cdfbb None None] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:08:32.606901 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 389/772] 10.222.0.18 () {58 vars in 1024 bytes} [Tue Sep 13 03:08:32 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 13 03:08:32.655845 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61e62d79-03b1-4626-9836-bb1e7d36cd69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:08:32.656478 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61e62d79-03b1-4626-9836-bb1e7d36cd69 None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:08:32.656748 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61e62d79-03b1-4626-9836-bb1e7d36cd69 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:08:32.657011 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61e62d79-03b1-4626-9836-bb1e7d36cd69 None admin] PATH_INFO: `/v3/services/network` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:08:32.657581 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61e62d79-03b1-4626-9836-bb1e7d36cd69 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:08:32.658955 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61e62d79-03b1-4626-9836-bb1e7d36cd69 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:08:32.671927 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application [None req-61e62d79-03b1-4626-9836-bb1e7d36cd69 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Sep 13 03:08:32.671927 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 03:08:32.671927 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 13 03:08:32.671927 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 13 03:08:32.671927 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 13 03:08:32.671927 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 13 03:08:32.671927 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 13 03:08:32.671927 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 13 03:08:32.671927 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Sep 13 03:08:32.671927 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Sep 13 03:08:32.671927 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 13 03:08:32.671927 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Sep 13 03:08:32.671927 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 03:08:32.671927 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 03:08:32.671927 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 03:08:32.671927 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 03:08:32.671927 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 03:08:32.672921 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 03:08:32.672921 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 03:08:32.672921 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 03:08:32.672921 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 03:08:32.672921 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 03:08:32.672921 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 03:08:32.672921 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 13 03:08:32.672921 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 13 03:08:32.672921 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 13 03:08:32.672921 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 13 03:08:32.672921 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 03:08:32.672921 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 03:08:32.672921 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 03:08:32.672921 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 03:08:32.672921 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 03:08:32.672921 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 03:08:32.673770 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 03:08:32.673770 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application with Lock( Sep 13 03:08:32.673770 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 03:08:32.673770 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._enter() Sep 13 03:08:32.673770 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 03:08:32.673770 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 03:08:32.673770 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 03:08:32.673770 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.creator() Sep 13 03:08:32.673770 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 03:08:32.673770 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application created_value = creator( Sep 13 03:08:32.673770 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 13 03:08:32.673770 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 13 03:08:32.673770 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Sep 13 03:08:32.673770 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Sep 13 03:08:32.673770 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 384/773] 10.222.0.18 () {56 vars in 1111 bytes} [Tue Sep 13 03:08:32 2022] GET /identity/v3/services/network => generated 88 bytes in 241 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 03:08:32.678914 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa9fa76a-76c1-4e96-8488-b09344d6938e None None] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:08:32.684531 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61e62d79-03b1-4626-9836-bb1e7d36cd69 None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:08:32.746439 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:08:32.816080 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa9fa76a-76c1-4e96-8488-b09344d6938e None None] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:08:32.857730 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79fa24d1-d9e0-4ef0-bc92-ca3a7f461112 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:08:32.860064 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79fa24d1-d9e0-4ef0-bc92-ca3a7f461112 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:08:32.862463 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79fa24d1-d9e0-4ef0-bc92-ca3a7f461112 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:08:32.862777 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79fa24d1-d9e0-4ef0-bc92-ca3a7f461112 None admin] PATH_INFO: `/v3/services` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:08:32.863420 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79fa24d1-d9e0-4ef0-bc92-ca3a7f461112 None admin] RBAC: Adding query filter params (name=network) {{(pid=81400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 03:08:32.863722 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79fa24d1-d9e0-4ef0-bc92-ca3a7f461112 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:08:32.865108 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79fa24d1-d9e0-4ef0-bc92-ca3a7f461112 None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:08:32.880425 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 390/774] 10.222.0.18 () {56 vars in 1112 bytes} [Tue Sep 13 03:08:32 2022] GET /identity/v3/services?name=network => generated 124 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 03:08:32.882586 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61e62d79-03b1-4626-9836-bb1e7d36cd69 None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:08:32.884941 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79fa24d1-d9e0-4ef0-bc92-ca3a7f461112 None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:08:32.938713 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4521fe7-9f14-4ec9-be6e-37485df14fe3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:08:32.939274 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4521fe7-9f14-4ec9-be6e-37485df14fe3 None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:08:32.939551 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4521fe7-9f14-4ec9-be6e-37485df14fe3 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:08:32.939819 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4521fe7-9f14-4ec9-be6e-37485df14fe3 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:08:32.940407 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4521fe7-9f14-4ec9-be6e-37485df14fe3 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:08:32.941800 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4521fe7-9f14-4ec9-be6e-37485df14fe3 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:08:32.954312 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application [None req-a4521fe7-9f14-4ec9-be6e-37485df14fe3 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 13 03:08:32.954312 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 03:08:32.954312 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 13 03:08:32.954312 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 13 03:08:32.954312 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 13 03:08:32.954312 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 13 03:08:32.954312 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 13 03:08:32.954312 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 13 03:08:32.954312 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 13 03:08:32.954312 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Sep 13 03:08:32.954312 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 13 03:08:32.954312 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Sep 13 03:08:32.954312 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 03:08:32.954312 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 03:08:32.954312 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 03:08:32.954312 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 03:08:32.954312 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 03:08:32.955429 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 03:08:32.955429 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 03:08:32.955429 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 03:08:32.955429 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 03:08:32.955429 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 03:08:32.955429 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 03:08:32.955429 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 13 03:08:32.955429 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 13 03:08:32.955429 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 13 03:08:32.955429 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 13 03:08:32.955429 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 03:08:32.955429 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 03:08:32.955429 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 03:08:32.955429 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 03:08:32.955429 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 03:08:32.955429 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 03:08:32.956279 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 03:08:32.956279 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application with Lock( Sep 13 03:08:32.956279 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 03:08:32.956279 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._enter() Sep 13 03:08:32.956279 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 03:08:32.956279 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 03:08:32.956279 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 03:08:32.956279 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.creator() Sep 13 03:08:32.956279 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 03:08:32.956279 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application created_value = creator( Sep 13 03:08:32.956279 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 13 03:08:32.956279 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 13 03:08:32.956279 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 13 03:08:32.956279 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Sep 13 03:08:32.959174 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 385/775] 10.222.0.18 () {56 vars in 1129 bytes} [Tue Sep 13 03:08:32 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 275 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 13 03:08:32.960361 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4521fe7-9f14-4ec9-be6e-37485df14fe3 None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:08:32.998955 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:08:33.032760 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79fa24d1-d9e0-4ef0-bc92-ca3a7f461112 None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:08:33.036444 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:08:33.087591 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc6b0a0c-6c7f-41a8-b0ec-a293adc93acc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:08:33.088718 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc6b0a0c-6c7f-41a8-b0ec-a293adc93acc None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:08:33.089634 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc6b0a0c-6c7f-41a8-b0ec-a293adc93acc None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:08:33.090347 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc6b0a0c-6c7f-41a8-b0ec-a293adc93acc None admin] PATH_INFO: `/v3/services` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:08:33.091204 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc6b0a0c-6c7f-41a8-b0ec-a293adc93acc None admin] RBAC: Adding query filter params (type=network) {{(pid=81400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 03:08:33.091701 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc6b0a0c-6c7f-41a8-b0ec-a293adc93acc None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:08:33.091926 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4521fe7-9f14-4ec9-be6e-37485df14fe3 None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:08:33.095461 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc6b0a0c-6c7f-41a8-b0ec-a293adc93acc None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:08:33.101478 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 391/776] 10.222.0.18 () {56 vars in 1112 bytes} [Tue Sep 13 03:08:32 2022] GET /identity/v3/services?type=network => generated 350 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 03:08:33.102981 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc6b0a0c-6c7f-41a8-b0ec-a293adc93acc None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:08:33.128574 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c33ceaa-c3e4-4bc2-a8b0-15ff2e12c5e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:08:33.129206 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c33ceaa-c3e4-4bc2-a8b0-15ff2e12c5e7 None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:08:33.129477 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c33ceaa-c3e4-4bc2-a8b0-15ff2e12c5e7 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:08:33.129746 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c33ceaa-c3e4-4bc2-a8b0-15ff2e12c5e7 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:08:33.141394 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c33ceaa-c3e4-4bc2-a8b0-15ff2e12c5e7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 03:08:33.148125 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application [None req-2c33ceaa-c3e4-4bc2-a8b0-15ff2e12c5e7 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 13 03:08:33.148125 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 03:08:33.148125 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 03:08:33.148125 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 03:08:33.148125 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 03:08:33.148125 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 03:08:33.148125 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 03:08:33.148125 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 03:08:33.148125 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 03:08:33.148125 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 03:08:33.148125 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 03:08:33.148125 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 03:08:33.148125 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 03:08:33.148125 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 03:08:33.148125 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 03:08:33.148125 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 03:08:33.149096 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 03:08:33.149096 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 03:08:33.149096 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 03:08:33.149096 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 03:08:33.149096 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 03:08:33.149096 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 03:08:33.149096 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 03:08:33.149096 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 03:08:33.149096 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 03:08:33.149096 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 03:08:33.149096 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 03:08:33.149096 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application with Lock( Sep 13 03:08:33.149096 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 03:08:33.149096 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._enter() Sep 13 03:08:33.149096 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 03:08:33.149096 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 03:08:33.149938 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 03:08:33.149938 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.creator() Sep 13 03:08:33.149938 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 03:08:33.149938 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application created_value = creator( Sep 13 03:08:33.149938 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 03:08:33.149938 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 03:08:33.149938 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 03:08:33.149938 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 03:08:33.149938 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 03:08:33.149938 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 03:08:33.149938 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 13 03:08:33.149938 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Sep 13 03:08:33.153586 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 386/777] 10.222.0.18 () {56 vars in 1096 bytes} [Tue Sep 13 03:08:32 2022] GET /identity/v3/roles/admin => generated 83 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 03:08:33.154428 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c33ceaa-c3e4-4bc2-a8b0-15ff2e12c5e7 None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:08:33.238592 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc6b0a0c-6c7f-41a8-b0ec-a293adc93acc None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:08:33.244339 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:08:33.274723 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c33ceaa-c3e4-4bc2-a8b0-15ff2e12c5e7 None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:08:33.289520 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcac3e2f-7f5c-4ab8-aa6d-11ccc814d320 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:08:33.290627 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcac3e2f-7f5c-4ab8-aa6d-11ccc814d320 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:08:33.291060 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcac3e2f-7f5c-4ab8-aa6d-11ccc814d320 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:08:33.291463 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcac3e2f-7f5c-4ab8-aa6d-11ccc814d320 None admin] PATH_INFO: `/v3/services` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:08:33.292098 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcac3e2f-7f5c-4ab8-aa6d-11ccc814d320 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=81400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 03:08:33.292652 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcac3e2f-7f5c-4ab8-aa6d-11ccc814d320 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:08:33.294427 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcac3e2f-7f5c-4ab8-aa6d-11ccc814d320 None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:08:33.302720 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 392/778] 10.222.0.18 () {56 vars in 1124 bytes} [Tue Sep 13 03:08:33 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 03:08:33.303665 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcac3e2f-7f5c-4ab8-aa6d-11ccc814d320 None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:08:33.325269 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8d38296-c7c5-4ddd-9566-567532034bf7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:08:33.326140 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d38296-c7c5-4ddd-9566-567532034bf7 None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:08:33.326425 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d38296-c7c5-4ddd-9566-567532034bf7 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:08:33.327352 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d38296-c7c5-4ddd-9566-567532034bf7 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:08:33.328425 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8d38296-c7c5-4ddd-9566-567532034bf7 None admin] RBAC: Adding query filter params (interface=public, service_id=2dccd7ec197d494e83e5eb4a6ede334e, region_id=RegionOne) {{(pid=81399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 03:08:33.328736 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8d38296-c7c5-4ddd-9566-567532034bf7 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:08:33.330110 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8d38296-c7c5-4ddd-9566-567532034bf7 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:08:33.339910 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:08:33.341219 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 387/779] 10.222.0.18 () {56 vars in 1251 bytes} [Tue Sep 13 03:08:33 2022] GET /identity/v3/endpoints?service_id=2dccd7ec197d494e83e5eb4a6ede334e&interface=public®ion_id=RegionOne => generated 194 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 03:08:33.342279 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8d38296-c7c5-4ddd-9566-567532034bf7 None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:08:33.414803 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcac3e2f-7f5c-4ab8-aa6d-11ccc814d320 None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:08:33.449974 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e7043bb-976b-4845-99fe-277928ca8de4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:08:33.450578 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e7043bb-976b-4845-99fe-277928ca8de4 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:08:33.450998 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e7043bb-976b-4845-99fe-277928ca8de4 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:08:33.451278 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e7043bb-976b-4845-99fe-277928ca8de4 None admin] PATH_INFO: `/v3/roles` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:08:33.452469 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e7043bb-976b-4845-99fe-277928ca8de4 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 03:08:33.452766 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e7043bb-976b-4845-99fe-277928ca8de4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:08:33.454123 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e7043bb-976b-4845-99fe-277928ca8de4 None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:08:33.458922 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8d38296-c7c5-4ddd-9566-567532034bf7 None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:08:33.467469 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 393/780] 10.222.0.18 () {56 vars in 1099 bytes} [Tue Sep 13 03:08:33 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 03:08:33.468430 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e7043bb-976b-4845-99fe-277928ca8de4 None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:08:33.496727 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-534cd9a7-7163-4b9d-b426-db8ef5dc797a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:08:33.497598 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-534cd9a7-7163-4b9d-b426-db8ef5dc797a None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:08:33.497885 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-534cd9a7-7163-4b9d-b426-db8ef5dc797a None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:08:33.498947 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-534cd9a7-7163-4b9d-b426-db8ef5dc797a None admin] PATH_INFO: `/v3/services` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:08:33.499583 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-534cd9a7-7163-4b9d-b426-db8ef5dc797a None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=81399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 03:08:33.499885 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-534cd9a7-7163-4b9d-b426-db8ef5dc797a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:08:33.501245 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-534cd9a7-7163-4b9d-b426-db8ef5dc797a None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:08:33.507320 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 388/781] 10.222.0.18 () {56 vars in 1124 bytes} [Tue Sep 13 03:08:33 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 03:08:33.508096 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-534cd9a7-7163-4b9d-b426-db8ef5dc797a None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:08:33.512346 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:08:33.570182 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:08:33.578469 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e7043bb-976b-4845-99fe-277928ca8de4 None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:08:33.616510 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42e39935-d1c0-40cf-99b8-1b72f6eae4d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:08:33.617490 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42e39935-d1c0-40cf-99b8-1b72f6eae4d2 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:08:33.617789 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42e39935-d1c0-40cf-99b8-1b72f6eae4d2 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:08:33.618061 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42e39935-d1c0-40cf-99b8-1b72f6eae4d2 None admin] PATH_INFO: `/v3/services` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:08:33.618944 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42e39935-d1c0-40cf-99b8-1b72f6eae4d2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:08:33.620403 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42e39935-d1c0-40cf-99b8-1b72f6eae4d2 None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:08:33.620873 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-534cd9a7-7163-4b9d-b426-db8ef5dc797a None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:08:33.634653 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 394/782] 10.222.0.18 () {56 vars in 1087 bytes} [Tue Sep 13 03:08:33 2022] GET /identity/v3/services => generated 772 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 03:08:33.634653 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42e39935-d1c0-40cf-99b8-1b72f6eae4d2 None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:08:33.701338 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d090bc6-e8b4-4501-96f3-b744f7acba4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:08:33.703497 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d090bc6-e8b4-4501-96f3-b744f7acba4b None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:08:33.703774 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d090bc6-e8b4-4501-96f3-b744f7acba4b None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:08:33.704041 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d090bc6-e8b4-4501-96f3-b744f7acba4b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:08:33.732338 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d090bc6-e8b4-4501-96f3-b744f7acba4b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:08:33.734503 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d090bc6-e8b4-4501-96f3-b744f7acba4b None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:08:33.772938 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application [None req-5d090bc6-e8b4-4501-96f3-b744f7acba4b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 03:08:33.772938 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 03:08:33.772938 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 03:08:33.772938 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 03:08:33.772938 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 03:08:33.772938 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 03:08:33.772938 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 03:08:33.772938 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 03:08:33.772938 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 03:08:33.772938 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 03:08:33.772938 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 03:08:33.772938 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 03:08:33.772938 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 13 03:08:33.772938 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 13 03:08:33.772938 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 13 03:08:33.772938 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 13 03:08:33.773940 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 03:08:33.773940 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 03:08:33.773940 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 03:08:33.773940 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 03:08:33.773940 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 03:08:33.773940 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 03:08:33.773940 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 03:08:33.773940 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application with Lock( Sep 13 03:08:33.773940 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 03:08:33.773940 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._enter() Sep 13 03:08:33.773940 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 03:08:33.773940 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 03:08:33.773940 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 03:08:33.773940 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.creator() Sep 13 03:08:33.773940 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 03:08:33.773940 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application created_value = creator( Sep 13 03:08:33.773940 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 13 03:08:33.774893 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 13 03:08:33.774893 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 03:08:33.774893 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Sep 13 03:08:33.774893 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 389/783] 10.222.0.18 () {56 vars in 1108 bytes} [Tue Sep 13 03:08:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 266 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 03:08:33.781703 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d090bc6-e8b4-4501-96f3-b744f7acba4b None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:08:33.826013 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42e39935-d1c0-40cf-99b8-1b72f6eae4d2 None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:08:33.872525 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2de21e75-2a57-43c6-9e65-b4ad29e6a25e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:08:33.873154 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2de21e75-2a57-43c6-9e65-b4ad29e6a25e None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:08:33.873388 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2de21e75-2a57-43c6-9e65-b4ad29e6a25e None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:08:33.873692 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2de21e75-2a57-43c6-9e65-b4ad29e6a25e None admin] PATH_INFO: `/v3/endpoints` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:08:33.874757 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2de21e75-2a57-43c6-9e65-b4ad29e6a25e None admin] RBAC: Adding query filter params (interface=public, service_id=a8a130effe8d4e9e93d58d586019bcf8, region_id=RegionOne) {{(pid=81400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 03:08:33.874866 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2de21e75-2a57-43c6-9e65-b4ad29e6a25e None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:08:33.876579 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2de21e75-2a57-43c6-9e65-b4ad29e6a25e None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:08:33.877994 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:08:33.888654 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 395/784] 10.222.0.18 () {56 vars in 1251 bytes} [Tue Sep 13 03:08:33 2022] GET /identity/v3/endpoints?service_id=a8a130effe8d4e9e93d58d586019bcf8&interface=public®ion_id=RegionOne => generated 194 bytes in 260 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 03:08:33.895539 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2de21e75-2a57-43c6-9e65-b4ad29e6a25e None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:08:33.935853 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:08:33.938765 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d090bc6-e8b4-4501-96f3-b744f7acba4b None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:08:33.979791 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34eaee79-50e2-4f45-bab1-3188b875fe43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:08:33.980420 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34eaee79-50e2-4f45-bab1-3188b875fe43 None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:08:33.980699 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34eaee79-50e2-4f45-bab1-3188b875fe43 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:08:33.980973 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34eaee79-50e2-4f45-bab1-3188b875fe43 None admin] PATH_INFO: `/v3/domains` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:08:33.981610 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34eaee79-50e2-4f45-bab1-3188b875fe43 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 03:08:33.981911 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34eaee79-50e2-4f45-bab1-3188b875fe43 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:08:33.983417 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34eaee79-50e2-4f45-bab1-3188b875fe43 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:08:33.995922 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 390/785] 10.222.0.18 () {56 vars in 1109 bytes} [Tue Sep 13 03:08:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 03:08:34.007006 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34eaee79-50e2-4f45-bab1-3188b875fe43 None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:08:34.040958 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2de21e75-2a57-43c6-9e65-b4ad29e6a25e None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:08:34.082475 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-176e708a-f5ee-4d45-b922-c599a916b931 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:08:34.083355 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-176e708a-f5ee-4d45-b922-c599a916b931 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:08:34.083799 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-176e708a-f5ee-4d45-b922-c599a916b931 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:08:34.084239 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-176e708a-f5ee-4d45-b922-c599a916b931 None admin] PATH_INFO: `/v3/services` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:08:34.084977 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-176e708a-f5ee-4d45-b922-c599a916b931 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:08:34.086522 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-176e708a-f5ee-4d45-b922-c599a916b931 None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:08:34.092297 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 396/786] 10.222.0.18 () {56 vars in 1087 bytes} [Tue Sep 13 03:08:33 2022] GET /identity/v3/services => generated 772 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 03:08:34.184397 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34eaee79-50e2-4f45-bab1-3188b875fe43 None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:08:34.205030 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:08:34.237911 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d58a60b-943e-435f-a83f-f4e94f03bbf6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:08:34.238570 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d58a60b-943e-435f-a83f-f4e94f03bbf6 None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:08:34.238877 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d58a60b-943e-435f-a83f-f4e94f03bbf6 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:08:34.239151 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d58a60b-943e-435f-a83f-f4e94f03bbf6 None admin] PATH_INFO: `/v3/users/swift` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:08:34.247509 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d58a60b-943e-435f-a83f-f4e94f03bbf6 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:08:34.249530 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d58a60b-943e-435f-a83f-f4e94f03bbf6 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:08:34.262490 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application [None req-6d58a60b-943e-435f-a83f-f4e94f03bbf6 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Sep 13 03:08:34.262490 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 03:08:34.262490 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 03:08:34.262490 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 03:08:34.262490 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 03:08:34.262490 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 03:08:34.262490 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 03:08:34.262490 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 03:08:34.262490 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 03:08:34.262490 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 03:08:34.262490 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 03:08:34.262490 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 03:08:34.262490 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 13 03:08:34.262490 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 13 03:08:34.262490 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 13 03:08:34.262490 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 13 03:08:34.263562 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 03:08:34.263562 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 03:08:34.263562 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 13 03:08:34.263562 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 13 03:08:34.263562 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 13 03:08:34.263562 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 13 03:08:34.263562 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 03:08:34.263562 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 03:08:34.263562 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 03:08:34.263562 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 03:08:34.263562 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 03:08:34.263562 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application with Lock( Sep 13 03:08:34.263562 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 03:08:34.263562 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._enter() Sep 13 03:08:34.263562 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 03:08:34.263562 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 03:08:34.263562 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 03:08:34.263562 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.creator() Sep 13 03:08:34.265094 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 03:08:34.265094 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application created_value = creator( Sep 13 03:08:34.265094 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 13 03:08:34.265094 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 13 03:08:34.265094 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 13 03:08:34.265094 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 13 03:08:34.265094 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 13 03:08:34.265094 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 13 03:08:34.265094 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Sep 13 03:08:34.265094 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Sep 13 03:08:34.265094 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 391/787] 10.222.0.18 () {56 vars in 1096 bytes} [Tue Sep 13 03:08:34 2022] GET /identity/v3/users/swift => generated 83 bytes in 258 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 03:08:34.272290 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-176e708a-f5ee-4d45-b922-c599a916b931 None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:08:34.294006 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7bbca02-9258-489f-a4df-35e2324e9600 None None] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:08:34.294006 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7bbca02-9258-489f-a4df-35e2324e9600 None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:08:34.294248 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7bbca02-9258-489f-a4df-35e2324e9600 None None] PATH_INFO: `/` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:08:34.296683 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 392/788] 10.222.0.18 () {54 vars in 927 bytes} [Tue Sep 13 03:08:34 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 03:08:34.302709 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef873125-e953-4315-aa6d-7ef6a4a367cf None None] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:08:34.302709 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef873125-e953-4315-aa6d-7ef6a4a367cf None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:08:34.302709 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef873125-e953-4315-aa6d-7ef6a4a367cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:08:34.322227 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.auth.core [None req-ef873125-e953-4315-aa6d-7ef6a4a367cf None None] MFA Rules not processed for user `49f280ab086b4610855c4ffa378517d9`. Rule list: `[]` (Enabled: `True`). {{(pid=81399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 03:08:34.323471 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-ef873125-e953-4315-aa6d-7ef6a4a367cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:08:34.401059 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 393/789] 10.222.0.18 () {58 vars in 1024 bytes} [Tue Sep 13 03:08:34 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 13 03:08:34.413862 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:08:34.430726 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-176e708a-f5ee-4d45-b922-c599a916b931 None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:08:34.469932 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0128d5ac-6168-4652-a56a-e89e0b1cb53e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:08:34.470842 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0128d5ac-6168-4652-a56a-e89e0b1cb53e None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:08:34.471259 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0128d5ac-6168-4652-a56a-e89e0b1cb53e None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:08:34.471715 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0128d5ac-6168-4652-a56a-e89e0b1cb53e None admin] PATH_INFO: `/v3/users` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:08:34.472572 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0128d5ac-6168-4652-a56a-e89e0b1cb53e None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 03:08:34.473014 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0128d5ac-6168-4652-a56a-e89e0b1cb53e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:08:34.474588 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0128d5ac-6168-4652-a56a-e89e0b1cb53e None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:08:34.493399 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 397/790] 10.222.0.18 () {56 vars in 1135 bytes} [Tue Sep 13 03:08:34 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 03:08:34.499780 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef873125-e953-4315-aa6d-7ef6a4a367cf None None] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:08:34.603082 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef873125-e953-4315-aa6d-7ef6a4a367cf None None] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:08:34.611411 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:08:34.639675 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bfce37c-d480-4370-adcf-5a6b3ba3d24c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:08:34.640671 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bfce37c-d480-4370-adcf-5a6b3ba3d24c None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:08:34.641094 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bfce37c-d480-4370-adcf-5a6b3ba3d24c None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:08:34.641513 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bfce37c-d480-4370-adcf-5a6b3ba3d24c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:08:34.653611 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bfce37c-d480-4370-adcf-5a6b3ba3d24c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:08:34.655927 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bfce37c-d480-4370-adcf-5a6b3ba3d24c None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:08:34.673803 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application [None req-1bfce37c-d480-4370-adcf-5a6b3ba3d24c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 03:08:34.673803 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 03:08:34.673803 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 03:08:34.673803 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 03:08:34.673803 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 03:08:34.673803 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 03:08:34.673803 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 03:08:34.673803 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 03:08:34.673803 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 03:08:34.673803 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 03:08:34.673803 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 03:08:34.673803 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 03:08:34.673803 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 13 03:08:34.673803 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 13 03:08:34.673803 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 13 03:08:34.673803 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 13 03:08:34.675026 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 03:08:34.675026 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 03:08:34.675026 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 03:08:34.675026 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 03:08:34.675026 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 03:08:34.675026 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 03:08:34.675026 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 03:08:34.675026 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application with Lock( Sep 13 03:08:34.675026 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 03:08:34.675026 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._enter() Sep 13 03:08:34.675026 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 03:08:34.675026 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 03:08:34.675026 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 03:08:34.675026 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.creator() Sep 13 03:08:34.675026 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 03:08:34.675026 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application created_value = creator( Sep 13 03:08:34.675026 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 13 03:08:34.676135 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 13 03:08:34.676135 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 03:08:34.676135 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Sep 13 03:08:34.677851 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 394/791] 10.222.0.18 () {56 vars in 1108 bytes} [Tue Sep 13 03:08:34 2022] GET /identity/v3/domains/Default => generated 87 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 03:08:34.683962 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0128d5ac-6168-4652-a56a-e89e0b1cb53e None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:08:34.786749 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0128d5ac-6168-4652-a56a-e89e0b1cb53e None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:08:34.803435 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bfce37c-d480-4370-adcf-5a6b3ba3d24c None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:08:34.823481 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84b12ee6-471c-4e18-9bfc-ba41d7ce4f47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:08:34.824346 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b12ee6-471c-4e18-9bfc-ba41d7ce4f47 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:08:34.824766 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b12ee6-471c-4e18-9bfc-ba41d7ce4f47 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:08:34.825189 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b12ee6-471c-4e18-9bfc-ba41d7ce4f47 None admin] PATH_INFO: `/v3/domains` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:08:34.825965 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84b12ee6-471c-4e18-9bfc-ba41d7ce4f47 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 03:08:34.826405 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84b12ee6-471c-4e18-9bfc-ba41d7ce4f47 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:08:34.828104 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84b12ee6-471c-4e18-9bfc-ba41d7ce4f47 None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:08:34.840128 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 398/792] 10.222.0.18 () {56 vars in 1109 bytes} [Tue Sep 13 03:08:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 03:08:34.846234 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84b12ee6-471c-4e18-9bfc-ba41d7ce4f47 None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:08:34.855561 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:08:34.955528 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bfce37c-d480-4370-adcf-5a6b3ba3d24c None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:08:34.958628 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84b12ee6-471c-4e18-9bfc-ba41d7ce4f47 None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:08:34.995420 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fbe2cf4-ff18-4a93-9ad8-2f95e1dbe466 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:08:34.996328 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fbe2cf4-ff18-4a93-9ad8-2f95e1dbe466 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:08:34.996678 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fbe2cf4-ff18-4a93-9ad8-2f95e1dbe466 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:08:34.996934 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36804a0e-1d3f-4761-a687-9022c76c999c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:08:34.997071 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fbe2cf4-ff18-4a93-9ad8-2f95e1dbe466 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:08:34.997677 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36804a0e-1d3f-4761-a687-9022c76c999c None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:08:34.998016 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36804a0e-1d3f-4761-a687-9022c76c999c None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:08:34.998344 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36804a0e-1d3f-4761-a687-9022c76c999c None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:08:35.004383 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fbe2cf4-ff18-4a93-9ad8-2f95e1dbe466 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:08:35.006095 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fbe2cf4-ff18-4a93-9ad8-2f95e1dbe466 None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:08:35.010064 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36804a0e-1d3f-4761-a687-9022c76c999c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 03:08:35.016697 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application [None req-36804a0e-1d3f-4761-a687-9022c76c999c None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 13 03:08:35.016697 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 03:08:35.016697 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 03:08:35.016697 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 03:08:35.016697 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 03:08:35.016697 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 03:08:35.016697 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 03:08:35.016697 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 03:08:35.016697 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 03:08:35.016697 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 03:08:35.016697 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 03:08:35.016697 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 03:08:35.016697 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 03:08:35.016697 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 03:08:35.016697 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 03:08:35.016697 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 03:08:35.017891 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 03:08:35.017891 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 03:08:35.017891 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 03:08:35.017891 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 03:08:35.017891 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 03:08:35.017891 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 03:08:35.017891 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 03:08:35.017891 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 03:08:35.017891 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 03:08:35.017891 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 03:08:35.017891 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 03:08:35.017891 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application with Lock( Sep 13 03:08:35.017891 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 03:08:35.017891 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._enter() Sep 13 03:08:35.017891 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 03:08:35.017891 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 03:08:35.019834 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 03:08:35.019834 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.creator() Sep 13 03:08:35.019834 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 03:08:35.019834 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application created_value = creator( Sep 13 03:08:35.019834 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 03:08:35.019834 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 03:08:35.019834 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 03:08:35.019834 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 03:08:35.019834 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 03:08:35.019834 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 03:08:35.019834 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 13 03:08:35.019834 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Sep 13 03:08:35.019834 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 395/793] 10.222.0.18 () {56 vars in 1096 bytes} [Tue Sep 13 03:08:34 2022] GET /identity/v3/roles/admin => generated 83 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 13 03:08:35.019834 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5925117b-d5ec-4199-bf3c-63ab37c84dca None None] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:08:35.021465 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5925117b-d5ec-4199-bf3c-63ab37c84dca None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:08:35.021465 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5925117b-d5ec-4199-bf3c-63ab37c84dca None None] PATH_INFO: `/` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:08:35.021623 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application [None req-7fbe2cf4-ff18-4a93-9ad8-2f95e1dbe466 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 13 03:08:35.021623 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 03:08:35.021623 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 03:08:35.021623 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 03:08:35.021623 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 03:08:35.021623 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 03:08:35.021623 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 03:08:35.021623 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 03:08:35.021623 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 03:08:35.021623 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 03:08:35.021623 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 03:08:35.021623 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 03:08:35.021623 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 13 03:08:35.021623 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 13 03:08:35.021623 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 13 03:08:35.022632 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 13 03:08:35.022632 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 03:08:35.022632 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 03:08:35.022632 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 03:08:35.022632 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 03:08:35.022632 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 03:08:35.022632 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 03:08:35.022632 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 03:08:35.022632 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application with Lock( Sep 13 03:08:35.022632 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 03:08:35.022632 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._enter() Sep 13 03:08:35.022632 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 03:08:35.022632 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 03:08:35.022632 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 03:08:35.022632 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.creator() Sep 13 03:08:35.022632 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 03:08:35.022632 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application created_value = creator( Sep 13 03:08:35.023536 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 13 03:08:35.023536 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 13 03:08:35.023536 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 13 03:08:35.023536 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 13 03:08:35.023536 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 13 03:08:35.023536 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 13 03:08:35.023536 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 13 03:08:35.023536 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Sep 13 03:08:35.023964 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 396/794] 10.222.0.18 () {54 vars in 927 bytes} [Tue Sep 13 03:08:35 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 03:08:35.030680 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 399/795] 10.222.0.18 () {56 vars in 1111 bytes} [Tue Sep 13 03:08:34 2022] GET /identity/v3/projects/service => generated 88 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 13 03:08:35.030813 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5925117b-d5ec-4199-bf3c-63ab37c84dca None None] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:08:35.030906 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c64a9acb-6a3d-4b6d-94a6-44da1bde62dd None None] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:08:35.030906 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c64a9acb-6a3d-4b6d-94a6-44da1bde62dd None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:08:35.030906 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c64a9acb-6a3d-4b6d-94a6-44da1bde62dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:08:35.049824 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.auth.core [None req-c64a9acb-6a3d-4b6d-94a6-44da1bde62dd None None] MFA Rules not processed for user `49f280ab086b4610855c4ffa378517d9`. Rule list: `[]` (Enabled: `True`). {{(pid=81400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 03:08:35.051138 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-c64a9acb-6a3d-4b6d-94a6-44da1bde62dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:08:35.064529 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:08:35.120596 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:08:35.138753 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 400/796] 10.222.0.18 () {58 vars in 1024 bytes} [Tue Sep 13 03:08:35 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 13 03:08:35.140325 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c64a9acb-6a3d-4b6d-94a6-44da1bde62dd None None] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:08:35.172072 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5925117b-d5ec-4199-bf3c-63ab37c84dca None None] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:08:35.209906 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14e29ca6-c9bd-47c9-98bb-4d367f0145a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:08:35.210935 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14e29ca6-c9bd-47c9-98bb-4d367f0145a4 None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:08:35.211447 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14e29ca6-c9bd-47c9-98bb-4d367f0145a4 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:08:35.211918 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14e29ca6-c9bd-47c9-98bb-4d367f0145a4 None admin] PATH_INFO: `/v3/roles` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:08:35.212774 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14e29ca6-c9bd-47c9-98bb-4d367f0145a4 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 03:08:35.213276 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14e29ca6-c9bd-47c9-98bb-4d367f0145a4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:08:35.214925 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14e29ca6-c9bd-47c9-98bb-4d367f0145a4 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:08:35.226389 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 397/797] 10.222.0.18 () {56 vars in 1099 bytes} [Tue Sep 13 03:08:35 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 03:08:35.231964 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14e29ca6-c9bd-47c9-98bb-4d367f0145a4 None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:08:35.322915 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:08:35.347925 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14e29ca6-c9bd-47c9-98bb-4d367f0145a4 None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:08:35.386509 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ca353a5-8e22-4fa0-8ec7-5b746240d9f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:08:35.387558 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ca353a5-8e22-4fa0-8ec7-5b746240d9f4 None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:08:35.388042 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ca353a5-8e22-4fa0-8ec7-5b746240d9f4 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:08:35.388518 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ca353a5-8e22-4fa0-8ec7-5b746240d9f4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:08:35.389813 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c64a9acb-6a3d-4b6d-94a6-44da1bde62dd None None] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:08:35.401563 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ca353a5-8e22-4fa0-8ec7-5b746240d9f4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:08:35.403313 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ca353a5-8e22-4fa0-8ec7-5b746240d9f4 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:08:35.420506 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application [None req-0ca353a5-8e22-4fa0-8ec7-5b746240d9f4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 03:08:35.420506 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 03:08:35.420506 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 03:08:35.420506 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 03:08:35.420506 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 03:08:35.420506 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 03:08:35.420506 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 03:08:35.420506 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 03:08:35.420506 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 03:08:35.420506 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 03:08:35.420506 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 03:08:35.420506 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 03:08:35.420506 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 13 03:08:35.420506 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 13 03:08:35.420506 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 13 03:08:35.420506 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 13 03:08:35.421479 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 03:08:35.421479 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 03:08:35.421479 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 03:08:35.421479 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 03:08:35.421479 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 03:08:35.421479 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 03:08:35.421479 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 03:08:35.421479 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application with Lock( Sep 13 03:08:35.421479 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 03:08:35.421479 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._enter() Sep 13 03:08:35.421479 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 03:08:35.421479 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 03:08:35.421479 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 03:08:35.421479 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.creator() Sep 13 03:08:35.421479 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 03:08:35.421479 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application created_value = creator( Sep 13 03:08:35.421479 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 13 03:08:35.422396 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 13 03:08:35.422396 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 03:08:35.422396 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Sep 13 03:08:35.424574 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 398/798] 10.222.0.18 () {56 vars in 1108 bytes} [Tue Sep 13 03:08:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 13 03:08:35.430066 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ca353a5-8e22-4fa0-8ec7-5b746240d9f4 None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:08:35.465641 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81d6fc9f-4309-4870-b245-75afc3f411cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:08:35.466571 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81d6fc9f-4309-4870-b245-75afc3f411cc None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:08:35.467318 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81d6fc9f-4309-4870-b245-75afc3f411cc None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:08:35.470209 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81d6fc9f-4309-4870-b245-75afc3f411cc None admin] PATH_INFO: `/v3/projects` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:08:35.471213 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81d6fc9f-4309-4870-b245-75afc3f411cc None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 03:08:35.471727 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81d6fc9f-4309-4870-b245-75afc3f411cc None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:08:35.473408 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81d6fc9f-4309-4870-b245-75afc3f411cc None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:08:35.486516 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 401/799] 10.222.0.18 () {56 vars in 1148 bytes} [Tue Sep 13 03:08:35 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 347 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 03:08:35.487585 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81d6fc9f-4309-4870-b245-75afc3f411cc None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:08:35.555073 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:08:35.562854 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ca353a5-8e22-4fa0-8ec7-5b746240d9f4 None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:08:35.593029 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:08:35.599536 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f32bbd8-df54-433b-a5aa-0d6859e5ec07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:08:35.600641 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f32bbd8-df54-433b-a5aa-0d6859e5ec07 None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:08:35.601243 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f32bbd8-df54-433b-a5aa-0d6859e5ec07 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:08:35.601553 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f32bbd8-df54-433b-a5aa-0d6859e5ec07 None admin] PATH_INFO: `/v3/domains` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:08:35.602671 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f32bbd8-df54-433b-a5aa-0d6859e5ec07 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 03:08:35.603325 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f32bbd8-df54-433b-a5aa-0d6859e5ec07 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:08:35.604705 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f32bbd8-df54-433b-a5aa-0d6859e5ec07 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:08:35.610882 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81d6fc9f-4309-4870-b245-75afc3f411cc None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:08:35.617436 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 399/800] 10.222.0.18 () {56 vars in 1109 bytes} [Tue Sep 13 03:08:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 03:08:35.618322 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f32bbd8-df54-433b-a5aa-0d6859e5ec07 None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:08:35.649454 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7bd14b6-d5c0-43a3-99fd-5073092605e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:08:35.650105 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7bd14b6-d5c0-43a3-99fd-5073092605e1 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:08:35.650389 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7bd14b6-d5c0-43a3-99fd-5073092605e1 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:08:35.650683 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7bd14b6-d5c0-43a3-99fd-5073092605e1 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:08:35.661714 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7bd14b6-d5c0-43a3-99fd-5073092605e1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 03:08:35.667650 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application [None req-d7bd14b6-d5c0-43a3-99fd-5073092605e1 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 13 03:08:35.667650 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 03:08:35.667650 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 03:08:35.667650 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 03:08:35.667650 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 03:08:35.667650 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 03:08:35.667650 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 03:08:35.667650 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 03:08:35.667650 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 03:08:35.667650 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 03:08:35.667650 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 03:08:35.667650 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 03:08:35.667650 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 03:08:35.667650 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 03:08:35.667650 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 03:08:35.667650 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 03:08:35.668581 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 03:08:35.668581 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 03:08:35.668581 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 03:08:35.668581 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 03:08:35.668581 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 03:08:35.668581 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 03:08:35.668581 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 03:08:35.668581 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 03:08:35.668581 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 03:08:35.668581 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 03:08:35.668581 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 03:08:35.668581 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application with Lock( Sep 13 03:08:35.668581 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 03:08:35.668581 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._enter() Sep 13 03:08:35.668581 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 03:08:35.668581 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 03:08:35.669422 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 03:08:35.669422 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.creator() Sep 13 03:08:35.669422 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 03:08:35.669422 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application created_value = creator( Sep 13 03:08:35.669422 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 03:08:35.669422 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 03:08:35.669422 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 03:08:35.669422 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 03:08:35.669422 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 03:08:35.669422 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 03:08:35.669422 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 13 03:08:35.669422 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Sep 13 03:08:35.673812 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 402/801] 10.222.0.18 () {56 vars in 1102 bytes} [Tue Sep 13 03:08:35 2022] GET /identity/v3/roles/service => generated 85 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 03:08:35.674686 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7bd14b6-d5c0-43a3-99fd-5073092605e1 None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:08:35.724461 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f32bbd8-df54-433b-a5aa-0d6859e5ec07 None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:08:35.760436 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2375cbff-758a-4ce7-9e3f-e0e0c93cec4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:08:35.761100 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2375cbff-758a-4ce7-9e3f-e0e0c93cec4e None admin] REQUEST_METHOD: `PUT` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:08:35.761381 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2375cbff-758a-4ce7-9e3f-e0e0c93cec4e None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:08:35.761656 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2375cbff-758a-4ce7-9e3f-e0e0c93cec4e None admin] PATH_INFO: `/v3/projects/b9d94f6a68f7470d9494f112175620dc/users/893b0c0d85964b7dbc9705c0a091682c/roles/425d4b481c7846e0bc07e7e805d5b15f` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:08:35.763737 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2375cbff-758a-4ce7-9e3f-e0e0c93cec4e None admin] RBAC: Authorizing `identity:create_grant(project_id=b9d94f6a68f7470d9494f112175620dc, user_id=893b0c0d85964b7dbc9705c0a091682c, role_id=425d4b481c7846e0bc07e7e805d5b15f)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:08:35.765786 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2375cbff-758a-4ce7-9e3f-e0e0c93cec4e None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:08:35.786347 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7bd14b6-d5c0-43a3-99fd-5073092605e1 None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:08:35.786858 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 400/802] 10.222.0.18 () {58 vars in 1439 bytes} [Tue Sep 13 03:08:35 2022] PUT /identity/v3/projects/b9d94f6a68f7470d9494f112175620dc/users/893b0c0d85964b7dbc9705c0a091682c/roles/425d4b481c7846e0bc07e7e805d5b15f => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 03:08:35.787896 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2375cbff-758a-4ce7-9e3f-e0e0c93cec4e None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:08:35.803014 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:08:35.826653 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0c684f3-072d-4f30-a7db-8244dbac1752 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:08:35.828093 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c684f3-072d-4f30-a7db-8244dbac1752 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:08:35.828386 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c684f3-072d-4f30-a7db-8244dbac1752 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:08:35.828664 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c684f3-072d-4f30-a7db-8244dbac1752 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:08:35.837115 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0c684f3-072d-4f30-a7db-8244dbac1752 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:08:35.838745 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0c684f3-072d-4f30-a7db-8244dbac1752 None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:08:35.851629 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application [None req-d0c684f3-072d-4f30-a7db-8244dbac1752 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 13 03:08:35.851629 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 03:08:35.851629 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 03:08:35.851629 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 03:08:35.851629 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 03:08:35.851629 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 03:08:35.851629 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 03:08:35.851629 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 03:08:35.851629 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 03:08:35.851629 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 03:08:35.851629 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 03:08:35.851629 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 03:08:35.851629 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 13 03:08:35.851629 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 13 03:08:35.851629 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 13 03:08:35.851629 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 13 03:08:35.852633 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 03:08:35.852633 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 03:08:35.852633 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 13 03:08:35.852633 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 13 03:08:35.852633 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 13 03:08:35.852633 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 13 03:08:35.852633 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 03:08:35.852633 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 03:08:35.852633 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 03:08:35.852633 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 03:08:35.852633 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 03:08:35.852633 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application with Lock( Sep 13 03:08:35.852633 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 03:08:35.852633 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._enter() Sep 13 03:08:35.852633 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 03:08:35.852633 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 03:08:35.852633 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 03:08:35.852633 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.creator() Sep 13 03:08:35.853590 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 03:08:35.853590 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application created_value = creator( Sep 13 03:08:35.853590 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 13 03:08:35.853590 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 13 03:08:35.853590 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 13 03:08:35.853590 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 13 03:08:35.853590 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 13 03:08:35.853590 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 13 03:08:35.853590 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 13 03:08:35.853590 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Sep 13 03:08:35.853590 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 403/803] 10.222.0.18 () {56 vars in 1093 bytes} [Tue Sep 13 03:08:35 2022] GET /identity/v3/users/nova => generated 82 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 03:08:35.859425 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0c684f3-072d-4f30-a7db-8244dbac1752 None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:08:35.899118 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:08:36.005383 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0c684f3-072d-4f30-a7db-8244dbac1752 None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:08:36.028670 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2375cbff-758a-4ce7-9e3f-e0e0c93cec4e None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:08:36.048178 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e9283f8-3d47-4acd-865f-be2f5b25e348 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:08:36.049319 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e9283f8-3d47-4acd-865f-be2f5b25e348 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:08:36.049601 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e9283f8-3d47-4acd-865f-be2f5b25e348 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:08:36.049867 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e9283f8-3d47-4acd-865f-be2f5b25e348 None admin] PATH_INFO: `/v3/users` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:08:36.050705 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e9283f8-3d47-4acd-865f-be2f5b25e348 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 03:08:36.051012 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e9283f8-3d47-4acd-865f-be2f5b25e348 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:08:36.052499 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e9283f8-3d47-4acd-865f-be2f5b25e348 None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:08:36.082963 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 404/804] 10.222.0.18 () {56 vars in 1133 bytes} [Tue Sep 13 03:08:35 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 03:08:36.087629 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e9283f8-3d47-4acd-865f-be2f5b25e348 None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:08:36.095880 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee868e7a-e525-4187-8963-4dd3db723c92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:08:36.097313 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee868e7a-e525-4187-8963-4dd3db723c92 None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:08:36.097837 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee868e7a-e525-4187-8963-4dd3db723c92 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:08:36.098336 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee868e7a-e525-4187-8963-4dd3db723c92 None admin] PATH_INFO: `/v3/roles` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:08:36.099691 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee868e7a-e525-4187-8963-4dd3db723c92 None admin] RBAC: Adding query filter params (name=service) {{(pid=81399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 03:08:36.100623 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee868e7a-e525-4187-8963-4dd3db723c92 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:08:36.102520 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee868e7a-e525-4187-8963-4dd3db723c92 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:08:36.114637 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 401/805] 10.222.0.18 () {56 vars in 1103 bytes} [Tue Sep 13 03:08:35 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 326 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 03:08:36.119650 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee868e7a-e525-4187-8963-4dd3db723c92 None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:08:36.125625 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:08:36.236609 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee868e7a-e525-4187-8963-4dd3db723c92 None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:08:36.253160 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e9283f8-3d47-4acd-865f-be2f5b25e348 None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:08:36.264076 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:08:36.277066 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1152f926-d4a5-43ed-a43c-8fa2ab3e2f97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:08:36.278050 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1152f926-d4a5-43ed-a43c-8fa2ab3e2f97 None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:08:36.278484 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1152f926-d4a5-43ed-a43c-8fa2ab3e2f97 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:08:36.278927 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1152f926-d4a5-43ed-a43c-8fa2ab3e2f97 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:08:36.293248 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1152f926-d4a5-43ed-a43c-8fa2ab3e2f97 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:08:36.295396 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1152f926-d4a5-43ed-a43c-8fa2ab3e2f97 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:08:36.296439 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9209ff3-84d7-4cdf-b70a-ba864bfd95ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:08:36.297122 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9209ff3-84d7-4cdf-b70a-ba864bfd95ca None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:08:36.297504 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9209ff3-84d7-4cdf-b70a-ba864bfd95ca None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:08:36.297863 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9209ff3-84d7-4cdf-b70a-ba864bfd95ca None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:08:36.313078 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application [None req-1152f926-d4a5-43ed-a43c-8fa2ab3e2f97 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 03:08:36.313078 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 03:08:36.313078 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 03:08:36.313078 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 03:08:36.313078 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 03:08:36.313078 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 03:08:36.313078 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 03:08:36.313078 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 03:08:36.313078 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 03:08:36.313078 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 03:08:36.313078 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 03:08:36.313078 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 03:08:36.313078 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 13 03:08:36.313078 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 13 03:08:36.313078 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 13 03:08:36.313078 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 13 03:08:36.314285 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 03:08:36.314285 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 03:08:36.314285 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 03:08:36.314285 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 03:08:36.314285 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 03:08:36.314285 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 03:08:36.314285 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 03:08:36.314285 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application with Lock( Sep 13 03:08:36.314285 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 03:08:36.314285 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._enter() Sep 13 03:08:36.314285 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 03:08:36.314285 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 03:08:36.314285 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 03:08:36.314285 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.creator() Sep 13 03:08:36.314285 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 03:08:36.314285 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application created_value = creator( Sep 13 03:08:36.314285 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 13 03:08:36.315401 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 13 03:08:36.315401 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 03:08:36.315401 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Sep 13 03:08:36.322065 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 402/806] 10.222.0.18 () {56 vars in 1108 bytes} [Tue Sep 13 03:08:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 13 03:08:36.322065 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d96aba75-f898-4951-bb4b-a55c644d6ffc None None] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:08:36.322065 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d96aba75-f898-4951-bb4b-a55c644d6ffc None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:08:36.322065 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d96aba75-f898-4951-bb4b-a55c644d6ffc None None] PATH_INFO: `/` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:08:36.322065 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 403/807] 10.222.0.18 () {54 vars in 927 bytes} [Tue Sep 13 03:08:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 03:08:36.324141 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d96aba75-f898-4951-bb4b-a55c644d6ffc None None] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:08:36.383143 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9209ff3-84d7-4cdf-b70a-ba864bfd95ca None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:08:36.384714 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9209ff3-84d7-4cdf-b70a-ba864bfd95ca None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:08:36.417558 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:08:36.452081 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d96aba75-f898-4951-bb4b-a55c644d6ffc None None] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:08:36.488736 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ae096c3-6d3c-4c7c-946d-274032167936 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:08:36.489797 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae096c3-6d3c-4c7c-946d-274032167936 None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:08:36.490271 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae096c3-6d3c-4c7c-946d-274032167936 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:08:36.490745 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae096c3-6d3c-4c7c-946d-274032167936 None admin] PATH_INFO: `/v3/domains` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:08:36.491559 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ae096c3-6d3c-4c7c-946d-274032167936 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 03:08:36.492054 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ae096c3-6d3c-4c7c-946d-274032167936 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:08:36.493611 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ae096c3-6d3c-4c7c-946d-274032167936 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:08:36.498539 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 405/808] 10.222.0.18 () {58 vars in 1282 bytes} [Tue Sep 13 03:08:36 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 411 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 13 03:08:36.499926 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8638c52-b2c0-42d3-b236-81d8afef6757 None None] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:08:36.500212 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8638c52-b2c0-42d3-b236-81d8afef6757 None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:08:36.500486 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8638c52-b2c0-42d3-b236-81d8afef6757 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:08:36.512305 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 404/809] 10.222.0.18 () {56 vars in 1109 bytes} [Tue Sep 13 03:08:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 03:08:36.513504 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ae096c3-6d3c-4c7c-946d-274032167936 None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:08:36.520067 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.auth.core [None req-a8638c52-b2c0-42d3-b236-81d8afef6757 None None] MFA Rules not processed for user `49f280ab086b4610855c4ffa378517d9`. Rule list: `[]` (Enabled: `True`). {{(pid=81400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 03:08:36.521188 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-a8638c52-b2c0-42d3-b236-81d8afef6757 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:08:36.544515 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:08:36.606986 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 406/810] 10.222.0.18 () {58 vars in 1024 bytes} [Tue Sep 13 03:08:36 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 13 03:08:36.608184 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8638c52-b2c0-42d3-b236-81d8afef6757 None None] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:08:36.625686 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ae096c3-6d3c-4c7c-946d-274032167936 None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:08:36.661962 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba4a4ffd-b944-435f-8311-7c5866c90c21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:08:36.662997 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba4a4ffd-b944-435f-8311-7c5866c90c21 None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:08:36.663585 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba4a4ffd-b944-435f-8311-7c5866c90c21 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:08:36.664061 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba4a4ffd-b944-435f-8311-7c5866c90c21 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:08:36.675357 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba4a4ffd-b944-435f-8311-7c5866c90c21 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:08:36.677118 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba4a4ffd-b944-435f-8311-7c5866c90c21 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:08:36.693789 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application [None req-ba4a4ffd-b944-435f-8311-7c5866c90c21 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 03:08:36.693789 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 03:08:36.693789 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 03:08:36.693789 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 03:08:36.693789 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 03:08:36.693789 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 03:08:36.693789 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 03:08:36.693789 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 03:08:36.693789 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 03:08:36.693789 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 03:08:36.693789 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 03:08:36.693789 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 03:08:36.693789 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 13 03:08:36.693789 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 13 03:08:36.693789 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 13 03:08:36.693789 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 13 03:08:36.694820 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 03:08:36.694820 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 03:08:36.694820 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 03:08:36.694820 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 03:08:36.694820 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 03:08:36.694820 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 03:08:36.694820 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 03:08:36.694820 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application with Lock( Sep 13 03:08:36.694820 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 03:08:36.694820 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._enter() Sep 13 03:08:36.694820 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 03:08:36.694820 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 03:08:36.694820 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 03:08:36.694820 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.creator() Sep 13 03:08:36.694820 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 03:08:36.694820 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application created_value = creator( Sep 13 03:08:36.694820 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 13 03:08:36.695738 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 13 03:08:36.695738 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 03:08:36.695738 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Sep 13 03:08:36.697611 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 405/811] 10.222.0.18 () {56 vars in 1108 bytes} [Tue Sep 13 03:08:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 03:08:36.703765 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba4a4ffd-b944-435f-8311-7c5866c90c21 None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:08:36.758982 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:08:36.771347 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8638c52-b2c0-42d3-b236-81d8afef6757 None None] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:08:36.797831 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:08:36.825273 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1beb9aea-415a-44f5-b8fa-205961ecad99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:08:36.825902 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1beb9aea-415a-44f5-b8fa-205961ecad99 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:08:36.826179 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1beb9aea-415a-44f5-b8fa-205961ecad99 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:08:36.826984 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1beb9aea-415a-44f5-b8fa-205961ecad99 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:08:36.829548 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba4a4ffd-b944-435f-8311-7c5866c90c21 None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:08:36.839642 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1beb9aea-415a-44f5-b8fa-205961ecad99 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:08:36.842152 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1beb9aea-415a-44f5-b8fa-205961ecad99 None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:08:36.856558 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application [None req-1beb9aea-415a-44f5-b8fa-205961ecad99 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 13 03:08:36.856558 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 03:08:36.856558 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 03:08:36.856558 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 03:08:36.856558 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 03:08:36.856558 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 03:08:36.856558 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 03:08:36.856558 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 03:08:36.856558 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 03:08:36.856558 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 03:08:36.856558 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 03:08:36.856558 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 03:08:36.856558 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 13 03:08:36.856558 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 13 03:08:36.856558 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 13 03:08:36.856558 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 13 03:08:36.857519 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 03:08:36.857519 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 03:08:36.857519 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 13 03:08:36.857519 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 13 03:08:36.857519 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 13 03:08:36.857519 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 13 03:08:36.857519 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 03:08:36.857519 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 03:08:36.857519 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 03:08:36.857519 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 03:08:36.857519 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 03:08:36.857519 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application with Lock( Sep 13 03:08:36.857519 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 03:08:36.857519 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._enter() Sep 13 03:08:36.857519 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 03:08:36.857519 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 03:08:36.857519 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 03:08:36.857519 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.creator() Sep 13 03:08:36.858514 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 03:08:36.858514 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application created_value = creator( Sep 13 03:08:36.858514 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 13 03:08:36.858514 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 13 03:08:36.858514 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 13 03:08:36.858514 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 13 03:08:36.858514 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 13 03:08:36.858514 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 13 03:08:36.858514 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 13 03:08:36.858514 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Sep 13 03:08:36.862348 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 407/812] 10.222.0.18 () {56 vars in 1117 bytes} [Tue Sep 13 03:08:36 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 255 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 03:08:36.863762 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cea2006-d6d8-4c70-9d27-5485859a397d None None] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:08:36.864254 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cea2006-d6d8-4c70-9d27-5485859a397d None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:08:36.864732 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cea2006-d6d8-4c70-9d27-5485859a397d None None] PATH_INFO: `/` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:08:36.870380 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 408/813] 10.222.0.18 () {54 vars in 927 bytes} [Tue Sep 13 03:08:36 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 03:08:36.871699 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cea2006-d6d8-4c70-9d27-5485859a397d None None] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:08:36.872806 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccacfc55-825e-40af-95ac-212f130c04ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:08:36.873465 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccacfc55-825e-40af-95ac-212f130c04ce None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:08:36.873791 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccacfc55-825e-40af-95ac-212f130c04ce None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:08:36.874070 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccacfc55-825e-40af-95ac-212f130c04ce None admin] PATH_INFO: `/v3/domains` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:08:36.874756 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccacfc55-825e-40af-95ac-212f130c04ce None admin] RBAC: Adding query filter params (name=Default) {{(pid=81399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 03:08:36.875062 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccacfc55-825e-40af-95ac-212f130c04ce None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:08:36.876499 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccacfc55-825e-40af-95ac-212f130c04ce None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:08:36.891252 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 406/814] 10.222.0.18 () {56 vars in 1109 bytes} [Tue Sep 13 03:08:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 03:08:36.892738 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e88ded08-2910-4888-b6f8-255485072b83 None None] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:08:36.893023 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e88ded08-2910-4888-b6f8-255485072b83 None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:08:36.893295 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e88ded08-2910-4888-b6f8-255485072b83 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:08:36.913064 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.auth.core [None req-e88ded08-2910-4888-b6f8-255485072b83 None None] MFA Rules not processed for user `49f280ab086b4610855c4ffa378517d9`. Rule list: `[]` (Enabled: `True`). {{(pid=81399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 03:08:36.914266 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-e88ded08-2910-4888-b6f8-255485072b83 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:08:36.985812 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cea2006-d6d8-4c70-9d27-5485859a397d None None] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:08:36.994702 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 407/815] 10.222.0.18 () {58 vars in 1024 bytes} [Tue Sep 13 03:08:36 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 13 03:08:36.994702 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e88ded08-2910-4888-b6f8-255485072b83 None None] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:08:37.011235 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:08:37.025328 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f1e3027-686d-452a-9e86-9d1931cc93bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:08:37.025866 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f1e3027-686d-452a-9e86-9d1931cc93bc None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:08:37.026169 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f1e3027-686d-452a-9e86-9d1931cc93bc None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:08:37.026409 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f1e3027-686d-452a-9e86-9d1931cc93bc None admin] PATH_INFO: `/v3/users` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:08:37.027172 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f1e3027-686d-452a-9e86-9d1931cc93bc None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 03:08:37.027475 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f1e3027-686d-452a-9e86-9d1931cc93bc None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:08:37.028950 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f1e3027-686d-452a-9e86-9d1931cc93bc None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:08:37.052603 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 409/816] 10.222.0.18 () {56 vars in 1149 bytes} [Tue Sep 13 03:08:36 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 03:08:37.052603 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f1e3027-686d-452a-9e86-9d1931cc93bc None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:08:37.091255 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:08:37.131551 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e88ded08-2910-4888-b6f8-255485072b83 None None] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:08:37.162116 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f1e3027-686d-452a-9e86-9d1931cc93bc None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:08:37.173340 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e162abd-94f5-4a2b-b712-b050bade1cab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:08:37.174727 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e162abd-94f5-4a2b-b712-b050bade1cab None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:08:37.175258 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e162abd-94f5-4a2b-b712-b050bade1cab None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:08:37.175678 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e162abd-94f5-4a2b-b712-b050bade1cab None admin] PATH_INFO: `/v3/projects/service` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:08:37.183923 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e162abd-94f5-4a2b-b712-b050bade1cab None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:08:37.186846 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e162abd-94f5-4a2b-b712-b050bade1cab None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:08:37.199439 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80c5291e-8298-429d-aaa7-33144f3a23dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:08:37.199980 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application [None req-1e162abd-94f5-4a2b-b712-b050bade1cab None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 13 03:08:37.199980 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 03:08:37.199980 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 03:08:37.199980 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 03:08:37.199980 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 03:08:37.199980 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 03:08:37.199980 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 03:08:37.199980 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 03:08:37.199980 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 03:08:37.199980 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 03:08:37.199980 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 03:08:37.199980 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 03:08:37.199980 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 13 03:08:37.199980 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 13 03:08:37.199980 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 13 03:08:37.201040 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 13 03:08:37.201040 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 03:08:37.201040 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 03:08:37.201040 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 03:08:37.201040 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 03:08:37.201040 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 03:08:37.201040 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 03:08:37.201040 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 03:08:37.201040 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application with Lock( Sep 13 03:08:37.201040 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 03:08:37.201040 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._enter() Sep 13 03:08:37.201040 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 03:08:37.201040 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 03:08:37.201040 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 03:08:37.201040 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.creator() Sep 13 03:08:37.201040 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 03:08:37.201040 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application created_value = creator( Sep 13 03:08:37.202822 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 13 03:08:37.202822 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 13 03:08:37.202822 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 13 03:08:37.202822 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 13 03:08:37.202822 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 13 03:08:37.202822 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 13 03:08:37.202822 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 13 03:08:37.202822 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Sep 13 03:08:37.203354 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c5291e-8298-429d-aaa7-33144f3a23dc None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:08:37.203642 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c5291e-8298-429d-aaa7-33144f3a23dc None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:08:37.204015 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c5291e-8298-429d-aaa7-33144f3a23dc None admin] PATH_INFO: `/v3/services/network` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:08:37.204787 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80c5291e-8298-429d-aaa7-33144f3a23dc None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:08:37.205440 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 408/817] 10.222.0.18 () {56 vars in 1111 bytes} [Tue Sep 13 03:08:36 2022] GET /identity/v3/projects/service => generated 88 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 03:08:37.206338 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80c5291e-8298-429d-aaa7-33144f3a23dc None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:08:37.206504 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e162abd-94f5-4a2b-b712-b050bade1cab None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:08:37.218858 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application [None req-80c5291e-8298-429d-aaa7-33144f3a23dc None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Sep 13 03:08:37.218858 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 03:08:37.218858 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 13 03:08:37.218858 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 13 03:08:37.218858 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 13 03:08:37.218858 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 13 03:08:37.218858 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 13 03:08:37.218858 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 13 03:08:37.218858 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Sep 13 03:08:37.218858 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Sep 13 03:08:37.218858 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 13 03:08:37.218858 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Sep 13 03:08:37.218858 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 03:08:37.218858 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 03:08:37.218858 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 03:08:37.218858 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 03:08:37.218858 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 03:08:37.220099 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 03:08:37.220099 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 03:08:37.220099 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 03:08:37.220099 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 03:08:37.220099 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 03:08:37.220099 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 03:08:37.220099 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 13 03:08:37.220099 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 13 03:08:37.220099 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 13 03:08:37.220099 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 13 03:08:37.220099 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 03:08:37.220099 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 03:08:37.220099 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 03:08:37.220099 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 03:08:37.220099 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 03:08:37.220099 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 03:08:37.221129 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 03:08:37.221129 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application with Lock( Sep 13 03:08:37.221129 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 03:08:37.221129 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._enter() Sep 13 03:08:37.221129 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 03:08:37.221129 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 03:08:37.221129 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 03:08:37.221129 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.creator() Sep 13 03:08:37.221129 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 03:08:37.221129 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application created_value = creator( Sep 13 03:08:37.221129 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 13 03:08:37.221129 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 13 03:08:37.221129 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Sep 13 03:08:37.221129 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Sep 13 03:08:37.223260 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 410/818] 10.222.0.18 () {56 vars in 1111 bytes} [Tue Sep 13 03:08:37 2022] GET /identity/v3/services/network => generated 88 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 03:08:37.224292 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80c5291e-8298-429d-aaa7-33144f3a23dc None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:08:37.275033 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:08:37.325793 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e162abd-94f5-4a2b-b712-b050bade1cab None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:08:37.333958 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80c5291e-8298-429d-aaa7-33144f3a23dc None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:08:37.352809 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:08:37.364223 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-609909df-080b-43c8-bd8f-fc0d1c90db6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:08:37.364849 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-609909df-080b-43c8-bd8f-fc0d1c90db6c None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:08:37.365131 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-609909df-080b-43c8-bd8f-fc0d1c90db6c None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:08:37.365402 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-609909df-080b-43c8-bd8f-fc0d1c90db6c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:08:37.370106 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f630697-0a62-429e-be3c-0582cdf01751 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:08:37.370729 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f630697-0a62-429e-be3c-0582cdf01751 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:08:37.371009 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f630697-0a62-429e-be3c-0582cdf01751 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:08:37.371280 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f630697-0a62-429e-be3c-0582cdf01751 None admin] PATH_INFO: `/v3/projects` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:08:37.371958 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f630697-0a62-429e-be3c-0582cdf01751 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 03:08:37.372679 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f630697-0a62-429e-be3c-0582cdf01751 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:08:37.375195 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f630697-0a62-429e-be3c-0582cdf01751 None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:08:37.376375 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-609909df-080b-43c8-bd8f-fc0d1c90db6c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:08:37.378348 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-609909df-080b-43c8-bd8f-fc0d1c90db6c None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:08:37.391828 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 411/819] 10.222.0.18 () {56 vars in 1148 bytes} [Tue Sep 13 03:08:37 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 03:08:37.391828 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f630697-0a62-429e-be3c-0582cdf01751 None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:08:37.399449 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application [None req-609909df-080b-43c8-bd8f-fc0d1c90db6c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 03:08:37.399449 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 03:08:37.399449 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 03:08:37.399449 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 03:08:37.399449 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 03:08:37.399449 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 03:08:37.399449 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 03:08:37.399449 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 03:08:37.399449 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 03:08:37.399449 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 03:08:37.399449 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 03:08:37.399449 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 03:08:37.399449 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 13 03:08:37.399449 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 13 03:08:37.399449 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 13 03:08:37.399449 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 13 03:08:37.400405 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 03:08:37.400405 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 03:08:37.400405 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 03:08:37.400405 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 03:08:37.400405 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 03:08:37.400405 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 03:08:37.400405 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 03:08:37.400405 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application with Lock( Sep 13 03:08:37.400405 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 03:08:37.400405 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._enter() Sep 13 03:08:37.400405 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 03:08:37.400405 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 03:08:37.400405 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 03:08:37.400405 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.creator() Sep 13 03:08:37.400405 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 03:08:37.400405 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application created_value = creator( Sep 13 03:08:37.400405 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 13 03:08:37.401315 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 13 03:08:37.401315 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 03:08:37.401315 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Sep 13 03:08:37.401315 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 409/820] 10.222.0.18 () {56 vars in 1108 bytes} [Tue Sep 13 03:08:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 03:08:37.401315 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-609909df-080b-43c8-bd8f-fc0d1c90db6c None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:08:37.502758 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f630697-0a62-429e-be3c-0582cdf01751 None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:08:37.505030 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:08:37.505966 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-609909df-080b-43c8-bd8f-fc0d1c90db6c None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:08:37.538554 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d786a14-0124-49c8-9974-f1d15754696c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:08:37.539180 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d786a14-0124-49c8-9974-f1d15754696c None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:08:37.539457 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d786a14-0124-49c8-9974-f1d15754696c None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:08:37.539724 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d786a14-0124-49c8-9974-f1d15754696c None admin] PATH_INFO: `/v3/services` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:08:37.540389 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d786a14-0124-49c8-9974-f1d15754696c None admin] RBAC: Adding query filter params (name=network) {{(pid=81400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 03:08:37.540644 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d786a14-0124-49c8-9974-f1d15754696c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:08:37.541980 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20adc21d-679b-4314-ae37-bcb48afc283b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:08:37.542099 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d786a14-0124-49c8-9974-f1d15754696c None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:08:37.542621 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20adc21d-679b-4314-ae37-bcb48afc283b None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:08:37.543024 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20adc21d-679b-4314-ae37-bcb48afc283b None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:08:37.543305 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20adc21d-679b-4314-ae37-bcb48afc283b None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:08:37.543926 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20adc21d-679b-4314-ae37-bcb48afc283b None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:08:37.545796 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20adc21d-679b-4314-ae37-bcb48afc283b None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:08:37.547210 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 412/821] 10.222.0.18 () {56 vars in 1112 bytes} [Tue Sep 13 03:08:37 2022] GET /identity/v3/services?name=network => generated 124 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 03:08:37.552235 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d786a14-0124-49c8-9974-f1d15754696c None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:08:37.555151 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application [None req-20adc21d-679b-4314-ae37-bcb48afc283b None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 13 03:08:37.555151 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 03:08:37.555151 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 13 03:08:37.555151 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 13 03:08:37.555151 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 13 03:08:37.555151 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 13 03:08:37.555151 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 13 03:08:37.555151 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 13 03:08:37.555151 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 13 03:08:37.555151 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Sep 13 03:08:37.555151 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 13 03:08:37.555151 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Sep 13 03:08:37.555151 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 03:08:37.555151 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 03:08:37.555151 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 03:08:37.555151 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 03:08:37.555151 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 03:08:37.556144 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 03:08:37.556144 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 03:08:37.556144 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 03:08:37.556144 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 03:08:37.556144 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 03:08:37.556144 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 03:08:37.556144 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 13 03:08:37.556144 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 13 03:08:37.556144 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 13 03:08:37.556144 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 13 03:08:37.556144 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 03:08:37.556144 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 03:08:37.556144 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 03:08:37.556144 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 03:08:37.556144 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 03:08:37.556144 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 03:08:37.557002 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 03:08:37.557002 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application with Lock( Sep 13 03:08:37.557002 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 03:08:37.557002 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._enter() Sep 13 03:08:37.557002 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 03:08:37.557002 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 03:08:37.557002 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 03:08:37.557002 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.creator() Sep 13 03:08:37.557002 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 03:08:37.557002 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application created_value = creator( Sep 13 03:08:37.557002 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 13 03:08:37.557002 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 13 03:08:37.557002 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 13 03:08:37.557002 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Sep 13 03:08:37.557002 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 410/822] 10.222.0.18 () {56 vars in 1129 bytes} [Tue Sep 13 03:08:37 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 03:08:37.559173 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20adc21d-679b-4314-ae37-bcb48afc283b None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:08:37.605962 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:08:37.663383 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d786a14-0124-49c8-9974-f1d15754696c None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:08:37.667151 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20adc21d-679b-4314-ae37-bcb48afc283b None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:08:37.699610 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98e63ccc-c196-4163-9389-ecd8f8242b57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:08:37.700211 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e63ccc-c196-4163-9389-ecd8f8242b57 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:08:37.700489 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e63ccc-c196-4163-9389-ecd8f8242b57 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:08:37.700759 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e63ccc-c196-4163-9389-ecd8f8242b57 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:08:37.701509 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98e63ccc-c196-4163-9389-ecd8f8242b57 None admin] RBAC: Adding query filter params (role.id=425d4b481c7846e0bc07e7e805d5b15f, scope.project.id=b9d94f6a68f7470d9494f112175620dc, user.id=c7377f5bee4a49959182e1bd34bc97a4) {{(pid=81400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 03:08:37.701821 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98e63ccc-c196-4163-9389-ecd8f8242b57 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:08:37.703304 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98e63ccc-c196-4163-9389-ecd8f8242b57 None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:08:37.706571 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-700b82c9-4cc2-4e70-b626-ce2284ee6f24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:08:37.707760 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-700b82c9-4cc2-4e70-b626-ce2284ee6f24 None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:08:37.708225 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-700b82c9-4cc2-4e70-b626-ce2284ee6f24 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:08:37.708696 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-700b82c9-4cc2-4e70-b626-ce2284ee6f24 None admin] PATH_INFO: `/v3/domains` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:08:37.709184 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 413/823] 10.222.0.18 () {56 vars in 1374 bytes} [Tue Sep 13 03:08:37 2022] GET /identity/v3/role_assignments?user.id=c7377f5bee4a49959182e1bd34bc97a4&scope.project.id=b9d94f6a68f7470d9494f112175620dc&role.id=425d4b481c7846e0bc07e7e805d5b15f => generated 608 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 03:08:37.709714 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-700b82c9-4cc2-4e70-b626-ce2284ee6f24 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 03:08:37.709974 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98e63ccc-c196-4163-9389-ecd8f8242b57 None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:08:37.710525 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-700b82c9-4cc2-4e70-b626-ce2284ee6f24 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:08:37.712108 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-700b82c9-4cc2-4e70-b626-ce2284ee6f24 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:08:37.732765 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 411/824] 10.222.0.18 () {56 vars in 1109 bytes} [Tue Sep 13 03:08:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 03:08:37.734718 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-700b82c9-4cc2-4e70-b626-ce2284ee6f24 None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:08:37.757120 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:08:37.840924 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98e63ccc-c196-4163-9389-ecd8f8242b57 None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:08:37.881044 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-700b82c9-4cc2-4e70-b626-ce2284ee6f24 None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:08:37.881723 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-555c9d63-5d46-434e-a321-4f10e82275af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:08:37.882551 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-555c9d63-5d46-434e-a321-4f10e82275af None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:08:37.883394 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-555c9d63-5d46-434e-a321-4f10e82275af None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:08:37.883846 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:08:37.884572 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-555c9d63-5d46-434e-a321-4f10e82275af None admin] PATH_INFO: `/v3/services` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:08:37.885393 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-555c9d63-5d46-434e-a321-4f10e82275af None admin] RBAC: Adding query filter params (type=network) {{(pid=81400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 03:08:37.885871 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-555c9d63-5d46-434e-a321-4f10e82275af None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:08:37.887795 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-555c9d63-5d46-434e-a321-4f10e82275af None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:08:37.895861 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 414/825] 10.222.0.18 () {56 vars in 1112 bytes} [Tue Sep 13 03:08:37 2022] GET /identity/v3/services?type=network => generated 350 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 03:08:37.897182 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-555c9d63-5d46-434e-a321-4f10e82275af None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:08:37.926830 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d62c4658-e714-40b7-8d39-13c7d20d4b71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:08:37.927824 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d62c4658-e714-40b7-8d39-13c7d20d4b71 None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:08:37.928252 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d62c4658-e714-40b7-8d39-13c7d20d4b71 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:08:37.928670 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d62c4658-e714-40b7-8d39-13c7d20d4b71 None admin] PATH_INFO: `/v3/services` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:08:37.929749 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d62c4658-e714-40b7-8d39-13c7d20d4b71 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=81399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 03:08:37.930375 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d62c4658-e714-40b7-8d39-13c7d20d4b71 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:08:37.932321 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d62c4658-e714-40b7-8d39-13c7d20d4b71 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:08:37.938062 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 412/826] 10.222.0.18 () {56 vars in 1124 bytes} [Tue Sep 13 03:08:37 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 03:08:37.941744 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d62c4658-e714-40b7-8d39-13c7d20d4b71 None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:08:38.017401 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:08:38.024995 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-555c9d63-5d46-434e-a321-4f10e82275af None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:08:38.054260 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d62c4658-e714-40b7-8d39-13c7d20d4b71 None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:08:38.065997 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fa830b5-4b17-46de-81fd-d7f07ea13083 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:08:38.066654 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa830b5-4b17-46de-81fd-d7f07ea13083 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:08:38.066951 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa830b5-4b17-46de-81fd-d7f07ea13083 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:08:38.067226 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa830b5-4b17-46de-81fd-d7f07ea13083 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:08:38.078247 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fa830b5-4b17-46de-81fd-d7f07ea13083 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:08:38.080287 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fa830b5-4b17-46de-81fd-d7f07ea13083 None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:08:38.094603 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application [None req-3fa830b5-4b17-46de-81fd-d7f07ea13083 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 13 03:08:38.094603 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 03:08:38.094603 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 03:08:38.094603 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 03:08:38.094603 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 03:08:38.094603 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 03:08:38.094603 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 03:08:38.094603 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 03:08:38.094603 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 03:08:38.094603 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 03:08:38.094603 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 03:08:38.094603 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 03:08:38.094603 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 13 03:08:38.094603 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 13 03:08:38.094603 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 13 03:08:38.095627 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 13 03:08:38.095627 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 03:08:38.095627 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 03:08:38.095627 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 03:08:38.095627 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 03:08:38.095627 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 03:08:38.095627 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 03:08:38.095627 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 03:08:38.095627 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application with Lock( Sep 13 03:08:38.095627 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 03:08:38.095627 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._enter() Sep 13 03:08:38.095627 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 03:08:38.095627 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 03:08:38.095627 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 03:08:38.095627 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.creator() Sep 13 03:08:38.095627 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 03:08:38.095627 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application created_value = creator( Sep 13 03:08:38.096538 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 13 03:08:38.096538 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 13 03:08:38.096538 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 13 03:08:38.096538 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 13 03:08:38.096538 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 13 03:08:38.096538 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 13 03:08:38.096538 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 13 03:08:38.096538 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Sep 13 03:08:38.103869 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 415/827] 10.222.0.18 () {56 vars in 1111 bytes} [Tue Sep 13 03:08:37 2022] GET /identity/v3/projects/service => generated 88 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 03:08:38.105469 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fa830b5-4b17-46de-81fd-d7f07ea13083 None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:08:38.107038 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-511675f7-ca9f-437d-8550-2e8ab2e325e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:08:38.108170 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-511675f7-ca9f-437d-8550-2e8ab2e325e1 None admin] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:08:38.108404 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-511675f7-ca9f-437d-8550-2e8ab2e325e1 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:08:38.108676 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-511675f7-ca9f-437d-8550-2e8ab2e325e1 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:08:38.109361 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-511675f7-ca9f-437d-8550-2e8ab2e325e1 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:08:38.110771 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-511675f7-ca9f-437d-8550-2e8ab2e325e1 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:08:38.142714 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 413/828] 10.222.0.18 () {60 vars in 1144 bytes} [Tue Sep 13 03:08:37 2022] POST /identity/v3/endpoints => generated 337 bytes in 199 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 03:08:38.142714 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-511675f7-ca9f-437d-8550-2e8ab2e325e1 None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:08:38.199569 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:08:38.271122 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fa830b5-4b17-46de-81fd-d7f07ea13083 None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:08:38.297400 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-511675f7-ca9f-437d-8550-2e8ab2e325e1 None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:08:38.312127 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78461ca0-a007-4400-b9ea-5d75e19414a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:08:38.312746 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78461ca0-a007-4400-b9ea-5d75e19414a5 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:08:38.313024 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78461ca0-a007-4400-b9ea-5d75e19414a5 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:08:38.313294 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78461ca0-a007-4400-b9ea-5d75e19414a5 None admin] PATH_INFO: `/v3/services` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:08:38.313932 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78461ca0-a007-4400-b9ea-5d75e19414a5 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=81400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 03:08:38.314233 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78461ca0-a007-4400-b9ea-5d75e19414a5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:08:38.315648 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78461ca0-a007-4400-b9ea-5d75e19414a5 None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:08:38.321468 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 416/829] 10.222.0.18 () {56 vars in 1124 bytes} [Tue Sep 13 03:08:38 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 03:08:38.322754 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-986e8456-5689-47fe-a83f-5c2f160c6df1 None None] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:08:38.323044 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-986e8456-5689-47fe-a83f-5c2f160c6df1 None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:08:38.323319 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-986e8456-5689-47fe-a83f-5c2f160c6df1 None None] PATH_INFO: `/` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:08:38.324866 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 417/830] 10.222.0.18 () {54 vars in 927 bytes} [Tue Sep 13 03:08:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 03:08:38.326980 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-986e8456-5689-47fe-a83f-5c2f160c6df1 None None] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:08:38.340099 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee7e6792-485c-4778-8d57-35b0d694a3c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:08:38.340772 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee7e6792-485c-4778-8d57-35b0d694a3c3 None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:08:38.341046 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee7e6792-485c-4778-8d57-35b0d694a3c3 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:08:38.341364 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee7e6792-485c-4778-8d57-35b0d694a3c3 None admin] PATH_INFO: `/v3/projects` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:08:38.342099 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee7e6792-485c-4778-8d57-35b0d694a3c3 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 03:08:38.342403 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee7e6792-485c-4778-8d57-35b0d694a3c3 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:08:38.344852 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee7e6792-485c-4778-8d57-35b0d694a3c3 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:08:38.355931 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:08:38.357346 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 414/831] 10.222.0.18 () {56 vars in 1148 bytes} [Tue Sep 13 03:08:38 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 03:08:38.359415 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e78c3d19-22c3-49a0-91ad-4709f417ff02 None None] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:08:38.359733 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e78c3d19-22c3-49a0-91ad-4709f417ff02 None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:08:38.360101 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e78c3d19-22c3-49a0-91ad-4709f417ff02 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:08:38.379571 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.auth.core [None req-e78c3d19-22c3-49a0-91ad-4709f417ff02 None None] MFA Rules not processed for user `49f280ab086b4610855c4ffa378517d9`. Rule list: `[]` (Enabled: `True`). {{(pid=81399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 03:08:38.381177 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-e78c3d19-22c3-49a0-91ad-4709f417ff02 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:08:38.460732 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 415/832] 10.222.0.18 () {58 vars in 1024 bytes} [Tue Sep 13 03:08:38 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 13 03:08:38.461557 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e78c3d19-22c3-49a0-91ad-4709f417ff02 None None] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:08:38.463468 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-986e8456-5689-47fe-a83f-5c2f160c6df1 None None] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:08:38.477713 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:08:38.502223 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-069ea660-2ea1-4f02-9433-09628bd18e7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:08:38.502851 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-069ea660-2ea1-4f02-9433-09628bd18e7f None admin] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:08:38.503128 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-069ea660-2ea1-4f02-9433-09628bd18e7f None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:08:38.503393 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-069ea660-2ea1-4f02-9433-09628bd18e7f None admin] PATH_INFO: `/v3/endpoints` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:08:38.504047 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-069ea660-2ea1-4f02-9433-09628bd18e7f None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:08:38.505413 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-069ea660-2ea1-4f02-9433-09628bd18e7f None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:08:38.534379 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 418/833] 10.222.0.18 () {60 vars in 1144 bytes} [Tue Sep 13 03:08:38 2022] POST /identity/v3/endpoints => generated 346 bytes in 208 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 03:08:38.581925 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e78c3d19-22c3-49a0-91ad-4709f417ff02 None None] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:08:38.629660 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53b2d669-e361-4911-a957-5da8aecabdbd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:08:38.630343 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53b2d669-e361-4911-a957-5da8aecabdbd None admin] REQUEST_METHOD: `PUT` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:08:38.630639 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53b2d669-e361-4911-a957-5da8aecabdbd None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:08:38.630949 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53b2d669-e361-4911-a957-5da8aecabdbd None admin] PATH_INFO: `/v3/projects/b9d94f6a68f7470d9494f112175620dc/users/4c2d20fb54a54ecc99b47f237be4917c/roles/817d6837cd044890b41cf16b303a092b` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:08:38.647836 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53b2d669-e361-4911-a957-5da8aecabdbd None admin] RBAC: Authorizing `identity:create_grant(project_id=b9d94f6a68f7470d9494f112175620dc, user_id=4c2d20fb54a54ecc99b47f237be4917c, role_id=817d6837cd044890b41cf16b303a092b)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:08:38.649303 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53b2d669-e361-4911-a957-5da8aecabdbd None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:08:38.666876 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 416/834] 10.222.0.18 () {58 vars in 1439 bytes} [Tue Sep 13 03:08:38 2022] PUT /identity/v3/projects/b9d94f6a68f7470d9494f112175620dc/users/4c2d20fb54a54ecc99b47f237be4917c/roles/817d6837cd044890b41cf16b303a092b => generated 0 bytes in 205 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 03:08:38.767489 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-069ea660-2ea1-4f02-9433-09628bd18e7f None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:08:38.856187 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:08:38.884169 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-069ea660-2ea1-4f02-9433-09628bd18e7f None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:08:38.921022 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af1e745b-e558-409a-a028-819321ca0d25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:08:38.921655 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af1e745b-e558-409a-a028-819321ca0d25 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:08:38.921934 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af1e745b-e558-409a-a028-819321ca0d25 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:08:38.922207 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af1e745b-e558-409a-a028-819321ca0d25 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:08:38.932804 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af1e745b-e558-409a-a028-819321ca0d25 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 03:08:38.939267 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application [None req-af1e745b-e558-409a-a028-819321ca0d25 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 13 03:08:38.939267 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 03:08:38.939267 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 03:08:38.939267 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 03:08:38.939267 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 03:08:38.939267 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 03:08:38.939267 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 03:08:38.939267 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 03:08:38.939267 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 03:08:38.939267 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 03:08:38.939267 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 03:08:38.939267 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 03:08:38.939267 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 03:08:38.939267 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 03:08:38.939267 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 03:08:38.939267 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 03:08:38.940209 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 03:08:38.940209 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 03:08:38.940209 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 03:08:38.940209 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 03:08:38.940209 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 03:08:38.940209 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 03:08:38.940209 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 03:08:38.940209 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 03:08:38.940209 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 03:08:38.940209 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 03:08:38.940209 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 03:08:38.940209 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application with Lock( Sep 13 03:08:38.940209 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 03:08:38.940209 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._enter() Sep 13 03:08:38.940209 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 03:08:38.940209 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 03:08:38.941046 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 03:08:38.941046 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.creator() Sep 13 03:08:38.941046 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 03:08:38.941046 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application created_value = creator( Sep 13 03:08:38.941046 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 03:08:38.941046 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 03:08:38.941046 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 03:08:38.941046 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 03:08:38.941046 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 03:08:38.941046 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 03:08:38.941046 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 13 03:08:38.941046 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Sep 13 03:08:38.941046 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 419/835] 10.222.0.18 () {56 vars in 1096 bytes} [Tue Sep 13 03:08:38 2022] GET /identity/v3/roles/admin => generated 83 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 03:08:38.946715 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53b2d669-e361-4911-a957-5da8aecabdbd None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:08:39.005168 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:08:39.051298 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53b2d669-e361-4911-a957-5da8aecabdbd None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:08:39.098097 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80edf9a6-e850-490f-9bff-3da76160634b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:08:39.098741 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80edf9a6-e850-490f-9bff-3da76160634b None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:08:39.099020 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80edf9a6-e850-490f-9bff-3da76160634b None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:08:39.099289 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80edf9a6-e850-490f-9bff-3da76160634b None admin] PATH_INFO: `/v3/roles` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:08:39.099952 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80edf9a6-e850-490f-9bff-3da76160634b None admin] RBAC: Adding query filter params (name=admin) {{(pid=81399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 03:08:39.100253 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80edf9a6-e850-490f-9bff-3da76160634b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:08:39.101701 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80edf9a6-e850-490f-9bff-3da76160634b None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:08:39.114763 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 417/836] 10.222.0.18 () {56 vars in 1099 bytes} [Tue Sep 13 03:08:38 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 03:08:39.116557 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af1e745b-e558-409a-a028-819321ca0d25 None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:08:39.220578 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af1e745b-e558-409a-a028-819321ca0d25 None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:08:39.260838 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eba28fd-6c6f-4596-9c13-73aa91c8cf2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:08:39.261468 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eba28fd-6c6f-4596-9c13-73aa91c8cf2e None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:08:39.261744 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eba28fd-6c6f-4596-9c13-73aa91c8cf2e None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:08:39.262017 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eba28fd-6c6f-4596-9c13-73aa91c8cf2e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:08:39.265410 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:08:39.274337 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eba28fd-6c6f-4596-9c13-73aa91c8cf2e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:08:39.276391 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eba28fd-6c6f-4596-9c13-73aa91c8cf2e None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:08:39.291441 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application [None req-0eba28fd-6c6f-4596-9c13-73aa91c8cf2e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 03:08:39.291441 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 03:08:39.291441 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 03:08:39.291441 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 03:08:39.291441 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 03:08:39.291441 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 03:08:39.291441 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 03:08:39.291441 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 03:08:39.291441 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 03:08:39.291441 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 03:08:39.291441 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 03:08:39.291441 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 03:08:39.291441 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 13 03:08:39.291441 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 13 03:08:39.291441 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 13 03:08:39.291441 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 13 03:08:39.292400 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 03:08:39.292400 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 03:08:39.292400 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 03:08:39.292400 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 03:08:39.292400 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 03:08:39.292400 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 03:08:39.292400 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 03:08:39.292400 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application with Lock( Sep 13 03:08:39.292400 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 03:08:39.292400 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._enter() Sep 13 03:08:39.292400 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 03:08:39.292400 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 03:08:39.292400 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 03:08:39.292400 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.creator() Sep 13 03:08:39.292400 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 03:08:39.292400 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application created_value = creator( Sep 13 03:08:39.292400 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 13 03:08:39.293316 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 13 03:08:39.293316 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 03:08:39.293316 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Sep 13 03:08:39.293316 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 420/837] 10.222.0.18 () {56 vars in 1108 bytes} [Tue Sep 13 03:08:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 03:08:39.298542 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80edf9a6-e850-490f-9bff-3da76160634b None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:08:39.404867 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80edf9a6-e850-490f-9bff-3da76160634b None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:08:39.421809 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:08:39.439817 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e405319b-244d-4230-8aef-2356ff0e0b24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:08:39.440422 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e405319b-244d-4230-8aef-2356ff0e0b24 None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:08:39.440749 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e405319b-244d-4230-8aef-2356ff0e0b24 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:08:39.441023 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e405319b-244d-4230-8aef-2356ff0e0b24 None admin] PATH_INFO: `/v3/domains` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:08:39.441659 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e405319b-244d-4230-8aef-2356ff0e0b24 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 03:08:39.441959 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e405319b-244d-4230-8aef-2356ff0e0b24 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:08:39.443359 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e405319b-244d-4230-8aef-2356ff0e0b24 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:08:39.456260 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 418/838] 10.222.0.18 () {56 vars in 1109 bytes} [Tue Sep 13 03:08:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 03:08:39.463762 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eba28fd-6c6f-4596-9c13-73aa91c8cf2e None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:08:39.576418 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eba28fd-6c6f-4596-9c13-73aa91c8cf2e None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:08:39.615690 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24d20bd5-637e-48d3-a215-306e280c73d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:08:39.616702 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d20bd5-637e-48d3-a215-306e280c73d2 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:08:39.617201 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d20bd5-637e-48d3-a215-306e280c73d2 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:08:39.617667 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d20bd5-637e-48d3-a215-306e280c73d2 None admin] PATH_INFO: `/v3/users/swift` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:08:39.626081 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24d20bd5-637e-48d3-a215-306e280c73d2 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:08:39.627768 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24d20bd5-637e-48d3-a215-306e280c73d2 None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:08:39.640247 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application [None req-24d20bd5-637e-48d3-a215-306e280c73d2 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Sep 13 03:08:39.640247 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 03:08:39.640247 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 03:08:39.640247 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 03:08:39.640247 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 03:08:39.640247 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 03:08:39.640247 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 03:08:39.640247 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 03:08:39.640247 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 03:08:39.640247 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 03:08:39.640247 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 03:08:39.640247 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 03:08:39.640247 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 13 03:08:39.640247 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 13 03:08:39.640247 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 13 03:08:39.640247 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 13 03:08:39.641244 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 03:08:39.641244 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 03:08:39.641244 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 13 03:08:39.641244 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 13 03:08:39.641244 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 13 03:08:39.641244 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 13 03:08:39.641244 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 03:08:39.641244 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 03:08:39.641244 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 03:08:39.641244 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 03:08:39.641244 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 03:08:39.641244 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application with Lock( Sep 13 03:08:39.641244 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 03:08:39.641244 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._enter() Sep 13 03:08:39.641244 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 03:08:39.641244 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 03:08:39.641244 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 03:08:39.641244 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.creator() Sep 13 03:08:39.642199 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 03:08:39.642199 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application created_value = creator( Sep 13 03:08:39.642199 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 13 03:08:39.642199 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 13 03:08:39.642199 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 13 03:08:39.642199 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 13 03:08:39.642199 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 13 03:08:39.642199 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 13 03:08:39.642199 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Sep 13 03:08:39.642199 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Sep 13 03:08:39.644798 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 421/839] 10.222.0.18 () {56 vars in 1096 bytes} [Tue Sep 13 03:08:39 2022] GET /identity/v3/users/swift => generated 83 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 03:08:39.653088 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e405319b-244d-4230-8aef-2356ff0e0b24 None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:08:39.761348 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e405319b-244d-4230-8aef-2356ff0e0b24 None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:08:39.794980 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be72db4c-316d-43be-a3fc-90911a7285c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:08:39.796106 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be72db4c-316d-43be-a3fc-90911a7285c0 None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:08:39.796380 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be72db4c-316d-43be-a3fc-90911a7285c0 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:08:39.796646 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be72db4c-316d-43be-a3fc-90911a7285c0 None admin] PATH_INFO: `/v3/users` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:08:39.797370 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be72db4c-316d-43be-a3fc-90911a7285c0 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 03:08:39.797652 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be72db4c-316d-43be-a3fc-90911a7285c0 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:08:39.799098 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be72db4c-316d-43be-a3fc-90911a7285c0 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:08:39.817025 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 419/840] 10.222.0.18 () {56 vars in 1135 bytes} [Tue Sep 13 03:08:39 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 03:08:39.822490 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24d20bd5-637e-48d3-a215-306e280c73d2 None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:08:39.873193 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:08:39.927827 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24d20bd5-637e-48d3-a215-306e280c73d2 None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:08:39.962812 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-364507d0-e448-4699-ad50-123bba99e889 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:08:39.963415 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-364507d0-e448-4699-ad50-123bba99e889 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:08:39.963686 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-364507d0-e448-4699-ad50-123bba99e889 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:08:39.963951 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-364507d0-e448-4699-ad50-123bba99e889 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:08:40.042479 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-364507d0-e448-4699-ad50-123bba99e889 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:08:40.044315 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-364507d0-e448-4699-ad50-123bba99e889 None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:08:40.094578 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:08:40.125142 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05a90253-193a-4372-a141-e58dd8cee9e1 None None] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:08:40.125640 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05a90253-193a-4372-a141-e58dd8cee9e1 None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:08:40.125718 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05a90253-193a-4372-a141-e58dd8cee9e1 None None] PATH_INFO: `/` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:08:40.127268 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 420/841] 10.222.0.18 () {54 vars in 927 bytes} [Tue Sep 13 03:08:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 03:08:40.135648 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dcca177-224f-4f4f-956f-0b167800ed94 None None] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:08:40.135922 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dcca177-224f-4f4f-956f-0b167800ed94 None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:08:40.136189 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dcca177-224f-4f4f-956f-0b167800ed94 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:08:40.156389 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.auth.core [None req-5dcca177-224f-4f4f-956f-0b167800ed94 None None] MFA Rules not processed for user `49f280ab086b4610855c4ffa378517d9`. Rule list: `[]` (Enabled: `True`). {{(pid=81399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 03:08:40.157552 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-5dcca177-224f-4f4f-956f-0b167800ed94 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:08:40.162446 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 422/842] 10.222.0.18 () {58 vars in 1282 bytes} [Tue Sep 13 03:08:39 2022] GET /identity/v3/auth/tokens => generated 1161 bytes in 340 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 13 03:08:40.167929 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-364507d0-e448-4699-ad50-123bba99e889 None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:08:40.195140 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:08:40.240331 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 421/843] 10.222.0.18 () {58 vars in 1024 bytes} [Tue Sep 13 03:08:40 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 13 03:08:40.272493 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-364507d0-e448-4699-ad50-123bba99e889 None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:08:40.421377 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6be55802-d449-4e11-941f-65cb3cabc282 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:08:40.421928 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6be55802-d449-4e11-941f-65cb3cabc282 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:08:40.422198 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6be55802-d449-4e11-941f-65cb3cabc282 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:08:40.422463 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6be55802-d449-4e11-941f-65cb3cabc282 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:08:40.432731 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6be55802-d449-4e11-941f-65cb3cabc282 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:08:40.434207 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6be55802-d449-4e11-941f-65cb3cabc282 None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:08:40.442848 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:08:40.449500 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application [None req-6be55802-d449-4e11-941f-65cb3cabc282 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 03:08:40.449500 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 03:08:40.449500 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 03:08:40.449500 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 03:08:40.449500 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 03:08:40.449500 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 03:08:40.449500 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 03:08:40.449500 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 03:08:40.449500 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 03:08:40.449500 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 03:08:40.449500 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 03:08:40.449500 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 03:08:40.449500 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 13 03:08:40.449500 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 13 03:08:40.449500 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 13 03:08:40.449500 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 13 03:08:40.450688 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 03:08:40.450688 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 03:08:40.450688 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 03:08:40.450688 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 03:08:40.450688 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 03:08:40.450688 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 03:08:40.450688 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 03:08:40.450688 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application with Lock( Sep 13 03:08:40.450688 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 03:08:40.450688 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._enter() Sep 13 03:08:40.450688 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 03:08:40.450688 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 03:08:40.450688 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 03:08:40.450688 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.creator() Sep 13 03:08:40.450688 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 03:08:40.450688 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application created_value = creator( Sep 13 03:08:40.450688 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 13 03:08:40.452234 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 13 03:08:40.452234 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 03:08:40.452234 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Sep 13 03:08:40.454227 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 423/844] 10.222.0.18 () {56 vars in 1108 bytes} [Tue Sep 13 03:08:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 286 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 13 03:08:40.461771 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dcca177-224f-4f4f-956f-0b167800ed94 None None] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:08:40.513506 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6be55802-d449-4e11-941f-65cb3cabc282 None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:08:40.573894 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dcca177-224f-4f4f-956f-0b167800ed94 None None] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:08:40.613495 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6be55802-d449-4e11-941f-65cb3cabc282 None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:08:40.614515 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c83e66ad-ae29-467c-8313-9234b408bf15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:08:40.615391 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c83e66ad-ae29-467c-8313-9234b408bf15 None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:08:40.616580 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c83e66ad-ae29-467c-8313-9234b408bf15 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:08:40.616949 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c83e66ad-ae29-467c-8313-9234b408bf15 None admin] PATH_INFO: `/v3/domains` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:08:40.617665 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c83e66ad-ae29-467c-8313-9234b408bf15 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 03:08:40.618032 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c83e66ad-ae29-467c-8313-9234b408bf15 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:08:40.619534 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c83e66ad-ae29-467c-8313-9234b408bf15 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:08:40.634132 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 422/845] 10.222.0.18 () {56 vars in 1109 bytes} [Tue Sep 13 03:08:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 03:08:40.640194 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c83e66ad-ae29-467c-8313-9234b408bf15 None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:08:40.656419 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b14d561-73cc-49bf-b113-e71147700618 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:08:40.657471 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b14d561-73cc-49bf-b113-e71147700618 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:08:40.657838 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b14d561-73cc-49bf-b113-e71147700618 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:08:40.658729 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b14d561-73cc-49bf-b113-e71147700618 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:08:40.659463 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b14d561-73cc-49bf-b113-e71147700618 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:08:40.661474 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b14d561-73cc-49bf-b113-e71147700618 None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:08:40.671290 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:08:40.675303 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application [None req-5b14d561-73cc-49bf-b113-e71147700618 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 13 03:08:40.675303 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 03:08:40.675303 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 13 03:08:40.675303 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 13 03:08:40.675303 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 13 03:08:40.675303 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 13 03:08:40.675303 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 13 03:08:40.675303 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 13 03:08:40.675303 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 13 03:08:40.675303 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Sep 13 03:08:40.675303 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 13 03:08:40.675303 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Sep 13 03:08:40.675303 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 03:08:40.675303 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 03:08:40.675303 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 03:08:40.675303 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 03:08:40.676346 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 03:08:40.676346 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 03:08:40.676346 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 03:08:40.676346 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 03:08:40.676346 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 03:08:40.676346 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 03:08:40.676346 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 03:08:40.676346 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 13 03:08:40.676346 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 13 03:08:40.676346 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 13 03:08:40.676346 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 13 03:08:40.676346 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 03:08:40.676346 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 03:08:40.676346 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 03:08:40.676346 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 03:08:40.676346 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 03:08:40.676346 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 03:08:40.677789 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 03:08:40.677789 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application with Lock( Sep 13 03:08:40.677789 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 03:08:40.677789 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._enter() Sep 13 03:08:40.677789 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 03:08:40.677789 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 03:08:40.677789 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 03:08:40.677789 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.creator() Sep 13 03:08:40.677789 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 03:08:40.677789 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application created_value = creator( Sep 13 03:08:40.677789 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 13 03:08:40.677789 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 13 03:08:40.677789 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 13 03:08:40.677789 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Sep 13 03:08:40.677789 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 424/846] 10.222.0.18 () {56 vars in 1132 bytes} [Tue Sep 13 03:08:40 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 13 03:08:40.683521 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b14d561-73cc-49bf-b113-e71147700618 None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:08:40.749831 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:08:40.772328 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c83e66ad-ae29-467c-8313-9234b408bf15 None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:08:40.791924 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b14d561-73cc-49bf-b113-e71147700618 None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:08:40.820303 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-971a6fe5-130d-486c-8401-5cd5e189a433 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:08:40.820969 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-971a6fe5-130d-486c-8401-5cd5e189a433 None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:08:40.821244 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-971a6fe5-130d-486c-8401-5cd5e189a433 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:08:40.821512 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-971a6fe5-130d-486c-8401-5cd5e189a433 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:08:40.831579 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-971a6fe5-130d-486c-8401-5cd5e189a433 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:08:40.832987 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-971a6fe5-130d-486c-8401-5cd5e189a433 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:08:40.833277 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff62fc6e-e2ee-4acd-9c59-db12b1acd066 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:08:40.834267 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff62fc6e-e2ee-4acd-9c59-db12b1acd066 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:08:40.834789 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff62fc6e-e2ee-4acd-9c59-db12b1acd066 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:08:40.835264 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff62fc6e-e2ee-4acd-9c59-db12b1acd066 None admin] PATH_INFO: `/v3/services` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:08:40.836085 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff62fc6e-e2ee-4acd-9c59-db12b1acd066 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=81400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 03:08:40.836605 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff62fc6e-e2ee-4acd-9c59-db12b1acd066 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:08:40.838146 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff62fc6e-e2ee-4acd-9c59-db12b1acd066 None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:08:40.843417 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 425/847] 10.222.0.18 () {56 vars in 1126 bytes} [Tue Sep 13 03:08:40 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 03:08:40.845686 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application [None req-971a6fe5-130d-486c-8401-5cd5e189a433 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 13 03:08:40.845686 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 03:08:40.845686 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 03:08:40.845686 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 03:08:40.845686 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 03:08:40.845686 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 03:08:40.845686 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 03:08:40.845686 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 03:08:40.845686 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 03:08:40.845686 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 03:08:40.845686 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 03:08:40.845686 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 03:08:40.845686 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 13 03:08:40.845686 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 13 03:08:40.845686 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 13 03:08:40.846583 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 13 03:08:40.846583 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 03:08:40.846583 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 03:08:40.846583 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 03:08:40.846583 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 03:08:40.846583 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 03:08:40.846583 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 03:08:40.846583 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 03:08:40.846583 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application with Lock( Sep 13 03:08:40.846583 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 03:08:40.846583 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._enter() Sep 13 03:08:40.846583 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 03:08:40.846583 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 03:08:40.846583 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 03:08:40.846583 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.creator() Sep 13 03:08:40.846583 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 03:08:40.846583 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application created_value = creator( Sep 13 03:08:40.848215 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 13 03:08:40.848215 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 13 03:08:40.848215 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 13 03:08:40.848215 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 13 03:08:40.848215 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 13 03:08:40.848215 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 13 03:08:40.848215 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 13 03:08:40.848215 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Sep 13 03:08:40.850676 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 423/848] 10.222.0.18 () {56 vars in 1111 bytes} [Tue Sep 13 03:08:40 2022] GET /identity/v3/projects/service => generated 88 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 13 03:08:40.852701 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff62fc6e-e2ee-4acd-9c59-db12b1acd066 None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:08:40.853867 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-971a6fe5-130d-486c-8401-5cd5e189a433 None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:08:40.958326 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:08:40.970953 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff62fc6e-e2ee-4acd-9c59-db12b1acd066 None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:08:40.975153 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-971a6fe5-130d-486c-8401-5cd5e189a433 None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:08:40.999456 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:08:41.008859 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3702d822-0080-46ab-8ba3-1144fe392e47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:08:41.009690 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3702d822-0080-46ab-8ba3-1144fe392e47 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:08:41.009968 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3702d822-0080-46ab-8ba3-1144fe392e47 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:08:41.010071 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2453a25f-5222-486f-971e-13ff59917b8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:08:41.010253 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3702d822-0080-46ab-8ba3-1144fe392e47 None admin] PATH_INFO: `/v3/services` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:08:41.010581 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2453a25f-5222-486f-971e-13ff59917b8e None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:08:41.010887 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2453a25f-5222-486f-971e-13ff59917b8e None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:08:41.010983 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3702d822-0080-46ab-8ba3-1144fe392e47 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=81400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 03:08:41.011145 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2453a25f-5222-486f-971e-13ff59917b8e None admin] PATH_INFO: `/v3/projects` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:08:41.011249 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3702d822-0080-46ab-8ba3-1144fe392e47 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:08:41.011817 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2453a25f-5222-486f-971e-13ff59917b8e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 03:08:41.012114 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2453a25f-5222-486f-971e-13ff59917b8e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:08:41.012538 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3702d822-0080-46ab-8ba3-1144fe392e47 None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:08:41.013513 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2453a25f-5222-486f-971e-13ff59917b8e None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:08:41.018121 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 426/849] 10.222.0.18 () {56 vars in 1126 bytes} [Tue Sep 13 03:08:40 2022] GET /identity/v3/services?type=compute_legacy => generated 131 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 03:08:41.019367 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-839ca406-2932-40e9-8d2f-e7bef7660fe6 None None] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:08:41.019656 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-839ca406-2932-40e9-8d2f-e7bef7660fe6 None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:08:41.019930 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-839ca406-2932-40e9-8d2f-e7bef7660fe6 None None] PATH_INFO: `/` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:08:41.021825 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 427/850] 10.222.0.18 () {54 vars in 927 bytes} [Tue Sep 13 03:08:41 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 03:08:41.022289 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-866bc5c7-f3d2-4692-8613-717627259246 None None] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:08:41.022642 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-866bc5c7-f3d2-4692-8613-717627259246 None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:08:41.022979 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-866bc5c7-f3d2-4692-8613-717627259246 None None] PATH_INFO: `/` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:08:41.024413 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 428/851] 10.222.0.18 () {54 vars in 927 bytes} [Tue Sep 13 03:08:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 03:08:41.028877 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c70097-e387-4869-9176-391466b87bc6 None None] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:08:41.029395 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c70097-e387-4869-9176-391466b87bc6 None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:08:41.029717 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c70097-e387-4869-9176-391466b87bc6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:08:41.036138 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 424/852] 10.222.0.18 () {56 vars in 1148 bytes} [Tue Sep 13 03:08:40 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 03:08:41.036138 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-321558c9-27c8-43d3-aea1-e284606e0501 None None] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:08:41.036138 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-321558c9-27c8-43d3-aea1-e284606e0501 None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:08:41.036138 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-321558c9-27c8-43d3-aea1-e284606e0501 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:08:41.051555 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.auth.core [None req-21c70097-e387-4869-9176-391466b87bc6 None None] MFA Rules not processed for user `49f280ab086b4610855c4ffa378517d9`. Rule list: `[]` (Enabled: `True`). {{(pid=81400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 03:08:41.051803 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.auth.core [None req-321558c9-27c8-43d3-aea1-e284606e0501 None None] MFA Rules not processed for user `49f280ab086b4610855c4ffa378517d9`. Rule list: `[]` (Enabled: `True`). {{(pid=81399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 03:08:41.052791 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-21c70097-e387-4869-9176-391466b87bc6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:08:41.052966 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-321558c9-27c8-43d3-aea1-e284606e0501 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:08:41.128976 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 429/853] 10.222.0.18 () {58 vars in 1024 bytes} [Tue Sep 13 03:08:41 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 13 03:08:41.130280 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21c70097-e387-4869-9176-391466b87bc6 None None] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:08:41.136582 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 425/854] 10.222.0.18 () {58 vars in 1024 bytes} [Tue Sep 13 03:08:41 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 13 03:08:41.237309 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21c70097-e387-4869-9176-391466b87bc6 None None] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:08:41.261187 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:08:41.273201 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bff5a73-1229-43b9-a642-1d9c2fef6202 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:08:41.273805 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bff5a73-1229-43b9-a642-1d9c2fef6202 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:08:41.274078 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bff5a73-1229-43b9-a642-1d9c2fef6202 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:08:41.274339 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bff5a73-1229-43b9-a642-1d9c2fef6202 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:08:41.275077 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bff5a73-1229-43b9-a642-1d9c2fef6202 None admin] RBAC: Adding query filter params (role.id=425d4b481c7846e0bc07e7e805d5b15f, scope.project.id=b9d94f6a68f7470d9494f112175620dc, user.id=893b0c0d85964b7dbc9705c0a091682c) {{(pid=81400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 03:08:41.275374 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bff5a73-1229-43b9-a642-1d9c2fef6202 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:08:41.276779 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bff5a73-1229-43b9-a642-1d9c2fef6202 None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:08:41.282838 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 430/855] 10.222.0.18 () {56 vars in 1374 bytes} [Tue Sep 13 03:08:41 2022] GET /identity/v3/role_assignments?user.id=893b0c0d85964b7dbc9705c0a091682c&scope.project.id=b9d94f6a68f7470d9494f112175620dc&role.id=425d4b481c7846e0bc07e7e805d5b15f => generated 608 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 03:08:41.402014 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-321558c9-27c8-43d3-aea1-e284606e0501 None None] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:08:41.412717 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bff5a73-1229-43b9-a642-1d9c2fef6202 None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:08:41.498681 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:08:41.526849 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-321558c9-27c8-43d3-aea1-e284606e0501 None None] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:08:41.538505 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bff5a73-1229-43b9-a642-1d9c2fef6202 None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:08:41.565649 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cae67a6-ac3f-4767-a305-581a32f50bae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:08:41.566284 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cae67a6-ac3f-4767-a305-581a32f50bae None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:08:41.566565 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cae67a6-ac3f-4767-a305-581a32f50bae None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:08:41.566855 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cae67a6-ac3f-4767-a305-581a32f50bae None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:08:41.567529 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cae67a6-ac3f-4767-a305-581a32f50bae None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:08:41.568808 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cae67a6-ac3f-4767-a305-581a32f50bae None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:08:41.579326 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application [None req-8cae67a6-ac3f-4767-a305-581a32f50bae None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 13 03:08:41.579326 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 03:08:41.579326 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 13 03:08:41.579326 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 13 03:08:41.579326 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 13 03:08:41.579326 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 13 03:08:41.579326 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 13 03:08:41.579326 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 13 03:08:41.579326 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 13 03:08:41.579326 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Sep 13 03:08:41.579326 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 13 03:08:41.579326 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Sep 13 03:08:41.579326 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 03:08:41.579326 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 03:08:41.579326 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 03:08:41.579326 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 03:08:41.579326 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 03:08:41.580347 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 03:08:41.580347 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 03:08:41.580347 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 03:08:41.580347 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 03:08:41.580347 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 03:08:41.580347 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 03:08:41.580347 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 13 03:08:41.580347 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 13 03:08:41.580347 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 13 03:08:41.580347 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 13 03:08:41.580347 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 03:08:41.580347 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 03:08:41.580347 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 03:08:41.580347 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 03:08:41.580347 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 03:08:41.580347 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 03:08:41.581211 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 03:08:41.581211 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application with Lock( Sep 13 03:08:41.581211 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 03:08:41.581211 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._enter() Sep 13 03:08:41.581211 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 03:08:41.581211 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 03:08:41.581211 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 03:08:41.581211 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.creator() Sep 13 03:08:41.581211 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 03:08:41.581211 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application created_value = creator( Sep 13 03:08:41.581211 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 13 03:08:41.581211 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 13 03:08:41.581211 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 13 03:08:41.581211 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Sep 13 03:08:41.581211 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 426/856] 10.222.0.18 () {56 vars in 1114 bytes} [Tue Sep 13 03:08:41 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 03:08:41.586053 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cae67a6-ac3f-4767-a305-581a32f50bae None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:08:41.586573 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c5ff5cb-2312-4a53-b15d-df8baefde2c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:08:41.587463 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c5ff5cb-2312-4a53-b15d-df8baefde2c0 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:08:41.587930 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c5ff5cb-2312-4a53-b15d-df8baefde2c0 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:08:41.588410 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c5ff5cb-2312-4a53-b15d-df8baefde2c0 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:08:41.603280 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c5ff5cb-2312-4a53-b15d-df8baefde2c0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 03:08:41.610073 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application [None req-0c5ff5cb-2312-4a53-b15d-df8baefde2c0 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 13 03:08:41.610073 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 03:08:41.610073 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 03:08:41.610073 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 03:08:41.610073 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 03:08:41.610073 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 03:08:41.610073 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 03:08:41.610073 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 03:08:41.610073 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 03:08:41.610073 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 03:08:41.610073 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 03:08:41.610073 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 03:08:41.610073 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 03:08:41.610073 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 03:08:41.610073 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 03:08:41.610073 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 03:08:41.611038 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 03:08:41.611038 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 03:08:41.611038 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 03:08:41.611038 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 03:08:41.611038 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 03:08:41.611038 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 03:08:41.611038 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 03:08:41.611038 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 03:08:41.611038 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 03:08:41.611038 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 03:08:41.611038 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 03:08:41.611038 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application with Lock( Sep 13 03:08:41.611038 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 03:08:41.611038 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._enter() Sep 13 03:08:41.611038 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 03:08:41.611038 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 03:08:41.611875 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 03:08:41.611875 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.creator() Sep 13 03:08:41.611875 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 03:08:41.611875 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application created_value = creator( Sep 13 03:08:41.611875 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 03:08:41.611875 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 03:08:41.611875 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 03:08:41.611875 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 03:08:41.611875 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 03:08:41.611875 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 03:08:41.611875 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 13 03:08:41.611875 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Sep 13 03:08:41.612512 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 431/857] 10.222.0.18 () {56 vars in 1102 bytes} [Tue Sep 13 03:08:41 2022] GET /identity/v3/roles/service => generated 85 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 03:08:41.618858 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c5ff5cb-2312-4a53-b15d-df8baefde2c0 None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:08:41.682636 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:08:41.718841 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cae67a6-ac3f-4767-a305-581a32f50bae None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:08:41.755470 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-195797aa-5274-4e28-8ae6-6f7363fddd43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:08:41.755998 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-195797aa-5274-4e28-8ae6-6f7363fddd43 None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:08:41.756306 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-195797aa-5274-4e28-8ae6-6f7363fddd43 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:08:41.756599 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-195797aa-5274-4e28-8ae6-6f7363fddd43 None admin] PATH_INFO: `/v3/services` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:08:41.757227 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-195797aa-5274-4e28-8ae6-6f7363fddd43 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=81399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 03:08:41.757530 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-195797aa-5274-4e28-8ae6-6f7363fddd43 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:08:41.758921 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-195797aa-5274-4e28-8ae6-6f7363fddd43 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:08:41.761735 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c5ff5cb-2312-4a53-b15d-df8baefde2c0 None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:08:41.769296 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 427/858] 10.222.0.18 () {56 vars in 1114 bytes} [Tue Sep 13 03:08:41 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 03:08:41.770167 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-195797aa-5274-4e28-8ae6-6f7363fddd43 None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:08:41.774671 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:08:41.811567 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3dcaf80-5cbf-4338-9a73-0b581c1d2d8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:08:41.812562 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3dcaf80-5cbf-4338-9a73-0b581c1d2d8f None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:08:41.813038 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3dcaf80-5cbf-4338-9a73-0b581c1d2d8f None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:08:41.814018 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3dcaf80-5cbf-4338-9a73-0b581c1d2d8f None admin] PATH_INFO: `/v3/roles` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:08:41.814907 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3dcaf80-5cbf-4338-9a73-0b581c1d2d8f None admin] RBAC: Adding query filter params (name=service) {{(pid=81400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 03:08:41.815402 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3dcaf80-5cbf-4338-9a73-0b581c1d2d8f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:08:41.817793 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3dcaf80-5cbf-4338-9a73-0b581c1d2d8f None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:08:41.834492 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 432/859] 10.222.0.18 () {56 vars in 1103 bytes} [Tue Sep 13 03:08:41 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 03:08:41.835341 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3dcaf80-5cbf-4338-9a73-0b581c1d2d8f None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:08:41.897572 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-195797aa-5274-4e28-8ae6-6f7363fddd43 None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:08:41.947708 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caba4a51-15ac-4043-af41-d8356c42b6f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:08:41.948501 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caba4a51-15ac-4043-af41-d8356c42b6f3 None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:08:41.948876 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caba4a51-15ac-4043-af41-d8356c42b6f3 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:08:41.949248 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caba4a51-15ac-4043-af41-d8356c42b6f3 None admin] PATH_INFO: `/v3/services` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:08:41.950060 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caba4a51-15ac-4043-af41-d8356c42b6f3 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=81399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 03:08:41.950468 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caba4a51-15ac-4043-af41-d8356c42b6f3 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:08:41.950738 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3dcaf80-5cbf-4338-9a73-0b581c1d2d8f None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:08:41.952660 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caba4a51-15ac-4043-af41-d8356c42b6f3 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:08:41.962832 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:08:41.963718 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 428/860] 10.222.0.18 () {56 vars in 1114 bytes} [Tue Sep 13 03:08:41 2022] GET /identity/v3/services?type=volumev3 => generated 125 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 03:08:41.994723 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd530862-3582-4346-9ce2-87f9f9e99bbd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:08:41.995721 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd530862-3582-4346-9ce2-87f9f9e99bbd None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:08:41.996225 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd530862-3582-4346-9ce2-87f9f9e99bbd None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:08:41.996637 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd530862-3582-4346-9ce2-87f9f9e99bbd None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:08:42.008936 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd530862-3582-4346-9ce2-87f9f9e99bbd None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:08:42.010737 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd530862-3582-4346-9ce2-87f9f9e99bbd None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:08:42.028088 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application [None req-fd530862-3582-4346-9ce2-87f9f9e99bbd None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 03:08:42.028088 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 03:08:42.028088 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 03:08:42.028088 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 03:08:42.028088 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 03:08:42.028088 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 03:08:42.028088 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 03:08:42.028088 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 03:08:42.028088 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 03:08:42.028088 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 03:08:42.028088 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 03:08:42.028088 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 03:08:42.028088 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 13 03:08:42.028088 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 13 03:08:42.028088 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 13 03:08:42.028088 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 13 03:08:42.029316 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 03:08:42.029316 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 03:08:42.029316 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 03:08:42.029316 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 03:08:42.029316 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 03:08:42.029316 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 03:08:42.029316 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 03:08:42.029316 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application with Lock( Sep 13 03:08:42.029316 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 03:08:42.029316 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._enter() Sep 13 03:08:42.029316 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 03:08:42.029316 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 03:08:42.029316 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 03:08:42.029316 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.creator() Sep 13 03:08:42.029316 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 03:08:42.029316 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application created_value = creator( Sep 13 03:08:42.029316 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 13 03:08:42.031020 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 13 03:08:42.031020 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 03:08:42.031020 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Sep 13 03:08:42.032787 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 433/861] 10.222.0.18 () {56 vars in 1108 bytes} [Tue Sep 13 03:08:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 13 03:08:42.038838 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caba4a51-15ac-4043-af41-d8356c42b6f3 None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:08:42.116890 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:08:42.155270 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caba4a51-15ac-4043-af41-d8356c42b6f3 None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:08:42.191240 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5f839e2-9423-409c-99ed-f8ca7917de8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:08:42.192085 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f839e2-9423-409c-99ed-f8ca7917de8a None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:08:42.192504 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f839e2-9423-409c-99ed-f8ca7917de8a None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:08:42.192913 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f839e2-9423-409c-99ed-f8ca7917de8a None admin] PATH_INFO: `/v3/domains` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:08:42.193708 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5f839e2-9423-409c-99ed-f8ca7917de8a None admin] RBAC: Adding query filter params (name=Default) {{(pid=81399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 03:08:42.194148 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5f839e2-9423-409c-99ed-f8ca7917de8a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:08:42.195677 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5f839e2-9423-409c-99ed-f8ca7917de8a None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:08:42.209135 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 429/862] 10.222.0.18 () {56 vars in 1109 bytes} [Tue Sep 13 03:08:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 03:08:42.214853 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd530862-3582-4346-9ce2-87f9f9e99bbd None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:08:42.323524 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd530862-3582-4346-9ce2-87f9f9e99bbd None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:08:42.358432 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0783e4d8-18f8-4a75-bb3e-68bbe6c205f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:08:42.359084 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0783e4d8-18f8-4a75-bb3e-68bbe6c205f3 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:08:42.359356 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0783e4d8-18f8-4a75-bb3e-68bbe6c205f3 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:08:42.359623 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0783e4d8-18f8-4a75-bb3e-68bbe6c205f3 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:08:42.367213 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0783e4d8-18f8-4a75-bb3e-68bbe6c205f3 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:08:42.368634 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0783e4d8-18f8-4a75-bb3e-68bbe6c205f3 None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:08:42.380350 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application [None req-0783e4d8-18f8-4a75-bb3e-68bbe6c205f3 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 13 03:08:42.380350 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 03:08:42.380350 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 03:08:42.380350 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 03:08:42.380350 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 03:08:42.380350 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 03:08:42.380350 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 03:08:42.380350 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 03:08:42.380350 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 03:08:42.380350 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 03:08:42.380350 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 03:08:42.380350 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 03:08:42.380350 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 13 03:08:42.380350 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 13 03:08:42.380350 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 13 03:08:42.380350 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 13 03:08:42.381328 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 03:08:42.381328 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 03:08:42.381328 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 13 03:08:42.381328 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 13 03:08:42.381328 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 13 03:08:42.381328 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 13 03:08:42.381328 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 03:08:42.381328 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 03:08:42.381328 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 03:08:42.381328 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 03:08:42.381328 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 03:08:42.381328 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application with Lock( Sep 13 03:08:42.381328 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 03:08:42.381328 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._enter() Sep 13 03:08:42.381328 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 03:08:42.381328 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 03:08:42.381328 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 03:08:42.381328 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.creator() Sep 13 03:08:42.382264 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 03:08:42.382264 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application created_value = creator( Sep 13 03:08:42.382264 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 13 03:08:42.382264 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 13 03:08:42.382264 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 13 03:08:42.382264 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 13 03:08:42.382264 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 13 03:08:42.382264 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 13 03:08:42.382264 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 13 03:08:42.382264 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Sep 13 03:08:42.382264 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 434/863] 10.222.0.18 () {56 vars in 1117 bytes} [Tue Sep 13 03:08:42 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 03:08:42.389699 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5f839e2-9423-409c-99ed-f8ca7917de8a None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:08:42.492549 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5f839e2-9423-409c-99ed-f8ca7917de8a None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:08:42.520767 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:08:42.528942 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5803dcb4-d50e-4ad4-a920-bc1f4aca133f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:08:42.529873 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5803dcb4-d50e-4ad4-a920-bc1f4aca133f None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:08:42.530368 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5803dcb4-d50e-4ad4-a920-bc1f4aca133f None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:08:42.530883 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5803dcb4-d50e-4ad4-a920-bc1f4aca133f None admin] PATH_INFO: `/v3/users` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:08:42.531790 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5803dcb4-d50e-4ad4-a920-bc1f4aca133f None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 03:08:42.532305 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5803dcb4-d50e-4ad4-a920-bc1f4aca133f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:08:42.533900 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5803dcb4-d50e-4ad4-a920-bc1f4aca133f None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:08:42.552683 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 430/864] 10.222.0.18 () {56 vars in 1149 bytes} [Tue Sep 13 03:08:42 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 03:08:42.558676 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0783e4d8-18f8-4a75-bb3e-68bbe6c205f3 None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:08:42.583074 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:08:42.666189 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0783e4d8-18f8-4a75-bb3e-68bbe6c205f3 None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:08:42.701909 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d25000f9-06dd-4234-bedc-0762cffe5e43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:08:42.702511 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d25000f9-06dd-4234-bedc-0762cffe5e43 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:08:42.702806 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d25000f9-06dd-4234-bedc-0762cffe5e43 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:08:42.703068 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d25000f9-06dd-4234-bedc-0762cffe5e43 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:08:42.776013 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d25000f9-06dd-4234-bedc-0762cffe5e43 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:08:42.777805 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d25000f9-06dd-4234-bedc-0762cffe5e43 None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:08:42.798952 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:08:42.880825 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 435/865] 10.222.0.18 () {58 vars in 1282 bytes} [Tue Sep 13 03:08:42 2022] GET /identity/v3/auth/tokens => generated 1161 bytes in 322 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 13 03:08:42.886467 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5803dcb4-d50e-4ad4-a920-bc1f4aca133f None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:08:42.994299 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5803dcb4-d50e-4ad4-a920-bc1f4aca133f None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:08:43.037419 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-287536a5-1ddf-4888-9293-04793507cdd4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:08:43.038412 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-287536a5-1ddf-4888-9293-04793507cdd4 None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:08:43.038931 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-287536a5-1ddf-4888-9293-04793507cdd4 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:08:43.039402 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-287536a5-1ddf-4888-9293-04793507cdd4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:08:43.050493 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-287536a5-1ddf-4888-9293-04793507cdd4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:08:43.052244 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-287536a5-1ddf-4888-9293-04793507cdd4 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:08:43.067990 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application [None req-287536a5-1ddf-4888-9293-04793507cdd4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 03:08:43.067990 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 03:08:43.067990 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 03:08:43.067990 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 03:08:43.067990 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 03:08:43.067990 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 03:08:43.067990 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 03:08:43.067990 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 03:08:43.067990 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 03:08:43.067990 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 03:08:43.067990 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 03:08:43.067990 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 03:08:43.067990 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 13 03:08:43.067990 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 13 03:08:43.067990 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 13 03:08:43.067990 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 13 03:08:43.068978 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 03:08:43.068978 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 03:08:43.068978 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 03:08:43.068978 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 03:08:43.068978 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 03:08:43.068978 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 03:08:43.068978 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 03:08:43.068978 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application with Lock( Sep 13 03:08:43.068978 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 03:08:43.068978 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._enter() Sep 13 03:08:43.068978 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 03:08:43.068978 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 03:08:43.068978 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 03:08:43.068978 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.creator() Sep 13 03:08:43.068978 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 03:08:43.068978 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application created_value = creator( Sep 13 03:08:43.068978 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 13 03:08:43.069891 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 13 03:08:43.069891 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 03:08:43.069891 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Sep 13 03:08:43.071984 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 431/866] 10.222.0.18 () {56 vars in 1108 bytes} [Tue Sep 13 03:08:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 03:08:43.077725 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d25000f9-06dd-4234-bedc-0762cffe5e43 None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:08:43.186089 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d25000f9-06dd-4234-bedc-0762cffe5e43 None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:08:43.206681 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:08:43.222351 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-426e48b7-178b-40e5-86e5-8b36c80b1f55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:08:43.222986 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-426e48b7-178b-40e5-86e5-8b36c80b1f55 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:08:43.223261 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-426e48b7-178b-40e5-86e5-8b36c80b1f55 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:08:43.223571 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-426e48b7-178b-40e5-86e5-8b36c80b1f55 None admin] PATH_INFO: `/v3/domains` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:08:43.224225 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-426e48b7-178b-40e5-86e5-8b36c80b1f55 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 03:08:43.224524 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-426e48b7-178b-40e5-86e5-8b36c80b1f55 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:08:43.225883 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-426e48b7-178b-40e5-86e5-8b36c80b1f55 None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:08:43.238376 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 436/867] 10.222.0.18 () {56 vars in 1109 bytes} [Tue Sep 13 03:08:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 03:08:43.243619 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-287536a5-1ddf-4888-9293-04793507cdd4 None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:08:43.277035 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f2185b7-1b96-4c3f-af98-8af4bc9207fe None None] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:08:43.277466 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f2185b7-1b96-4c3f-af98-8af4bc9207fe None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:08:43.277843 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f2185b7-1b96-4c3f-af98-8af4bc9207fe None None] PATH_INFO: `/` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:08:43.280132 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 437/868] 10.222.0.18 () {54 vars in 927 bytes} [Tue Sep 13 03:08:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 13 03:08:43.285343 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af854b28-24a6-4d86-8a45-45df1b00ee0d None None] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:08:43.285871 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af854b28-24a6-4d86-8a45-45df1b00ee0d None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:08:43.286322 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af854b28-24a6-4d86-8a45-45df1b00ee0d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:08:43.287344 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:08:43.307951 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.auth.core [None req-af854b28-24a6-4d86-8a45-45df1b00ee0d None None] MFA Rules not processed for user `49f280ab086b4610855c4ffa378517d9`. Rule list: `[]` (Enabled: `True`). {{(pid=81400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 03:08:43.309297 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-af854b28-24a6-4d86-8a45-45df1b00ee0d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:08:43.359337 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-287536a5-1ddf-4888-9293-04793507cdd4 None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:08:43.395315 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e35f132-ec1e-4518-904a-964ecb384406 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:08:43.396252 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e35f132-ec1e-4518-904a-964ecb384406 None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:08:43.396711 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e35f132-ec1e-4518-904a-964ecb384406 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:08:43.397163 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e35f132-ec1e-4518-904a-964ecb384406 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:08:43.402635 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 438/869] 10.222.0.18 () {58 vars in 1024 bytes} [Tue Sep 13 03:08:43 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 13 03:08:43.411092 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e35f132-ec1e-4518-904a-964ecb384406 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:08:43.412516 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e35f132-ec1e-4518-904a-964ecb384406 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:08:43.424079 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application [None req-6e35f132-ec1e-4518-904a-964ecb384406 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 13 03:08:43.424079 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 03:08:43.424079 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 03:08:43.424079 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 03:08:43.424079 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 03:08:43.424079 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 03:08:43.424079 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 03:08:43.424079 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 03:08:43.424079 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 03:08:43.424079 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 03:08:43.424079 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 03:08:43.424079 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 03:08:43.424079 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 13 03:08:43.424079 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 13 03:08:43.424079 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 13 03:08:43.424977 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 13 03:08:43.424977 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 03:08:43.424977 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 03:08:43.424977 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 03:08:43.424977 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 03:08:43.424977 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 03:08:43.424977 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 03:08:43.424977 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 03:08:43.424977 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application with Lock( Sep 13 03:08:43.424977 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 03:08:43.424977 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._enter() Sep 13 03:08:43.424977 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 03:08:43.424977 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 03:08:43.424977 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 03:08:43.424977 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.creator() Sep 13 03:08:43.424977 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 03:08:43.424977 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application created_value = creator( Sep 13 03:08:43.425896 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 13 03:08:43.425896 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 13 03:08:43.425896 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 13 03:08:43.425896 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 13 03:08:43.425896 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 13 03:08:43.425896 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 13 03:08:43.425896 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 13 03:08:43.425896 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Sep 13 03:08:43.425896 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 432/870] 10.222.0.18 () {56 vars in 1111 bytes} [Tue Sep 13 03:08:43 2022] GET /identity/v3/projects/service => generated 88 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 03:08:43.432787 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af854b28-24a6-4d86-8a45-45df1b00ee0d None None] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:08:43.541193 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af854b28-24a6-4d86-8a45-45df1b00ee0d None None] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:08:43.552514 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:08:43.578001 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fabc858b-80cf-42aa-bc15-ea01a2f926c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:08:43.578631 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fabc858b-80cf-42aa-bc15-ea01a2f926c5 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:08:43.578908 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fabc858b-80cf-42aa-bc15-ea01a2f926c5 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:08:43.579175 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fabc858b-80cf-42aa-bc15-ea01a2f926c5 None admin] PATH_INFO: `/v3/projects` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:08:43.579860 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fabc858b-80cf-42aa-bc15-ea01a2f926c5 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 03:08:43.580162 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fabc858b-80cf-42aa-bc15-ea01a2f926c5 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:08:43.581588 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fabc858b-80cf-42aa-bc15-ea01a2f926c5 None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:08:43.594300 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 439/871] 10.222.0.18 () {56 vars in 1148 bytes} [Tue Sep 13 03:08:43 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 03:08:43.601472 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e35f132-ec1e-4518-904a-964ecb384406 None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:08:43.630109 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5cdd78e-337b-48a6-a00c-43cf599b9a7f None None] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:08:43.630423 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5cdd78e-337b-48a6-a00c-43cf599b9a7f None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:08:43.630704 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5cdd78e-337b-48a6-a00c-43cf599b9a7f None None] PATH_INFO: `/` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:08:43.632229 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 440/872] 10.222.0.18 () {54 vars in 927 bytes} [Tue Sep 13 03:08:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 13 03:08:43.637001 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa57d007-86f4-443c-ac9f-e056edb979d2 None None] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:08:43.637288 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa57d007-86f4-443c-ac9f-e056edb979d2 None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:08:43.637560 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa57d007-86f4-443c-ac9f-e056edb979d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:08:43.658675 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.auth.core [None req-aa57d007-86f4-443c-ac9f-e056edb979d2 None None] MFA Rules not processed for user `49f280ab086b4610855c4ffa378517d9`. Rule list: `[]` (Enabled: `True`). {{(pid=81400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 03:08:43.659854 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-aa57d007-86f4-443c-ac9f-e056edb979d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:08:43.715289 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:08:43.719275 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e35f132-ec1e-4518-904a-964ecb384406 None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:08:43.737779 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 441/873] 10.222.0.18 () {58 vars in 1024 bytes} [Tue Sep 13 03:08:43 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 13 03:08:43.738940 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa57d007-86f4-443c-ac9f-e056edb979d2 None None] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:08:43.765105 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed0d2759-7cf1-4313-82a9-440b49ace844 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:08:43.766515 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed0d2759-7cf1-4313-82a9-440b49ace844 None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:08:43.767062 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed0d2759-7cf1-4313-82a9-440b49ace844 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:08:43.767786 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed0d2759-7cf1-4313-82a9-440b49ace844 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:08:43.768872 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed0d2759-7cf1-4313-82a9-440b49ace844 None admin] RBAC: Adding query filter params (role.id=817d6837cd044890b41cf16b303a092b, scope.project.id=b9d94f6a68f7470d9494f112175620dc, user.id=4c2d20fb54a54ecc99b47f237be4917c) {{(pid=81399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 03:08:43.769765 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed0d2759-7cf1-4313-82a9-440b49ace844 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:08:43.771878 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed0d2759-7cf1-4313-82a9-440b49ace844 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:08:43.781057 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 433/874] 10.222.0.18 () {56 vars in 1374 bytes} [Tue Sep 13 03:08:43 2022] GET /identity/v3/role_assignments?user.id=4c2d20fb54a54ecc99b47f237be4917c&scope.project.id=b9d94f6a68f7470d9494f112175620dc&role.id=817d6837cd044890b41cf16b303a092b => generated 608 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 03:08:43.859850 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa57d007-86f4-443c-ac9f-e056edb979d2 None None] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:08:43.862952 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:08:43.900193 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4551afd0-a7af-4ad7-aa69-21da6a50bc2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:08:43.900937 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4551afd0-a7af-4ad7-aa69-21da6a50bc2a None admin] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:08:43.901220 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4551afd0-a7af-4ad7-aa69-21da6a50bc2a None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:08:43.901494 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4551afd0-a7af-4ad7-aa69-21da6a50bc2a None admin] PATH_INFO: `/v3/services` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:08:43.902161 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4551afd0-a7af-4ad7-aa69-21da6a50bc2a None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:08:43.903579 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4551afd0-a7af-4ad7-aa69-21da6a50bc2a None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:08:43.931804 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 442/875] 10.222.0.18 () {60 vars in 1141 bytes} [Tue Sep 13 03:08:43 2022] POST /identity/v3/services => generated 269 bytes in 186 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 03:08:44.021531 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed0d2759-7cf1-4313-82a9-440b49ace844 None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:08:44.147864 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed0d2759-7cf1-4313-82a9-440b49ace844 None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:08:44.177844 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a2cef0-9aba-4925-95e6-5467ad6aa0cf None None] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:08:44.178159 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a2cef0-9aba-4925-95e6-5467ad6aa0cf None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:08:44.178428 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a2cef0-9aba-4925-95e6-5467ad6aa0cf None None] PATH_INFO: `/` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:08:44.180299 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 443/876] 10.222.0.18 () {54 vars in 927 bytes} [Tue Sep 13 03:08:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 03:08:44.185425 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab6ece29-e586-4ad2-8eb5-bb73f7700f03 None None] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:08:44.185747 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab6ece29-e586-4ad2-8eb5-bb73f7700f03 None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:08:44.186022 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab6ece29-e586-4ad2-8eb5-bb73f7700f03 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:08:44.192140 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e54c3d44-88b8-4ceb-bb4b-cf11254523f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:08:44.192781 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e54c3d44-88b8-4ceb-bb4b-cf11254523f8 None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:08:44.193059 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e54c3d44-88b8-4ceb-bb4b-cf11254523f8 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:08:44.193328 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e54c3d44-88b8-4ceb-bb4b-cf11254523f8 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:08:44.193911 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e54c3d44-88b8-4ceb-bb4b-cf11254523f8 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:08:44.195296 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e54c3d44-88b8-4ceb-bb4b-cf11254523f8 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:08:44.204927 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application [None req-e54c3d44-88b8-4ceb-bb4b-cf11254523f8 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 13 03:08:44.204927 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 03:08:44.204927 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 13 03:08:44.204927 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 13 03:08:44.204927 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 13 03:08:44.204927 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 13 03:08:44.204927 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 13 03:08:44.204927 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 13 03:08:44.204927 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 13 03:08:44.204927 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Sep 13 03:08:44.204927 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 13 03:08:44.204927 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Sep 13 03:08:44.204927 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 03:08:44.204927 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 03:08:44.204927 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 03:08:44.204927 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 03:08:44.204927 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 03:08:44.205938 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 03:08:44.205938 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 03:08:44.205938 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 03:08:44.205938 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 03:08:44.205938 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 03:08:44.205938 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 03:08:44.205938 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 13 03:08:44.205938 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 13 03:08:44.205938 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 13 03:08:44.205938 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 13 03:08:44.205938 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 03:08:44.205938 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 03:08:44.205938 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 03:08:44.205938 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 03:08:44.205938 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 03:08:44.205938 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 03:08:44.206822 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 03:08:44.206822 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application with Lock( Sep 13 03:08:44.206822 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 03:08:44.206822 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._enter() Sep 13 03:08:44.206822 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 03:08:44.206822 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 03:08:44.206822 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 03:08:44.206822 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.creator() Sep 13 03:08:44.206822 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 03:08:44.206822 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application created_value = creator( Sep 13 03:08:44.206822 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 13 03:08:44.206822 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 13 03:08:44.206822 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 13 03:08:44.206822 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Sep 13 03:08:44.207674 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.auth.core [None req-ab6ece29-e586-4ad2-8eb5-bb73f7700f03 None None] MFA Rules not processed for user `49f280ab086b4610855c4ffa378517d9`. Rule list: `[]` (Enabled: `True`). {{(pid=81400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 03:08:44.207760 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 434/877] 10.222.0.18 () {56 vars in 1126 bytes} [Tue Sep 13 03:08:44 2022] GET /identity/v3/services/object-store => generated 93 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 03:08:44.207831 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-ab6ece29-e586-4ad2-8eb5-bb73f7700f03 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:08:44.214143 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e54c3d44-88b8-4ceb-bb4b-cf11254523f8 None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:08:44.241104 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:08:44.283987 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 444/878] 10.222.0.18 () {58 vars in 1024 bytes} [Tue Sep 13 03:08:44 2022] POST /identity/v3/auth/tokens => generated 1271 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 13 03:08:44.322238 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e54c3d44-88b8-4ceb-bb4b-cf11254523f8 None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:08:44.358400 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6e016ad-e9bd-451d-8370-cf918be3c6b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:08:44.359454 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e016ad-e9bd-451d-8370-cf918be3c6b3 None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:08:44.360039 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e016ad-e9bd-451d-8370-cf918be3c6b3 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:08:44.360508 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e016ad-e9bd-451d-8370-cf918be3c6b3 None admin] PATH_INFO: `/v3/services` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:08:44.361343 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6e016ad-e9bd-451d-8370-cf918be3c6b3 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 03:08:44.361858 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6e016ad-e9bd-451d-8370-cf918be3c6b3 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:08:44.363617 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6e016ad-e9bd-451d-8370-cf918be3c6b3 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:08:44.368957 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 435/879] 10.222.0.18 () {56 vars in 1122 bytes} [Tue Sep 13 03:08:44 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 03:08:44.374809 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab6ece29-e586-4ad2-8eb5-bb73f7700f03 None None] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:08:44.472682 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:08:44.482842 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab6ece29-e586-4ad2-8eb5-bb73f7700f03 None None] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:08:44.519099 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eb24e2a-5ff5-4615-8e81-d5a067c35b9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:08:44.519709 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb24e2a-5ff5-4615-8e81-d5a067c35b9c None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:08:44.519986 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb24e2a-5ff5-4615-8e81-d5a067c35b9c None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:08:44.520259 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb24e2a-5ff5-4615-8e81-d5a067c35b9c None admin] PATH_INFO: `/v3/services` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:08:44.520887 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eb24e2a-5ff5-4615-8e81-d5a067c35b9c None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 03:08:44.521188 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eb24e2a-5ff5-4615-8e81-d5a067c35b9c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:08:44.522555 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eb24e2a-5ff5-4615-8e81-d5a067c35b9c None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:08:44.527672 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 445/880] 10.222.0.18 () {56 vars in 1122 bytes} [Tue Sep 13 03:08:44 2022] GET /identity/v3/services?type=object-store => generated 129 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 03:08:44.555970 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6e016ad-e9bd-451d-8370-cf918be3c6b3 None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:08:44.649195 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:08:44.665218 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6e016ad-e9bd-451d-8370-cf918be3c6b3 None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:08:44.703013 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d52f8df-7fd7-4201-a15d-9ed7ecfd08f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:08:44.703650 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d52f8df-7fd7-4201-a15d-9ed7ecfd08f7 None admin] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:08:44.703930 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d52f8df-7fd7-4201-a15d-9ed7ecfd08f7 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:08:44.704207 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d52f8df-7fd7-4201-a15d-9ed7ecfd08f7 None admin] PATH_INFO: `/v3/services` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:08:44.704893 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d52f8df-7fd7-4201-a15d-9ed7ecfd08f7 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:08:44.706274 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d52f8df-7fd7-4201-a15d-9ed7ecfd08f7 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:08:44.726582 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 436/881] 10.222.0.18 () {60 vars in 1141 bytes} [Tue Sep 13 03:08:44 2022] POST /identity/v3/services => generated 251 bytes in 171 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 03:08:46.075256 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-124f6c38-08b2-4e03-870e-5f04007e7b98 None None] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:08:46.075714 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-124f6c38-08b2-4e03-870e-5f04007e7b98 None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:08:46.076022 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-124f6c38-08b2-4e03-870e-5f04007e7b98 None None] PATH_INFO: `/` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:08:46.078551 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 446/882] 10.222.0.18 () {54 vars in 927 bytes} [Tue Sep 13 03:08:46 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 03:08:46.083572 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65752c9e-5423-4a8a-9ed5-f435b8248a43 None None] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:08:46.084412 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65752c9e-5423-4a8a-9ed5-f435b8248a43 None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:08:46.084905 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65752c9e-5423-4a8a-9ed5-f435b8248a43 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:08:46.103241 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.auth.core [None req-65752c9e-5423-4a8a-9ed5-f435b8248a43 None None] MFA Rules not processed for user `49f280ab086b4610855c4ffa378517d9`. Rule list: `[]` (Enabled: `True`). {{(pid=81399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 03:08:46.104705 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-65752c9e-5423-4a8a-9ed5-f435b8248a43 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:08:46.183066 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 437/883] 10.222.0.18 () {58 vars in 1024 bytes} [Tue Sep 13 03:08:46 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 13 03:08:46.246990 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b704ec-202d-4c80-ae4c-72f6052f8860 None None] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:08:46.247638 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b704ec-202d-4c80-ae4c-72f6052f8860 None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:08:46.248633 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b704ec-202d-4c80-ae4c-72f6052f8860 None None] PATH_INFO: `/` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:08:46.250427 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 447/884] 10.222.0.18 () {54 vars in 927 bytes} [Tue Sep 13 03:08:46 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 03:08:46.255773 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-341f3fc7-19ec-4f77-b3c9-9e0fb28e5d85 None None] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:08:46.256067 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-341f3fc7-19ec-4f77-b3c9-9e0fb28e5d85 None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:08:46.256600 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-341f3fc7-19ec-4f77-b3c9-9e0fb28e5d85 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:08:46.275669 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.auth.core [None req-341f3fc7-19ec-4f77-b3c9-9e0fb28e5d85 None None] MFA Rules not processed for user `49f280ab086b4610855c4ffa378517d9`. Rule list: `[]` (Enabled: `True`). {{(pid=81399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 03:08:46.277150 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-341f3fc7-19ec-4f77-b3c9-9e0fb28e5d85 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:08:46.346659 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 438/885] 10.222.0.18 () {58 vars in 1024 bytes} [Tue Sep 13 03:08:46 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 13 03:08:46.456228 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1b704ec-202d-4c80-ae4c-72f6052f8860 None None] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:08:46.559298 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1b704ec-202d-4c80-ae4c-72f6052f8860 None None] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:08:46.594175 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bba1c9d-8870-42d8-91c3-7956007cdeb2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:08:46.595281 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bba1c9d-8870-42d8-91c3-7956007cdeb2 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:08:46.595772 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bba1c9d-8870-42d8-91c3-7956007cdeb2 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:08:46.596247 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bba1c9d-8870-42d8-91c3-7956007cdeb2 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:08:46.607025 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bba1c9d-8870-42d8-91c3-7956007cdeb2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 03:08:46.612758 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application [None req-2bba1c9d-8870-42d8-91c3-7956007cdeb2 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 13 03:08:46.612758 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 03:08:46.612758 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 03:08:46.612758 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 03:08:46.612758 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 03:08:46.612758 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 03:08:46.612758 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 03:08:46.612758 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 03:08:46.612758 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 03:08:46.612758 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 03:08:46.612758 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 03:08:46.612758 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 03:08:46.612758 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 03:08:46.612758 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 03:08:46.612758 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 03:08:46.612758 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 03:08:46.613768 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 03:08:46.613768 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 03:08:46.613768 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 03:08:46.613768 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 03:08:46.613768 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 03:08:46.613768 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 03:08:46.613768 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 03:08:46.613768 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 03:08:46.613768 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 03:08:46.613768 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 03:08:46.613768 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 03:08:46.613768 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application with Lock( Sep 13 03:08:46.613768 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 03:08:46.613768 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._enter() Sep 13 03:08:46.613768 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 03:08:46.613768 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 03:08:46.614714 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 03:08:46.614714 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.creator() Sep 13 03:08:46.614714 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 03:08:46.614714 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application created_value = creator( Sep 13 03:08:46.614714 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 03:08:46.614714 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 03:08:46.614714 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 03:08:46.614714 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 03:08:46.614714 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 03:08:46.614714 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 03:08:46.614714 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 13 03:08:46.614714 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Sep 13 03:08:46.617357 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 448/886] 10.222.0.18 () {56 vars in 1120 bytes} [Tue Sep 13 03:08:46 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 03:08:46.624107 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-341f3fc7-19ec-4f77-b3c9-9e0fb28e5d85 None None] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:08:46.626698 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bba1c9d-8870-42d8-91c3-7956007cdeb2 None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:08:46.627741 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:08:46.644681 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:08:46.755466 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bba1c9d-8870-42d8-91c3-7956007cdeb2 None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:08:46.766289 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-341f3fc7-19ec-4f77-b3c9-9e0fb28e5d85 None None] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:08:46.797234 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfb227f3-f465-43cb-aee7-942749ab738b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:08:46.798195 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfb227f3-f465-43cb-aee7-942749ab738b None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:08:46.798479 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfb227f3-f465-43cb-aee7-942749ab738b None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:08:46.799053 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfb227f3-f465-43cb-aee7-942749ab738b None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:08:46.799642 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfb227f3-f465-43cb-aee7-942749ab738b None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:08:46.801652 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfb227f3-f465-43cb-aee7-942749ab738b None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:08:46.809643 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9422aedf-d970-4c9a-9789-46735c3286ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:08:46.810315 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9422aedf-d970-4c9a-9789-46735c3286ff None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:08:46.810650 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9422aedf-d970-4c9a-9789-46735c3286ff None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:08:46.810937 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9422aedf-d970-4c9a-9789-46735c3286ff None admin] PATH_INFO: `/v3/roles` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:08:46.811604 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9422aedf-d970-4c9a-9789-46735c3286ff None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 03:08:46.811907 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9422aedf-d970-4c9a-9789-46735c3286ff None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:08:46.812081 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application [None req-cfb227f3-f465-43cb-aee7-942749ab738b None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 13 03:08:46.812081 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 03:08:46.812081 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 13 03:08:46.812081 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 13 03:08:46.812081 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 13 03:08:46.812081 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 13 03:08:46.812081 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 13 03:08:46.812081 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 13 03:08:46.812081 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 13 03:08:46.812081 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Sep 13 03:08:46.812081 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 13 03:08:46.812081 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Sep 13 03:08:46.812081 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 03:08:46.812081 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 03:08:46.812081 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 03:08:46.812081 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 03:08:46.813113 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 03:08:46.813113 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 03:08:46.813113 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 03:08:46.813113 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 03:08:46.813113 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 03:08:46.813113 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 03:08:46.813113 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 03:08:46.813113 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 13 03:08:46.813113 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 13 03:08:46.813113 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 13 03:08:46.813113 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 13 03:08:46.813113 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 03:08:46.813113 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 03:08:46.813113 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 03:08:46.813113 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 03:08:46.813113 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 03:08:46.813113 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 03:08:46.814021 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 03:08:46.814021 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application with Lock( Sep 13 03:08:46.814021 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 03:08:46.814021 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._enter() Sep 13 03:08:46.814021 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 03:08:46.814021 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 03:08:46.814021 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 03:08:46.814021 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.creator() Sep 13 03:08:46.814021 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 03:08:46.814021 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application created_value = creator( Sep 13 03:08:46.814021 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 13 03:08:46.814021 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 13 03:08:46.814021 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 13 03:08:46.814021 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Sep 13 03:08:46.814821 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9422aedf-d970-4c9a-9789-46735c3286ff None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:08:46.814903 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 449/887] 10.222.0.18 () {56 vars in 1132 bytes} [Tue Sep 13 03:08:46 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 03:08:46.819321 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfb227f3-f465-43cb-aee7-942749ab738b None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:08:46.822553 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 439/888] 10.222.0.18 () {56 vars in 1115 bytes} [Tue Sep 13 03:08:46 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 03:08:46.824693 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4999d052-5d13-4139-b1d4-192b3c34cda0 None None] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:08:46.824977 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4999d052-5d13-4139-b1d4-192b3c34cda0 None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:08:46.825254 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4999d052-5d13-4139-b1d4-192b3c34cda0 None None] PATH_INFO: `/` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:08:46.826865 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 440/889] 10.222.0.18 () {54 vars in 927 bytes} [Tue Sep 13 03:08:46 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 03:08:46.827580 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4999d052-5d13-4139-b1d4-192b3c34cda0 None None] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:08:46.901430 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:08:46.910426 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:08:46.933369 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfb227f3-f465-43cb-aee7-942749ab738b None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:08:46.946328 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4999d052-5d13-4139-b1d4-192b3c34cda0 None None] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:08:46.969597 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-107f5be5-b5e4-46e4-923a-a3d402182c18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:08:46.970208 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-107f5be5-b5e4-46e4-923a-a3d402182c18 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:08:46.970487 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-107f5be5-b5e4-46e4-923a-a3d402182c18 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:08:46.970788 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-107f5be5-b5e4-46e4-923a-a3d402182c18 None admin] PATH_INFO: `/v3/services` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:08:46.971424 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-107f5be5-b5e4-46e4-923a-a3d402182c18 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=81400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 03:08:46.971709 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-107f5be5-b5e4-46e4-923a-a3d402182c18 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:08:46.973164 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-107f5be5-b5e4-46e4-923a-a3d402182c18 None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:08:46.978148 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 450/890] 10.222.0.18 () {56 vars in 1126 bytes} [Tue Sep 13 03:08:46 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 03:08:46.979379 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe00139b-39fa-4783-98de-760e35158500 None None] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:08:46.979664 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe00139b-39fa-4783-98de-760e35158500 None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:08:46.979937 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe00139b-39fa-4783-98de-760e35158500 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:08:46.986258 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d103b36-ba7b-4204-8cf9-39fb6b2c0e1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:08:46.986932 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d103b36-ba7b-4204-8cf9-39fb6b2c0e1e None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:08:46.987213 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d103b36-ba7b-4204-8cf9-39fb6b2c0e1e None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:08:46.987485 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d103b36-ba7b-4204-8cf9-39fb6b2c0e1e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:08:46.997964 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.auth.core [None req-fe00139b-39fa-4783-98de-760e35158500 None None] MFA Rules not processed for user `49f280ab086b4610855c4ffa378517d9`. Rule list: `[]` (Enabled: `True`). {{(pid=81400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 03:08:46.998143 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d103b36-ba7b-4204-8cf9-39fb6b2c0e1e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:08:46.999247 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-fe00139b-39fa-4783-98de-760e35158500 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:08:46.999606 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d103b36-ba7b-4204-8cf9-39fb6b2c0e1e None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:08:47.013559 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application [None req-1d103b36-ba7b-4204-8cf9-39fb6b2c0e1e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 03:08:47.013559 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 03:08:47.013559 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 03:08:47.013559 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 03:08:47.013559 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 03:08:47.013559 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 03:08:47.013559 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 03:08:47.013559 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 03:08:47.013559 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 03:08:47.013559 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 03:08:47.013559 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 03:08:47.013559 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 03:08:47.013559 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 13 03:08:47.013559 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 13 03:08:47.013559 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 13 03:08:47.013559 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 13 03:08:47.015077 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 03:08:47.015077 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 03:08:47.015077 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 03:08:47.015077 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 03:08:47.015077 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 03:08:47.015077 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 03:08:47.015077 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 03:08:47.015077 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application with Lock( Sep 13 03:08:47.015077 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 03:08:47.015077 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._enter() Sep 13 03:08:47.015077 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 03:08:47.015077 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 03:08:47.015077 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 03:08:47.015077 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.creator() Sep 13 03:08:47.015077 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 03:08:47.015077 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application created_value = creator( Sep 13 03:08:47.015077 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 13 03:08:47.016030 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 13 03:08:47.016030 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 03:08:47.016030 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Sep 13 03:08:47.016030 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 441/891] 10.222.0.18 () {56 vars in 1108 bytes} [Tue Sep 13 03:08:46 2022] GET /identity/v3/domains/Default => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 03:08:47.016437 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73b66bb3-afd0-4107-82ae-a057f7db0fd0 None None] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:08:47.016730 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73b66bb3-afd0-4107-82ae-a057f7db0fd0 None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:08:47.016985 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73b66bb3-afd0-4107-82ae-a057f7db0fd0 None None] PATH_INFO: `/` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:08:47.018305 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 442/892] 10.222.0.18 () {54 vars in 927 bytes} [Tue Sep 13 03:08:47 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 03:08:47.018928 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73b66bb3-afd0-4107-82ae-a057f7db0fd0 None None] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:08:47.071876 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 451/893] 10.222.0.18 () {58 vars in 1024 bytes} [Tue Sep 13 03:08:46 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 13 03:08:47.072771 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe00139b-39fa-4783-98de-760e35158500 None None] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:08:47.123051 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73b66bb3-afd0-4107-82ae-a057f7db0fd0 None None] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:08:47.138369 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:08:47.148712 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:08:47.160692 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11fd617f-2a56-4d87-b6fa-1f95f094237e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:08:47.161227 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11fd617f-2a56-4d87-b6fa-1f95f094237e None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:08:47.161494 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11fd617f-2a56-4d87-b6fa-1f95f094237e None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:08:47.161760 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11fd617f-2a56-4d87-b6fa-1f95f094237e None admin] PATH_INFO: `/v3/services` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:08:47.162376 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11fd617f-2a56-4d87-b6fa-1f95f094237e None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=81399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 03:08:47.162697 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11fd617f-2a56-4d87-b6fa-1f95f094237e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:08:47.164052 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11fd617f-2a56-4d87-b6fa-1f95f094237e None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:08:47.168731 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 443/894] 10.222.0.18 () {56 vars in 1126 bytes} [Tue Sep 13 03:08:47 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 03:08:47.169835 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c1d2bbd-d6d4-4ffb-8caa-00076499a8b8 None None] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:08:47.170136 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c1d2bbd-d6d4-4ffb-8caa-00076499a8b8 None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:08:47.170403 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c1d2bbd-d6d4-4ffb-8caa-00076499a8b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:08:47.186892 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe00139b-39fa-4783-98de-760e35158500 None None] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:08:47.188508 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.auth.core [None req-9c1d2bbd-d6d4-4ffb-8caa-00076499a8b8 None None] MFA Rules not processed for user `49f280ab086b4610855c4ffa378517d9`. Rule list: `[]` (Enabled: `True`). {{(pid=81399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 03:08:47.190428 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-9c1d2bbd-d6d4-4ffb-8caa-00076499a8b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:08:47.221579 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96ec0444-5d77-4159-9bad-a5a899112764 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:08:47.222160 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ec0444-5d77-4159-9bad-a5a899112764 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:08:47.222430 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ec0444-5d77-4159-9bad-a5a899112764 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:08:47.222720 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ec0444-5d77-4159-9bad-a5a899112764 None admin] PATH_INFO: `/v3/domains` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:08:47.223342 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96ec0444-5d77-4159-9bad-a5a899112764 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 03:08:47.223632 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96ec0444-5d77-4159-9bad-a5a899112764 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:08:47.224979 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96ec0444-5d77-4159-9bad-a5a899112764 None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:08:47.240577 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 452/895] 10.222.0.18 () {56 vars in 1109 bytes} [Tue Sep 13 03:08:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 03:08:47.240577 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96ec0444-5d77-4159-9bad-a5a899112764 None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:08:47.269450 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 444/896] 10.222.0.18 () {58 vars in 1024 bytes} [Tue Sep 13 03:08:47 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 13 03:08:47.270483 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c1d2bbd-d6d4-4ffb-8caa-00076499a8b8 None None] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:08:47.366306 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96ec0444-5d77-4159-9bad-a5a899112764 None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:08:47.387062 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c1d2bbd-d6d4-4ffb-8caa-00076499a8b8 None None] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:08:47.399616 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:08:47.401286 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:08:47.406365 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5217e985-b2c7-4b5c-9b15-fd6bb9106f26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:08:47.407576 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5217e985-b2c7-4b5c-9b15-fd6bb9106f26 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:08:47.408506 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5217e985-b2c7-4b5c-9b15-fd6bb9106f26 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:08:47.408787 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5217e985-b2c7-4b5c-9b15-fd6bb9106f26 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:08:47.409485 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5217e985-b2c7-4b5c-9b15-fd6bb9106f26 None admin] RBAC: Adding query filter params (interface=public, service_id=10525112140f40a3939ac1766da8f1b3, region_id=RegionOne) {{(pid=81400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 03:08:47.409801 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5217e985-b2c7-4b5c-9b15-fd6bb9106f26 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:08:47.411720 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5217e985-b2c7-4b5c-9b15-fd6bb9106f26 None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:08:47.422226 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 453/897] 10.222.0.18 () {56 vars in 1251 bytes} [Tue Sep 13 03:08:47 2022] GET /identity/v3/endpoints?service_id=10525112140f40a3939ac1766da8f1b3&interface=public®ion_id=RegionOne => generated 194 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 03:08:47.422226 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5217e985-b2c7-4b5c-9b15-fd6bb9106f26 None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:08:47.425488 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3a21743-7aa0-4550-9a66-5e5c6b5ee3a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:08:47.426121 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3a21743-7aa0-4550-9a66-5e5c6b5ee3a9 None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:08:47.426399 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3a21743-7aa0-4550-9a66-5e5c6b5ee3a9 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:08:47.426698 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3a21743-7aa0-4550-9a66-5e5c6b5ee3a9 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:08:47.433957 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3a21743-7aa0-4550-9a66-5e5c6b5ee3a9 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:08:47.435412 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3a21743-7aa0-4550-9a66-5e5c6b5ee3a9 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:08:47.447601 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application [None req-a3a21743-7aa0-4550-9a66-5e5c6b5ee3a9 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 13 03:08:47.447601 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 03:08:47.447601 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 03:08:47.447601 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 03:08:47.447601 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 03:08:47.447601 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 03:08:47.447601 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 03:08:47.447601 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 03:08:47.447601 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 03:08:47.447601 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 03:08:47.447601 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 03:08:47.447601 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 03:08:47.447601 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 13 03:08:47.447601 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 13 03:08:47.447601 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 13 03:08:47.447601 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 13 03:08:47.448575 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 03:08:47.448575 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 03:08:47.448575 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 13 03:08:47.448575 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 13 03:08:47.448575 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 13 03:08:47.448575 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 13 03:08:47.448575 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 03:08:47.448575 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 03:08:47.448575 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 03:08:47.448575 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 03:08:47.448575 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 03:08:47.448575 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application with Lock( Sep 13 03:08:47.448575 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 03:08:47.448575 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._enter() Sep 13 03:08:47.448575 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 03:08:47.448575 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 03:08:47.448575 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 03:08:47.448575 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.creator() Sep 13 03:08:47.449567 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 03:08:47.449567 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application created_value = creator( Sep 13 03:08:47.449567 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 13 03:08:47.449567 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 13 03:08:47.449567 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 13 03:08:47.449567 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 13 03:08:47.449567 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 13 03:08:47.449567 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 13 03:08:47.449567 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 13 03:08:47.449567 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Sep 13 03:08:47.449567 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 445/898] 10.222.0.18 () {56 vars in 1117 bytes} [Tue Sep 13 03:08:47 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 03:08:47.449567 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3a21743-7aa0-4550-9a66-5e5c6b5ee3a9 None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:08:47.532787 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5217e985-b2c7-4b5c-9b15-fd6bb9106f26 None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:08:47.559096 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3a21743-7aa0-4550-9a66-5e5c6b5ee3a9 None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:08:47.568142 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-155f9523-0209-49de-b15a-151387fc5624 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:08:47.568820 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-155f9523-0209-49de-b15a-151387fc5624 None admin] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:08:47.569103 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-155f9523-0209-49de-b15a-151387fc5624 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:08:47.569375 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-155f9523-0209-49de-b15a-151387fc5624 None admin] PATH_INFO: `/v3/services` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:08:47.570019 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-155f9523-0209-49de-b15a-151387fc5624 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:08:47.571419 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-155f9523-0209-49de-b15a-151387fc5624 None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:08:47.585725 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 454/899] 10.222.0.18 () {60 vars in 1141 bytes} [Tue Sep 13 03:08:47 2022] POST /identity/v3/services => generated 241 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 03:08:47.586518 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-155f9523-0209-49de-b15a-151387fc5624 None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:08:47.602451 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84c07fe0-2b2f-4480-b110-6fe4af384adc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:08:47.603545 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c07fe0-2b2f-4480-b110-6fe4af384adc None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:08:47.604004 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c07fe0-2b2f-4480-b110-6fe4af384adc None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:08:47.604478 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c07fe0-2b2f-4480-b110-6fe4af384adc None admin] PATH_INFO: `/v3/services` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:08:47.605280 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84c07fe0-2b2f-4480-b110-6fe4af384adc None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:08:47.607421 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84c07fe0-2b2f-4480-b110-6fe4af384adc None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:08:47.626652 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 446/900] 10.222.0.18 () {56 vars in 1087 bytes} [Tue Sep 13 03:08:47 2022] GET /identity/v3/services => generated 1497 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 03:08:47.626652 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84c07fe0-2b2f-4480-b110-6fe4af384adc None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:08:47.666228 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:08:47.675519 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:08:47.738863 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-155f9523-0209-49de-b15a-151387fc5624 None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:08:47.768268 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84c07fe0-2b2f-4480-b110-6fe4af384adc None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:08:47.789552 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaeaad2a-93f4-474f-9090-b698b3db6076 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:08:47.790207 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaeaad2a-93f4-474f-9090-b698b3db6076 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:08:47.790485 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaeaad2a-93f4-474f-9090-b698b3db6076 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:08:47.790789 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaeaad2a-93f4-474f-9090-b698b3db6076 None admin] PATH_INFO: `/v3/users` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:08:47.791531 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaeaad2a-93f4-474f-9090-b698b3db6076 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 03:08:47.791837 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaeaad2a-93f4-474f-9090-b698b3db6076 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:08:47.793293 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaeaad2a-93f4-474f-9090-b698b3db6076 None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:08:47.814574 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87a676ad-23ad-44e2-8b91-ead20cf1841c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:08:47.816258 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 455/901] 10.222.0.18 () {56 vars in 1149 bytes} [Tue Sep 13 03:08:47 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 03:08:47.816394 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87a676ad-23ad-44e2-8b91-ead20cf1841c None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:08:47.816394 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87a676ad-23ad-44e2-8b91-ead20cf1841c None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:08:47.816394 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87a676ad-23ad-44e2-8b91-ead20cf1841c None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:08:47.816607 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87a676ad-23ad-44e2-8b91-ead20cf1841c None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:08:47.817938 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87a676ad-23ad-44e2-8b91-ead20cf1841c None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:08:47.821301 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaeaad2a-93f4-474f-9090-b698b3db6076 None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:08:47.831219 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application [None req-87a676ad-23ad-44e2-8b91-ead20cf1841c None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 13 03:08:47.831219 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 03:08:47.831219 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 13 03:08:47.831219 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 13 03:08:47.831219 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 13 03:08:47.831219 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 13 03:08:47.831219 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 13 03:08:47.831219 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 13 03:08:47.831219 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 13 03:08:47.831219 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Sep 13 03:08:47.831219 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 13 03:08:47.831219 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Sep 13 03:08:47.831219 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 03:08:47.831219 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 03:08:47.831219 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 03:08:47.831219 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 03:08:47.831219 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 03:08:47.832263 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 03:08:47.832263 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 03:08:47.832263 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 03:08:47.832263 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 03:08:47.832263 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 03:08:47.832263 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 03:08:47.832263 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 13 03:08:47.832263 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 13 03:08:47.832263 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 13 03:08:47.832263 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 13 03:08:47.832263 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 03:08:47.832263 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 03:08:47.832263 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 03:08:47.832263 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 03:08:47.832263 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 03:08:47.832263 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 03:08:47.833126 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 03:08:47.833126 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application with Lock( Sep 13 03:08:47.833126 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 03:08:47.833126 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._enter() Sep 13 03:08:47.833126 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 03:08:47.833126 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 03:08:47.833126 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 03:08:47.833126 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.creator() Sep 13 03:08:47.833126 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 03:08:47.833126 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application created_value = creator( Sep 13 03:08:47.833126 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 13 03:08:47.833126 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 13 03:08:47.833126 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 13 03:08:47.833126 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Sep 13 03:08:47.833886 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 447/902] 10.222.0.18 () {56 vars in 1114 bytes} [Tue Sep 13 03:08:47 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 03:08:47.839449 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87a676ad-23ad-44e2-8b91-ead20cf1841c None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:08:47.940759 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaeaad2a-93f4-474f-9090-b698b3db6076 None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:08:47.945248 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:08:47.953738 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87a676ad-23ad-44e2-8b91-ead20cf1841c None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:08:47.961342 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:08:47.987924 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f104133-5a49-426a-a6fc-4794bc9d1083 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:08:47.988590 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f104133-5a49-426a-a6fc-4794bc9d1083 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:08:47.988870 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f104133-5a49-426a-a6fc-4794bc9d1083 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:08:47.989147 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f104133-5a49-426a-a6fc-4794bc9d1083 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:08:47.998292 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b4def9b-93fe-4bf3-8888-da6d59e7d596 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:08:47.999234 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b4def9b-93fe-4bf3-8888-da6d59e7d596 None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:08:47.999652 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b4def9b-93fe-4bf3-8888-da6d59e7d596 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:08:48.000060 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b4def9b-93fe-4bf3-8888-da6d59e7d596 None admin] PATH_INFO: `/v3/services` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:08:48.000849 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b4def9b-93fe-4bf3-8888-da6d59e7d596 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=81399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 03:08:48.001377 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b4def9b-93fe-4bf3-8888-da6d59e7d596 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:08:48.003165 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b4def9b-93fe-4bf3-8888-da6d59e7d596 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:08:48.008484 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 448/903] 10.222.0.18 () {56 vars in 1114 bytes} [Tue Sep 13 03:08:47 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 03:08:48.014661 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b4def9b-93fe-4bf3-8888-da6d59e7d596 None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:08:48.090914 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f104133-5a49-426a-a6fc-4794bc9d1083 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:08:48.093555 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f104133-5a49-426a-a6fc-4794bc9d1083 None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:08:48.142285 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b4def9b-93fe-4bf3-8888-da6d59e7d596 None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:08:48.182807 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26d21a77-90cc-4a70-83d2-c286a56b1916 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:08:48.183429 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d21a77-90cc-4a70-83d2-c286a56b1916 None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:08:48.184029 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d21a77-90cc-4a70-83d2-c286a56b1916 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:08:48.184309 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d21a77-90cc-4a70-83d2-c286a56b1916 None admin] PATH_INFO: `/v3/services` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:08:48.185223 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26d21a77-90cc-4a70-83d2-c286a56b1916 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=81399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 03:08:48.186071 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26d21a77-90cc-4a70-83d2-c286a56b1916 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:08:48.187470 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26d21a77-90cc-4a70-83d2-c286a56b1916 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:08:48.194739 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 449/904] 10.222.0.18 () {56 vars in 1114 bytes} [Tue Sep 13 03:08:48 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 03:08:48.200531 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26d21a77-90cc-4a70-83d2-c286a56b1916 None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:08:48.225287 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 456/905] 10.222.0.18 () {58 vars in 1282 bytes} [Tue Sep 13 03:08:47 2022] GET /identity/v3/auth/tokens => generated 1474 bytes in 400 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Sep 13 03:08:48.227452 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f104133-5a49-426a-a6fc-4794bc9d1083 None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:08:48.235706 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:08:48.247686 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:08:48.327665 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26d21a77-90cc-4a70-83d2-c286a56b1916 None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:08:48.367319 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f104133-5a49-426a-a6fc-4794bc9d1083 None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:08:48.369071 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb09cde5-1e16-4968-a7a9-b1f3b80107f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:08:48.370818 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb09cde5-1e16-4968-a7a9-b1f3b80107f3 None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:08:48.371330 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb09cde5-1e16-4968-a7a9-b1f3b80107f3 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:08:48.371877 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb09cde5-1e16-4968-a7a9-b1f3b80107f3 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:08:48.372788 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb09cde5-1e16-4968-a7a9-b1f3b80107f3 None admin] RBAC: Adding query filter params (interface=public, service_id=fa23abb7aee24b6ca4374c127456ce77, region_id=RegionOne) {{(pid=81399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 03:08:48.373834 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb09cde5-1e16-4968-a7a9-b1f3b80107f3 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:08:48.375983 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb09cde5-1e16-4968-a7a9-b1f3b80107f3 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:08:48.383074 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 450/906] 10.222.0.18 () {56 vars in 1251 bytes} [Tue Sep 13 03:08:48 2022] GET /identity/v3/endpoints?service_id=fa23abb7aee24b6ca4374c127456ce77&interface=public®ion_id=RegionOne => generated 194 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 03:08:48.388375 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb09cde5-1e16-4968-a7a9-b1f3b80107f3 None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:08:48.410564 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c643d8d0-8475-4582-a8ad-3b1dd7465bd7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:08:48.411276 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c643d8d0-8475-4582-a8ad-3b1dd7465bd7 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:08:48.411559 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c643d8d0-8475-4582-a8ad-3b1dd7465bd7 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:08:48.411875 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c643d8d0-8475-4582-a8ad-3b1dd7465bd7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:08:48.422475 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c643d8d0-8475-4582-a8ad-3b1dd7465bd7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:08:48.424003 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c643d8d0-8475-4582-a8ad-3b1dd7465bd7 None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:08:48.440451 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application [None req-c643d8d0-8475-4582-a8ad-3b1dd7465bd7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 03:08:48.440451 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 03:08:48.440451 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 03:08:48.440451 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 03:08:48.440451 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 03:08:48.440451 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 03:08:48.440451 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 03:08:48.440451 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 03:08:48.440451 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 03:08:48.440451 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 03:08:48.440451 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 03:08:48.440451 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 03:08:48.440451 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 13 03:08:48.440451 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 13 03:08:48.440451 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 13 03:08:48.440451 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 13 03:08:48.441476 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 03:08:48.441476 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 03:08:48.441476 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 03:08:48.441476 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 03:08:48.441476 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 03:08:48.441476 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 03:08:48.441476 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 03:08:48.441476 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application with Lock( Sep 13 03:08:48.441476 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 03:08:48.441476 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._enter() Sep 13 03:08:48.441476 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 03:08:48.441476 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 03:08:48.441476 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 03:08:48.441476 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.creator() Sep 13 03:08:48.441476 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 03:08:48.441476 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application created_value = creator( Sep 13 03:08:48.441476 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 13 03:08:48.442388 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 13 03:08:48.442388 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 03:08:48.442388 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Sep 13 03:08:48.446372 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 457/907] 10.222.0.18 () {56 vars in 1108 bytes} [Tue Sep 13 03:08:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 13 03:08:48.447698 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c643d8d0-8475-4582-a8ad-3b1dd7465bd7 None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:08:48.493807 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:08:48.513479 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb09cde5-1e16-4968-a7a9-b1f3b80107f3 None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:08:48.519748 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:08:48.559756 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c643d8d0-8475-4582-a8ad-3b1dd7465bd7 None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:08:48.565448 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7758cd52-d8d3-4afc-a8c5-281601d49a8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:08:48.566308 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7758cd52-d8d3-4afc-a8c5-281601d49a8f None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:08:48.566844 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7758cd52-d8d3-4afc-a8c5-281601d49a8f None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:08:48.567854 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7758cd52-d8d3-4afc-a8c5-281601d49a8f None admin] PATH_INFO: `/v3/services` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:08:48.568676 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7758cd52-d8d3-4afc-a8c5-281601d49a8f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:08:48.570719 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7758cd52-d8d3-4afc-a8c5-281601d49a8f None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:08:48.587294 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 451/908] 10.222.0.18 () {56 vars in 1087 bytes} [Tue Sep 13 03:08:48 2022] GET /identity/v3/services => generated 1497 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 03:08:48.603890 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be99d213-3b47-4303-ae67-0c03a9c9ba0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:08:48.604518 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be99d213-3b47-4303-ae67-0c03a9c9ba0c None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:08:48.604796 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be99d213-3b47-4303-ae67-0c03a9c9ba0c None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:08:48.605068 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be99d213-3b47-4303-ae67-0c03a9c9ba0c None admin] PATH_INFO: `/v3/domains` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:08:48.605787 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be99d213-3b47-4303-ae67-0c03a9c9ba0c None admin] RBAC: Adding query filter params (name=Default) {{(pid=81400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 03:08:48.606094 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be99d213-3b47-4303-ae67-0c03a9c9ba0c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:08:48.608088 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be99d213-3b47-4303-ae67-0c03a9c9ba0c None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:08:48.628552 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 458/909] 10.222.0.18 () {56 vars in 1109 bytes} [Tue Sep 13 03:08:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 03:08:48.630139 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7758cd52-d8d3-4afc-a8c5-281601d49a8f None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:08:48.747393 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7758cd52-d8d3-4afc-a8c5-281601d49a8f None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:08:48.786415 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e89487bd-5a85-4d97-bf7e-dd640e2b128d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:08:48.787093 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e89487bd-5a85-4d97-bf7e-dd640e2b128d None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:08:48.787372 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e89487bd-5a85-4d97-bf7e-dd640e2b128d None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:08:48.787647 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e89487bd-5a85-4d97-bf7e-dd640e2b128d None admin] PATH_INFO: `/v3/projects/service` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:08:48.794516 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e89487bd-5a85-4d97-bf7e-dd640e2b128d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:08:48.796510 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e89487bd-5a85-4d97-bf7e-dd640e2b128d None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:08:48.808901 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application [None req-e89487bd-5a85-4d97-bf7e-dd640e2b128d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 13 03:08:48.808901 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 03:08:48.808901 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 03:08:48.808901 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 03:08:48.808901 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 03:08:48.808901 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 03:08:48.808901 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 03:08:48.808901 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 03:08:48.808901 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 03:08:48.808901 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 03:08:48.808901 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 03:08:48.808901 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 03:08:48.808901 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 13 03:08:48.808901 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 13 03:08:48.808901 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 13 03:08:48.809842 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 13 03:08:48.809842 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 03:08:48.809842 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 03:08:48.809842 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 03:08:48.809842 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 03:08:48.809842 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 03:08:48.809842 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 03:08:48.809842 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 03:08:48.809842 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application with Lock( Sep 13 03:08:48.809842 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 03:08:48.809842 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._enter() Sep 13 03:08:48.809842 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 03:08:48.809842 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 03:08:48.809842 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 03:08:48.809842 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.creator() Sep 13 03:08:48.809842 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 03:08:48.809842 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application created_value = creator( Sep 13 03:08:48.811962 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 13 03:08:48.811962 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 13 03:08:48.811962 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 13 03:08:48.811962 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 13 03:08:48.811962 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 13 03:08:48.811962 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 13 03:08:48.811962 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 13 03:08:48.811962 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Sep 13 03:08:48.811962 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 452/910] 10.222.0.18 () {56 vars in 1111 bytes} [Tue Sep 13 03:08:48 2022] GET /identity/v3/projects/service => generated 88 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 13 03:08:48.819280 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be99d213-3b47-4303-ae67-0c03a9c9ba0c None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:08:48.932623 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be99d213-3b47-4303-ae67-0c03a9c9ba0c None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:08:48.959721 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:08:48.970040 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bf72faf-a0f3-42e0-b3bd-900975d041c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:08:48.970706 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bf72faf-a0f3-42e0-b3bd-900975d041c2 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:08:48.970989 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bf72faf-a0f3-42e0-b3bd-900975d041c2 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:08:48.971259 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bf72faf-a0f3-42e0-b3bd-900975d041c2 None admin] PATH_INFO: `/v3/projects` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:08:48.971949 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bf72faf-a0f3-42e0-b3bd-900975d041c2 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 03:08:48.972252 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bf72faf-a0f3-42e0-b3bd-900975d041c2 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:08:48.973735 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bf72faf-a0f3-42e0-b3bd-900975d041c2 None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:08:48.990994 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 459/911] 10.222.0.18 () {56 vars in 1148 bytes} [Tue Sep 13 03:08:48 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 03:08:48.992979 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e89487bd-5a85-4d97-bf7e-dd640e2b128d None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:08:49.006454 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:08:49.109308 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e89487bd-5a85-4d97-bf7e-dd640e2b128d None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:08:49.143493 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-983ea28f-3146-44e5-9dbe-838c650ff301 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:08:49.144135 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-983ea28f-3146-44e5-9dbe-838c650ff301 None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:08:49.144424 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-983ea28f-3146-44e5-9dbe-838c650ff301 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:08:49.144705 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-983ea28f-3146-44e5-9dbe-838c650ff301 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:08:49.145500 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-983ea28f-3146-44e5-9dbe-838c650ff301 None admin] RBAC: Adding query filter params (role.id=57ec3ac460b749a5910aea16ad1d643e, scope.project.id=b9d94f6a68f7470d9494f112175620dc, user.id=4c2d20fb54a54ecc99b47f237be4917c) {{(pid=81399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 03:08:49.145809 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-983ea28f-3146-44e5-9dbe-838c650ff301 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:08:49.147266 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-983ea28f-3146-44e5-9dbe-838c650ff301 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:08:49.153166 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 453/912] 10.222.0.18 () {56 vars in 1374 bytes} [Tue Sep 13 03:08:48 2022] GET /identity/v3/role_assignments?user.id=4c2d20fb54a54ecc99b47f237be4917c&scope.project.id=b9d94f6a68f7470d9494f112175620dc&role.id=57ec3ac460b749a5910aea16ad1d643e => generated 259 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 03:08:49.798503 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74877900-8162-4e30-8b9f-945d9ca8fec6 None None] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:08:49.799062 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74877900-8162-4e30-8b9f-945d9ca8fec6 None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:08:49.799062 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74877900-8162-4e30-8b9f-945d9ca8fec6 None None] PATH_INFO: `/` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:08:49.800752 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 460/913] 10.222.0.18 () {54 vars in 927 bytes} [Tue Sep 13 03:08:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 03:08:49.807082 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1fd6f04-88d0-446d-b6ce-ab9397aacc32 None None] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:08:49.807409 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1fd6f04-88d0-446d-b6ce-ab9397aacc32 None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:08:49.807632 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1fd6f04-88d0-446d-b6ce-ab9397aacc32 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:08:49.831675 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.auth.core [None req-a1fd6f04-88d0-446d-b6ce-ab9397aacc32 None None] MFA Rules not processed for user `49f280ab086b4610855c4ffa378517d9`. Rule list: `[]` (Enabled: `True`). {{(pid=81399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 03:08:49.832861 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-a1fd6f04-88d0-446d-b6ce-ab9397aacc32 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:08:49.902088 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 454/914] 10.222.0.18 () {58 vars in 1024 bytes} [Tue Sep 13 03:08:49 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 13 03:08:49.907284 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e894d5-03a8-4b67-b68e-208c1f6a7824 None None] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:08:49.907562 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e894d5-03a8-4b67-b68e-208c1f6a7824 None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:08:49.907832 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e894d5-03a8-4b67-b68e-208c1f6a7824 None None] PATH_INFO: `/` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:08:49.909385 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 461/915] 10.222.0.18 () {54 vars in 927 bytes} [Tue Sep 13 03:08:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 03:08:49.914408 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-584a8718-e88f-436c-a146-4ca252777c3a None None] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:08:49.914728 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-584a8718-e88f-436c-a146-4ca252777c3a None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:08:49.915002 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-584a8718-e88f-436c-a146-4ca252777c3a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:08:49.922262 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:08:49.933686 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.auth.core [None req-584a8718-e88f-436c-a146-4ca252777c3a None None] MFA Rules not processed for user `49f280ab086b4610855c4ffa378517d9`. Rule list: `[]` (Enabled: `True`). {{(pid=81399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 03:08:49.934980 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-584a8718-e88f-436c-a146-4ca252777c3a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:08:50.008005 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 455/916] 10.222.0.18 () {58 vars in 1024 bytes} [Tue Sep 13 03:08:49 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 13 03:08:50.170823 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89e894d5-03a8-4b67-b68e-208c1f6a7824 None None] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:08:50.276959 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89e894d5-03a8-4b67-b68e-208c1f6a7824 None None] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:08:50.284262 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-584a8718-e88f-436c-a146-4ca252777c3a None None] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:08:50.318067 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-653fa271-668a-4a73-982a-dd7f890bb6bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:08:50.318734 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-653fa271-668a-4a73-982a-dd7f890bb6bd None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:08:50.319007 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-653fa271-668a-4a73-982a-dd7f890bb6bd None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:08:50.319272 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-653fa271-668a-4a73-982a-dd7f890bb6bd None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:08:50.320055 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-653fa271-668a-4a73-982a-dd7f890bb6bd None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:08:50.321478 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-653fa271-668a-4a73-982a-dd7f890bb6bd None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:08:50.333947 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application [None req-653fa271-668a-4a73-982a-dd7f890bb6bd None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 13 03:08:50.333947 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 03:08:50.333947 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 13 03:08:50.333947 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 13 03:08:50.333947 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 13 03:08:50.333947 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 13 03:08:50.333947 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 13 03:08:50.333947 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 13 03:08:50.333947 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 13 03:08:50.333947 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Sep 13 03:08:50.333947 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 13 03:08:50.333947 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Sep 13 03:08:50.333947 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 03:08:50.333947 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 03:08:50.333947 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 03:08:50.333947 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 03:08:50.333947 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 03:08:50.335031 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 03:08:50.335031 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 03:08:50.335031 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 03:08:50.335031 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 03:08:50.335031 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 03:08:50.335031 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 03:08:50.335031 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 13 03:08:50.335031 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 13 03:08:50.335031 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 13 03:08:50.335031 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 13 03:08:50.335031 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 03:08:50.335031 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 03:08:50.335031 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 03:08:50.335031 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 03:08:50.335031 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 03:08:50.335031 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 03:08:50.335882 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 03:08:50.335882 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application with Lock( Sep 13 03:08:50.335882 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 03:08:50.335882 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._enter() Sep 13 03:08:50.335882 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 03:08:50.335882 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 03:08:50.335882 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 03:08:50.335882 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.creator() Sep 13 03:08:50.335882 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 03:08:50.335882 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application created_value = creator( Sep 13 03:08:50.335882 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 13 03:08:50.335882 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 13 03:08:50.335882 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 13 03:08:50.335882 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Sep 13 03:08:50.337606 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 462/917] 10.222.0.18 () {56 vars in 1126 bytes} [Tue Sep 13 03:08:50 2022] GET /identity/v3/services/object-store => generated 93 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 03:08:50.342730 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-653fa271-668a-4a73-982a-dd7f890bb6bd None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:08:50.404974 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:08:50.415159 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-584a8718-e88f-436c-a146-4ca252777c3a None None] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:08:50.455512 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10c9ba54-63fb-4779-a1c3-73ff990b4cd7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:08:50.456439 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10c9ba54-63fb-4779-a1c3-73ff990b4cd7 None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:08:50.456724 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10c9ba54-63fb-4779-a1c3-73ff990b4cd7 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:08:50.457268 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10c9ba54-63fb-4779-a1c3-73ff990b4cd7 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:08:50.458280 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10c9ba54-63fb-4779-a1c3-73ff990b4cd7 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:08:50.460222 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10c9ba54-63fb-4779-a1c3-73ff990b4cd7 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:08:50.462358 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:08:50.472885 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-653fa271-668a-4a73-982a-dd7f890bb6bd None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:08:50.473216 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application [None req-10c9ba54-63fb-4779-a1c3-73ff990b4cd7 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 13 03:08:50.473216 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 03:08:50.473216 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 13 03:08:50.473216 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 13 03:08:50.473216 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 13 03:08:50.473216 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 13 03:08:50.473216 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 13 03:08:50.473216 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 13 03:08:50.473216 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 13 03:08:50.473216 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Sep 13 03:08:50.473216 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 13 03:08:50.473216 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Sep 13 03:08:50.473216 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 03:08:50.473216 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 03:08:50.473216 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 03:08:50.473216 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 03:08:50.474148 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 03:08:50.474148 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 03:08:50.474148 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 03:08:50.474148 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 03:08:50.474148 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 03:08:50.474148 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 03:08:50.474148 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 03:08:50.474148 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 13 03:08:50.474148 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 13 03:08:50.474148 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 13 03:08:50.474148 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 13 03:08:50.474148 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 03:08:50.474148 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 03:08:50.474148 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 03:08:50.474148 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 03:08:50.474148 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 03:08:50.474148 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 03:08:50.475084 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 03:08:50.475084 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application with Lock( Sep 13 03:08:50.475084 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 03:08:50.475084 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._enter() Sep 13 03:08:50.475084 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 03:08:50.475084 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 03:08:50.475084 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 03:08:50.475084 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.creator() Sep 13 03:08:50.475084 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 03:08:50.475084 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application created_value = creator( Sep 13 03:08:50.475084 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 13 03:08:50.475084 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 13 03:08:50.475084 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 13 03:08:50.475084 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Sep 13 03:08:50.482955 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 456/918] 10.222.0.18 () {56 vars in 1132 bytes} [Tue Sep 13 03:08:50 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 03:08:50.484072 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10c9ba54-63fb-4779-a1c3-73ff990b4cd7 None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:08:50.510076 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93e72df0-1e09-4399-ba01-ab29f65200de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:08:50.510914 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e72df0-1e09-4399-ba01-ab29f65200de None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:08:50.511039 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e72df0-1e09-4399-ba01-ab29f65200de None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:08:50.511315 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e72df0-1e09-4399-ba01-ab29f65200de None admin] PATH_INFO: `/v3/services` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:08:50.511940 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93e72df0-1e09-4399-ba01-ab29f65200de None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 03:08:50.512249 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93e72df0-1e09-4399-ba01-ab29f65200de None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:08:50.513644 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93e72df0-1e09-4399-ba01-ab29f65200de None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:08:50.518982 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 463/919] 10.222.0.18 () {56 vars in 1122 bytes} [Tue Sep 13 03:08:50 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 03:08:50.524069 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93e72df0-1e09-4399-ba01-ab29f65200de None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:08:50.602919 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10c9ba54-63fb-4779-a1c3-73ff990b4cd7 None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:08:50.645807 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93e72df0-1e09-4399-ba01-ab29f65200de None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:08:50.646139 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9e3f288-7842-4e08-a43a-45aeeb29f4f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:08:50.646798 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e3f288-7842-4e08-a43a-45aeeb29f4f8 None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:08:50.647082 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e3f288-7842-4e08-a43a-45aeeb29f4f8 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:08:50.647358 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e3f288-7842-4e08-a43a-45aeeb29f4f8 None admin] PATH_INFO: `/v3/services` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:08:50.647986 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9e3f288-7842-4e08-a43a-45aeeb29f4f8 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=81399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 03:08:50.648292 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9e3f288-7842-4e08-a43a-45aeeb29f4f8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:08:50.649882 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9e3f288-7842-4e08-a43a-45aeeb29f4f8 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:08:50.658674 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 457/920] 10.222.0.18 () {56 vars in 1126 bytes} [Tue Sep 13 03:08:50 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 03:08:50.660934 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9e3f288-7842-4e08-a43a-45aeeb29f4f8 None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:08:50.673400 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:08:50.697219 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24bacf1b-75fd-413d-877f-ca1caa45e498 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:08:50.698177 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24bacf1b-75fd-413d-877f-ca1caa45e498 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:08:50.698699 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24bacf1b-75fd-413d-877f-ca1caa45e498 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:08:50.699264 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24bacf1b-75fd-413d-877f-ca1caa45e498 None admin] PATH_INFO: `/v3/services` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:08:50.700170 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24bacf1b-75fd-413d-877f-ca1caa45e498 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 03:08:50.700677 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24bacf1b-75fd-413d-877f-ca1caa45e498 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:08:50.702397 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24bacf1b-75fd-413d-877f-ca1caa45e498 None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:08:50.708116 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 464/921] 10.222.0.18 () {56 vars in 1122 bytes} [Tue Sep 13 03:08:50 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 03:08:50.713389 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24bacf1b-75fd-413d-877f-ca1caa45e498 None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:08:50.734713 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:08:50.778327 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9e3f288-7842-4e08-a43a-45aeeb29f4f8 None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:08:50.815519 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-897b82ce-4bb7-4c58-abc0-236688c003bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:08:50.816146 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-897b82ce-4bb7-4c58-abc0-236688c003bb None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:08:50.816427 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-897b82ce-4bb7-4c58-abc0-236688c003bb None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:08:50.816699 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-897b82ce-4bb7-4c58-abc0-236688c003bb None admin] PATH_INFO: `/v3/services` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:08:50.817343 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-897b82ce-4bb7-4c58-abc0-236688c003bb None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=81399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 03:08:50.817652 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-897b82ce-4bb7-4c58-abc0-236688c003bb None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:08:50.819361 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-897b82ce-4bb7-4c58-abc0-236688c003bb None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:08:50.825786 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 458/922] 10.222.0.18 () {56 vars in 1126 bytes} [Tue Sep 13 03:08:50 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 03:08:50.826762 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24bacf1b-75fd-413d-877f-ca1caa45e498 None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:08:50.833077 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-897b82ce-4bb7-4c58-abc0-236688c003bb None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:08:50.872353 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3568dbb5-bce8-416a-90a3-3f110cfbac3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:08:50.872906 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3568dbb5-bce8-416a-90a3-3f110cfbac3f None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:08:50.873183 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3568dbb5-bce8-416a-90a3-3f110cfbac3f None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:08:50.873594 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3568dbb5-bce8-416a-90a3-3f110cfbac3f None admin] PATH_INFO: `/v3/endpoints` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:08:50.874588 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3568dbb5-bce8-416a-90a3-3f110cfbac3f None admin] RBAC: Adding query filter params (interface=public, service_id=a7a456d8462644079858efd21c4a1488, region_id=RegionOne) {{(pid=81400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 03:08:50.874930 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3568dbb5-bce8-416a-90a3-3f110cfbac3f None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:08:50.876612 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3568dbb5-bce8-416a-90a3-3f110cfbac3f None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:08:50.887783 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 465/923] 10.222.0.18 () {56 vars in 1251 bytes} [Tue Sep 13 03:08:50 2022] GET /identity/v3/endpoints?service_id=a7a456d8462644079858efd21c4a1488&interface=public®ion_id=RegionOne => generated 194 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 03:08:50.890686 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3568dbb5-bce8-416a-90a3-3f110cfbac3f None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:08:50.945162 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:08:50.958826 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-897b82ce-4bb7-4c58-abc0-236688c003bb None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:08:50.985499 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:08:50.994576 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d469ed53-4b18-4fdf-ba8c-68b6579acc82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:08:50.995220 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d469ed53-4b18-4fdf-ba8c-68b6579acc82 None admin] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:08:50.995501 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d469ed53-4b18-4fdf-ba8c-68b6579acc82 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:08:50.995778 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d469ed53-4b18-4fdf-ba8c-68b6579acc82 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:08:50.996441 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d469ed53-4b18-4fdf-ba8c-68b6579acc82 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:08:50.997815 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d469ed53-4b18-4fdf-ba8c-68b6579acc82 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:08:51.018828 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3568dbb5-bce8-416a-90a3-3f110cfbac3f None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:08:51.026782 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 459/924] 10.222.0.18 () {60 vars in 1144 bytes} [Tue Sep 13 03:08:50 2022] POST /identity/v3/endpoints => generated 347 bytes in 187 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 13 03:08:51.026782 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e80fcb-c442-49be-bfff-ccc7928265b6 None None] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:08:51.026782 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e80fcb-c442-49be-bfff-ccc7928265b6 None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:08:51.026782 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e80fcb-c442-49be-bfff-ccc7928265b6 None None] PATH_INFO: `/` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:08:51.026782 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 460/925] 10.222.0.18 () {54 vars in 927 bytes} [Tue Sep 13 03:08:51 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 03:08:51.028378 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d69e874-5e42-4840-ab51-74764d400248 None None] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:08:51.028663 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d69e874-5e42-4840-ab51-74764d400248 None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:08:51.030401 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d69e874-5e42-4840-ab51-74764d400248 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:08:51.049487 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.auth.core [None req-9d69e874-5e42-4840-ab51-74764d400248 None None] MFA Rules not processed for user `49f280ab086b4610855c4ffa378517d9`. Rule list: `[]` (Enabled: `True`). {{(pid=81399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 03:08:51.050738 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-9d69e874-5e42-4840-ab51-74764d400248 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:08:51.095802 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b336c8c-be50-46f8-b9e1-045566ede1e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:08:51.096550 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b336c8c-be50-46f8-b9e1-045566ede1e2 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:08:51.096881 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b336c8c-be50-46f8-b9e1-045566ede1e2 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:08:51.097155 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b336c8c-be50-46f8-b9e1-045566ede1e2 None admin] PATH_INFO: `/v3/services` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:08:51.097832 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b336c8c-be50-46f8-b9e1-045566ede1e2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:08:51.100712 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b336c8c-be50-46f8-b9e1-045566ede1e2 None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:08:51.108011 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 466/926] 10.222.0.18 () {56 vars in 1087 bytes} [Tue Sep 13 03:08:50 2022] GET /identity/v3/services => generated 1497 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 03:08:51.150021 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 461/927] 10.222.0.18 () {58 vars in 1024 bytes} [Tue Sep 13 03:08:51 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 13 03:08:51.390443 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-735f0660-0a08-458e-9f7f-5e0f42f0e33c None None] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:08:51.390837 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-735f0660-0a08-458e-9f7f-5e0f42f0e33c None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:08:51.391033 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-735f0660-0a08-458e-9f7f-5e0f42f0e33c None None] PATH_INFO: `/` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:08:51.392617 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 467/928] 10.222.0.18 () {54 vars in 927 bytes} [Tue Sep 13 03:08:51 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 03:08:51.397798 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b775f4b4-6b60-4cb4-96e5-c0748cbf315b None None] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:08:51.398287 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b775f4b4-6b60-4cb4-96e5-c0748cbf315b None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:08:51.398366 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b775f4b4-6b60-4cb4-96e5-c0748cbf315b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:08:51.416546 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.auth.core [None req-b775f4b4-6b60-4cb4-96e5-c0748cbf315b None None] MFA Rules not processed for user `49f280ab086b4610855c4ffa378517d9`. Rule list: `[]` (Enabled: `True`). {{(pid=81399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 03:08:51.417649 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-b775f4b4-6b60-4cb4-96e5-c0748cbf315b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:08:51.425915 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-735f0660-0a08-458e-9f7f-5e0f42f0e33c None None] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:08:51.489404 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 462/929] 10.222.0.18 () {58 vars in 1024 bytes} [Tue Sep 13 03:08:51 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 13 03:08:51.545734 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-735f0660-0a08-458e-9f7f-5e0f42f0e33c None None] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:08:51.563075 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:08:51.583532 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b31d217-09c7-4c1b-aa35-ca952a9974e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:08:51.584168 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b31d217-09c7-4c1b-aa35-ca952a9974e2 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:08:51.584445 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b31d217-09c7-4c1b-aa35-ca952a9974e2 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:08:51.584718 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b31d217-09c7-4c1b-aa35-ca952a9974e2 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:08:51.585296 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b31d217-09c7-4c1b-aa35-ca952a9974e2 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:08:51.586813 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b31d217-09c7-4c1b-aa35-ca952a9974e2 None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:08:51.595931 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application [None req-6b31d217-09c7-4c1b-aa35-ca952a9974e2 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 13 03:08:51.595931 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 03:08:51.595931 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 13 03:08:51.595931 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 13 03:08:51.595931 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 13 03:08:51.595931 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 13 03:08:51.595931 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 13 03:08:51.595931 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 13 03:08:51.595931 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 13 03:08:51.595931 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Sep 13 03:08:51.595931 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 13 03:08:51.595931 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Sep 13 03:08:51.595931 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 03:08:51.595931 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 03:08:51.595931 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 03:08:51.595931 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 03:08:51.595931 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 03:08:51.596912 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 03:08:51.596912 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 03:08:51.596912 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 03:08:51.596912 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 03:08:51.596912 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 03:08:51.596912 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 03:08:51.596912 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 13 03:08:51.596912 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 13 03:08:51.596912 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 13 03:08:51.596912 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 13 03:08:51.596912 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 03:08:51.596912 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 03:08:51.596912 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 03:08:51.596912 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 03:08:51.596912 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 03:08:51.596912 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 03:08:51.597768 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 03:08:51.597768 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application with Lock( Sep 13 03:08:51.597768 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 03:08:51.597768 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._enter() Sep 13 03:08:51.597768 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 03:08:51.597768 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 03:08:51.597768 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 03:08:51.597768 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.creator() Sep 13 03:08:51.597768 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 03:08:51.597768 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application created_value = creator( Sep 13 03:08:51.597768 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 13 03:08:51.597768 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 13 03:08:51.597768 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 13 03:08:51.597768 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Sep 13 03:08:51.600961 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 468/930] 10.222.0.18 () {56 vars in 1114 bytes} [Tue Sep 13 03:08:51 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 03:08:51.603498 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b775f4b4-6b60-4cb4-96e5-c0748cbf315b None None] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:08:51.650304 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:08:51.711961 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b775f4b4-6b60-4cb4-96e5-c0748cbf315b None None] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:08:51.751822 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1150a61-a642-41ac-b075-08240d71e1b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:08:51.752429 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1150a61-a642-41ac-b075-08240d71e1b5 None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:08:51.752705 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1150a61-a642-41ac-b075-08240d71e1b5 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:08:51.752977 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1150a61-a642-41ac-b075-08240d71e1b5 None admin] PATH_INFO: `/v3/services` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:08:51.753600 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1150a61-a642-41ac-b075-08240d71e1b5 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=81399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 03:08:51.753900 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1150a61-a642-41ac-b075-08240d71e1b5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:08:51.755278 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1150a61-a642-41ac-b075-08240d71e1b5 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:08:51.760423 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 463/931] 10.222.0.18 () {56 vars in 1114 bytes} [Tue Sep 13 03:08:51 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 03:08:51.768008 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b31d217-09c7-4c1b-aa35-ca952a9974e2 None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:08:51.773279 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1150a61-a642-41ac-b075-08240d71e1b5 None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:08:51.885846 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b31d217-09c7-4c1b-aa35-ca952a9974e2 None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:08:51.890200 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1150a61-a642-41ac-b075-08240d71e1b5 None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:08:51.899225 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:08:51.924994 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58612b36-407d-42fd-8783-fbdff942af01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:08:51.925935 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58612b36-407d-42fd-8783-fbdff942af01 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:08:51.926474 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58612b36-407d-42fd-8783-fbdff942af01 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:08:51.926907 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58612b36-407d-42fd-8783-fbdff942af01 None admin] PATH_INFO: `/v3/services` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:08:51.927691 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58612b36-407d-42fd-8783-fbdff942af01 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=81400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 03:08:51.928173 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58612b36-407d-42fd-8783-fbdff942af01 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:08:51.929953 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58612b36-407d-42fd-8783-fbdff942af01 None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:08:51.931639 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26251cb5-6575-4ed0-84f5-a84bc1dfaa8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:08:51.932274 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26251cb5-6575-4ed0-84f5-a84bc1dfaa8d None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:08:51.932556 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26251cb5-6575-4ed0-84f5-a84bc1dfaa8d None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:08:51.933144 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26251cb5-6575-4ed0-84f5-a84bc1dfaa8d None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:08:51.936395 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 469/932] 10.222.0.18 () {56 vars in 1114 bytes} [Tue Sep 13 03:08:51 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 03:08:51.944182 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58612b36-407d-42fd-8783-fbdff942af01 None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:08:51.949082 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26251cb5-6575-4ed0-84f5-a84bc1dfaa8d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 03:08:51.954216 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application [None req-26251cb5-6575-4ed0-84f5-a84bc1dfaa8d None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 13 03:08:51.954216 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 03:08:51.954216 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 03:08:51.954216 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 03:08:51.954216 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 03:08:51.954216 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 03:08:51.954216 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 03:08:51.954216 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 03:08:51.954216 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 03:08:51.954216 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 03:08:51.954216 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 03:08:51.954216 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 03:08:51.954216 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 03:08:51.954216 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 03:08:51.954216 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 03:08:51.954216 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 03:08:51.955174 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 03:08:51.955174 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 03:08:51.955174 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 03:08:51.955174 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 03:08:51.955174 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 03:08:51.955174 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 03:08:51.955174 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 03:08:51.955174 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 03:08:51.955174 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 03:08:51.955174 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 03:08:51.955174 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 03:08:51.955174 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application with Lock( Sep 13 03:08:51.955174 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 03:08:51.955174 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._enter() Sep 13 03:08:51.955174 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 03:08:51.955174 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 03:08:51.956027 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 03:08:51.956027 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.creator() Sep 13 03:08:51.956027 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 03:08:51.956027 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application created_value = creator( Sep 13 03:08:51.956027 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 03:08:51.956027 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 03:08:51.956027 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 03:08:51.956027 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 03:08:51.956027 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 03:08:51.956027 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 03:08:51.956027 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 13 03:08:51.956027 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Sep 13 03:08:51.956027 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 464/933] 10.222.0.18 () {56 vars in 1120 bytes} [Tue Sep 13 03:08:51 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 13 03:08:51.963530 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26251cb5-6575-4ed0-84f5-a84bc1dfaa8d None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:08:51.984566 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:08:52.086086 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58612b36-407d-42fd-8783-fbdff942af01 None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:08:52.092835 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26251cb5-6575-4ed0-84f5-a84bc1dfaa8d None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:08:52.130362 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1f60996-7827-4e28-b4be-6e97ad62124d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:08:52.131341 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f60996-7827-4e28-b4be-6e97ad62124d None admin] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:08:52.131796 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f60996-7827-4e28-b4be-6e97ad62124d None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:08:52.132067 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74e03df5-1fa4-4b87-acd7-26205892926d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:08:52.132468 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f60996-7827-4e28-b4be-6e97ad62124d None admin] PATH_INFO: `/v3/endpoints` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:08:52.132806 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e03df5-1fa4-4b87-acd7-26205892926d None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:08:52.133156 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e03df5-1fa4-4b87-acd7-26205892926d None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:08:52.133500 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e03df5-1fa4-4b87-acd7-26205892926d None admin] PATH_INFO: `/v3/roles` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:08:52.133875 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1f60996-7827-4e28-b4be-6e97ad62124d None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:08:52.134280 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74e03df5-1fa4-4b87-acd7-26205892926d None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 03:08:52.134678 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74e03df5-1fa4-4b87-acd7-26205892926d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:08:52.135816 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1f60996-7827-4e28-b4be-6e97ad62124d None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:08:52.136226 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74e03df5-1fa4-4b87-acd7-26205892926d None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:08:52.149773 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 465/934] 10.222.0.18 () {56 vars in 1115 bytes} [Tue Sep 13 03:08:51 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 03:08:52.150799 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74e03df5-1fa4-4b87-acd7-26205892926d None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:08:52.168020 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 470/935] 10.222.0.18 () {60 vars in 1144 bytes} [Tue Sep 13 03:08:51 2022] POST /identity/v3/endpoints => generated 346 bytes in 224 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 03:08:52.189715 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:08:52.272374 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74e03df5-1fa4-4b87-acd7-26205892926d None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:08:52.316342 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-323e978f-6177-4ada-9af0-1e5f7a037731 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:08:52.317392 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-323e978f-6177-4ada-9af0-1e5f7a037731 None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:08:52.317871 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-323e978f-6177-4ada-9af0-1e5f7a037731 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:08:52.318361 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-323e978f-6177-4ada-9af0-1e5f7a037731 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:08:52.331881 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-323e978f-6177-4ada-9af0-1e5f7a037731 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:08:52.333732 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-323e978f-6177-4ada-9af0-1e5f7a037731 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:08:52.350192 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application [None req-323e978f-6177-4ada-9af0-1e5f7a037731 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 03:08:52.350192 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 03:08:52.350192 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 03:08:52.350192 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 03:08:52.350192 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 03:08:52.350192 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 03:08:52.350192 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 03:08:52.350192 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 03:08:52.350192 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 03:08:52.350192 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 03:08:52.350192 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 03:08:52.350192 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 03:08:52.350192 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 13 03:08:52.350192 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 13 03:08:52.350192 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 13 03:08:52.350192 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 13 03:08:52.351307 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 03:08:52.351307 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 03:08:52.351307 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 03:08:52.351307 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 03:08:52.351307 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 03:08:52.351307 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 03:08:52.351307 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 03:08:52.351307 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application with Lock( Sep 13 03:08:52.351307 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 03:08:52.351307 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._enter() Sep 13 03:08:52.351307 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 03:08:52.351307 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 03:08:52.351307 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 03:08:52.351307 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.creator() Sep 13 03:08:52.351307 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 03:08:52.351307 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application created_value = creator( Sep 13 03:08:52.351307 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 13 03:08:52.352224 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 13 03:08:52.352224 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 03:08:52.352224 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Sep 13 03:08:52.354332 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 466/936] 10.222.0.18 () {56 vars in 1108 bytes} [Tue Sep 13 03:08:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 03:08:52.360334 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1f60996-7827-4e28-b4be-6e97ad62124d None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:08:52.453848 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:08:52.469936 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1f60996-7827-4e28-b4be-6e97ad62124d None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:08:52.504861 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f910df63-de1c-429d-aeb8-b9333daacecc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:08:52.505463 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f910df63-de1c-429d-aeb8-b9333daacecc None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:08:52.505739 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f910df63-de1c-429d-aeb8-b9333daacecc None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:08:52.506008 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f910df63-de1c-429d-aeb8-b9333daacecc None admin] PATH_INFO: `/v3/domains` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:08:52.506664 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f910df63-de1c-429d-aeb8-b9333daacecc None admin] RBAC: Adding query filter params (name=Default) {{(pid=81400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 03:08:52.506971 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f910df63-de1c-429d-aeb8-b9333daacecc None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:08:52.508327 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f910df63-de1c-429d-aeb8-b9333daacecc None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:08:52.520971 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 471/937] 10.222.0.18 () {56 vars in 1109 bytes} [Tue Sep 13 03:08:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 03:08:52.526356 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-323e978f-6177-4ada-9af0-1e5f7a037731 None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:08:52.607023 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:08:52.642168 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-323e978f-6177-4ada-9af0-1e5f7a037731 None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:08:52.680411 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-479d4187-654c-466e-85b2-345ae70c3dbb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:08:52.681044 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-479d4187-654c-466e-85b2-345ae70c3dbb None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:08:52.681318 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-479d4187-654c-466e-85b2-345ae70c3dbb None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:08:52.681588 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-479d4187-654c-466e-85b2-345ae70c3dbb None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:08:52.689792 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-479d4187-654c-466e-85b2-345ae70c3dbb None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:08:52.691274 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-479d4187-654c-466e-85b2-345ae70c3dbb None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:08:52.703640 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application [None req-479d4187-654c-466e-85b2-345ae70c3dbb None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 13 03:08:52.703640 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 03:08:52.703640 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 03:08:52.703640 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 03:08:52.703640 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 03:08:52.703640 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 03:08:52.703640 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 03:08:52.703640 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 03:08:52.703640 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 03:08:52.703640 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 03:08:52.703640 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 03:08:52.703640 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 03:08:52.703640 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 13 03:08:52.703640 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 13 03:08:52.703640 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 13 03:08:52.703640 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 13 03:08:52.704673 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 03:08:52.704673 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 03:08:52.704673 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 13 03:08:52.704673 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 13 03:08:52.704673 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 13 03:08:52.704673 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 13 03:08:52.704673 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 03:08:52.704673 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 03:08:52.704673 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 03:08:52.704673 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 03:08:52.704673 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 03:08:52.704673 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application with Lock( Sep 13 03:08:52.704673 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 03:08:52.704673 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._enter() Sep 13 03:08:52.704673 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 03:08:52.704673 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 03:08:52.704673 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 03:08:52.704673 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.creator() Sep 13 03:08:52.706257 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 03:08:52.706257 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application created_value = creator( Sep 13 03:08:52.706257 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 13 03:08:52.706257 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 13 03:08:52.706257 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 13 03:08:52.706257 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 13 03:08:52.706257 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 13 03:08:52.706257 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 13 03:08:52.706257 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 13 03:08:52.706257 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Sep 13 03:08:52.706257 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 467/938] 10.222.0.18 () {56 vars in 1117 bytes} [Tue Sep 13 03:08:52 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 03:08:52.710719 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f910df63-de1c-429d-aeb8-b9333daacecc None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:08:52.815319 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f910df63-de1c-429d-aeb8-b9333daacecc None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:08:52.850299 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52c089d0-6a26-413b-acf5-a615b1959594 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:08:52.850939 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52c089d0-6a26-413b-acf5-a615b1959594 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:08:52.851215 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52c089d0-6a26-413b-acf5-a615b1959594 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:08:52.851486 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52c089d0-6a26-413b-acf5-a615b1959594 None admin] PATH_INFO: `/v3/users` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:08:52.852214 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52c089d0-6a26-413b-acf5-a615b1959594 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 03:08:52.852514 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52c089d0-6a26-413b-acf5-a615b1959594 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:08:52.854051 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52c089d0-6a26-413b-acf5-a615b1959594 None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:08:52.873482 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 472/939] 10.222.0.18 () {56 vars in 1149 bytes} [Tue Sep 13 03:08:52 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 03:08:52.879232 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-479d4187-654c-466e-85b2-345ae70c3dbb None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:08:52.985200 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-479d4187-654c-466e-85b2-345ae70c3dbb None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:08:53.021580 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6123bdd-f653-4ebd-949b-0b95572716d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:08:53.022214 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6123bdd-f653-4ebd-949b-0b95572716d4 None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:08:53.022490 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6123bdd-f653-4ebd-949b-0b95572716d4 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:08:53.022792 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6123bdd-f653-4ebd-949b-0b95572716d4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:08:53.025520 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:08:53.034305 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6123bdd-f653-4ebd-949b-0b95572716d4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:08:53.035808 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6123bdd-f653-4ebd-949b-0b95572716d4 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:08:53.051372 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application [None req-c6123bdd-f653-4ebd-949b-0b95572716d4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 03:08:53.051372 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 03:08:53.051372 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 03:08:53.051372 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 03:08:53.051372 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 03:08:53.051372 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 03:08:53.051372 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 03:08:53.051372 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 03:08:53.051372 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 03:08:53.051372 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 03:08:53.051372 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 03:08:53.051372 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 03:08:53.051372 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 13 03:08:53.051372 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 13 03:08:53.051372 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 13 03:08:53.051372 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 13 03:08:53.053648 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 03:08:53.053648 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 03:08:53.053648 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 03:08:53.053648 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 03:08:53.053648 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 03:08:53.053648 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 03:08:53.053648 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 03:08:53.053648 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application with Lock( Sep 13 03:08:53.053648 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 03:08:53.053648 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._enter() Sep 13 03:08:53.053648 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 03:08:53.053648 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 03:08:53.053648 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 03:08:53.053648 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.creator() Sep 13 03:08:53.053648 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 03:08:53.053648 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application created_value = creator( Sep 13 03:08:53.053648 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 13 03:08:53.056167 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 13 03:08:53.056167 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 03:08:53.056167 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Sep 13 03:08:53.056167 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 468/940] 10.222.0.18 () {56 vars in 1108 bytes} [Tue Sep 13 03:08:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 03:08:53.058874 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52c089d0-6a26-413b-acf5-a615b1959594 None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:08:53.069086 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:08:53.172142 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52c089d0-6a26-413b-acf5-a615b1959594 None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:08:53.208582 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-195b22fb-7a5a-4cea-a9e1-5fe7e4699824 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:08:53.209190 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-195b22fb-7a5a-4cea-a9e1-5fe7e4699824 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:08:53.209465 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-195b22fb-7a5a-4cea-a9e1-5fe7e4699824 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:08:53.209733 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-195b22fb-7a5a-4cea-a9e1-5fe7e4699824 None admin] PATH_INFO: `/v3/domains` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:08:53.210370 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-195b22fb-7a5a-4cea-a9e1-5fe7e4699824 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 03:08:53.210690 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-195b22fb-7a5a-4cea-a9e1-5fe7e4699824 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:08:53.212051 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-195b22fb-7a5a-4cea-a9e1-5fe7e4699824 None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:08:53.223950 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 473/941] 10.222.0.18 () {56 vars in 1109 bytes} [Tue Sep 13 03:08:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 03:08:53.229326 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6123bdd-f653-4ebd-949b-0b95572716d4 None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:08:53.342027 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6123bdd-f653-4ebd-949b-0b95572716d4 None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:08:53.377057 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ca93d6f-a304-41dc-83ff-39417ef415c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:08:53.377690 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ca93d6f-a304-41dc-83ff-39417ef415c4 None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:08:53.377968 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ca93d6f-a304-41dc-83ff-39417ef415c4 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:08:53.378241 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ca93d6f-a304-41dc-83ff-39417ef415c4 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:08:53.385115 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ca93d6f-a304-41dc-83ff-39417ef415c4 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:08:53.386963 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ca93d6f-a304-41dc-83ff-39417ef415c4 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:08:53.387189 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b873080-2fb8-4401-b94f-8e3e472051d1 None None] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:08:53.387364 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b873080-2fb8-4401-b94f-8e3e472051d1 None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:08:53.387640 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b873080-2fb8-4401-b94f-8e3e472051d1 None None] PATH_INFO: `/` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:08:53.389143 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 474/942] 10.222.0.18 () {54 vars in 927 bytes} [Tue Sep 13 03:08:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 13 03:08:53.390148 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21bb091a-2314-4058-aba9-35659f03fac1 None None] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:08:53.390421 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21bb091a-2314-4058-aba9-35659f03fac1 None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:08:53.394470 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21bb091a-2314-4058-aba9-35659f03fac1 None None] PATH_INFO: `/` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:08:53.395882 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 475/943] 10.222.0.18 () {54 vars in 927 bytes} [Tue Sep 13 03:08:53 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 03:08:53.396777 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f0fc9ce-3da0-49df-a23f-5ce407fb5e7e None None] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:08:53.397054 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f0fc9ce-3da0-49df-a23f-5ce407fb5e7e None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:08:53.397321 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f0fc9ce-3da0-49df-a23f-5ce407fb5e7e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:08:53.399266 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application [None req-5ca93d6f-a304-41dc-83ff-39417ef415c4 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 13 03:08:53.399266 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 03:08:53.399266 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 03:08:53.399266 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 03:08:53.399266 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 03:08:53.399266 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 03:08:53.399266 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 03:08:53.399266 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 03:08:53.399266 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 03:08:53.399266 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 03:08:53.399266 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 03:08:53.399266 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 03:08:53.399266 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 13 03:08:53.399266 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 13 03:08:53.399266 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 13 03:08:53.400161 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 13 03:08:53.400161 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 03:08:53.400161 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 03:08:53.400161 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 03:08:53.400161 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 03:08:53.400161 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 03:08:53.400161 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 03:08:53.400161 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 03:08:53.400161 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application with Lock( Sep 13 03:08:53.400161 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 03:08:53.400161 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._enter() Sep 13 03:08:53.400161 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 03:08:53.400161 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 03:08:53.400161 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 03:08:53.400161 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.creator() Sep 13 03:08:53.400161 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 03:08:53.400161 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application created_value = creator( Sep 13 03:08:53.402731 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 13 03:08:53.402731 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 13 03:08:53.402731 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 13 03:08:53.402731 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 13 03:08:53.402731 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 13 03:08:53.402731 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 13 03:08:53.402731 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 13 03:08:53.402731 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Sep 13 03:08:53.402731 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 469/944] 10.222.0.18 () {56 vars in 1111 bytes} [Tue Sep 13 03:08:53 2022] GET /identity/v3/projects/service => generated 88 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 03:08:53.403888 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f84aaf60-576c-405b-a139-016348b07508 None None] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:08:53.403888 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f84aaf60-576c-405b-a139-016348b07508 None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:08:53.403888 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f84aaf60-576c-405b-a139-016348b07508 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:08:53.422053 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.auth.core [None req-f84aaf60-576c-405b-a139-016348b07508 None None] MFA Rules not processed for user `49f280ab086b4610855c4ffa378517d9`. Rule list: `[]` (Enabled: `True`). {{(pid=81399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 03:08:53.423264 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-f84aaf60-576c-405b-a139-016348b07508 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:08:53.425502 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.auth.core [None req-0f0fc9ce-3da0-49df-a23f-5ce407fb5e7e None None] MFA Rules not processed for user `49f280ab086b4610855c4ffa378517d9`. Rule list: `[]` (Enabled: `True`). {{(pid=81400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 03:08:53.427320 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-0f0fc9ce-3da0-49df-a23f-5ce407fb5e7e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:08:53.470594 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:08:53.498856 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 470/945] 10.222.0.18 () {58 vars in 1024 bytes} [Tue Sep 13 03:08:53 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 13 03:08:53.498856 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f84aaf60-576c-405b-a139-016348b07508 None None] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:08:53.501619 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:08:53.518636 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 476/946] 10.222.0.18 () {58 vars in 1024 bytes} [Tue Sep 13 03:08:53 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 13 03:08:53.605460 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f84aaf60-576c-405b-a139-016348b07508 None None] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:08:53.640633 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b969034b-978d-4363-9bec-bf64a907ce3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:08:53.641234 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b969034b-978d-4363-9bec-bf64a907ce3c None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:08:53.641509 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b969034b-978d-4363-9bec-bf64a907ce3c None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:08:53.641778 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b969034b-978d-4363-9bec-bf64a907ce3c None admin] PATH_INFO: `/v3/projects` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:08:53.642457 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b969034b-978d-4363-9bec-bf64a907ce3c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 03:08:53.642788 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b969034b-978d-4363-9bec-bf64a907ce3c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:08:53.644197 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b969034b-978d-4363-9bec-bf64a907ce3c None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:08:53.655968 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 471/947] 10.222.0.18 () {56 vars in 1148 bytes} [Tue Sep 13 03:08:53 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 03:08:53.663674 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f0fc9ce-3da0-49df-a23f-5ce407fb5e7e None None] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:08:53.766511 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f0fc9ce-3da0-49df-a23f-5ce407fb5e7e None None] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:08:53.769833 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b969034b-978d-4363-9bec-bf64a907ce3c None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:08:53.804510 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3285f8c1-65d1-48df-92aa-80535ed567e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:08:53.805436 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3285f8c1-65d1-48df-92aa-80535ed567e5 None admin] REQUEST_METHOD: `PUT` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:08:53.805712 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3285f8c1-65d1-48df-92aa-80535ed567e5 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:08:53.805986 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3285f8c1-65d1-48df-92aa-80535ed567e5 None admin] PATH_INFO: `/v3/projects/b9d94f6a68f7470d9494f112175620dc/users/4c2d20fb54a54ecc99b47f237be4917c/roles/57ec3ac460b749a5910aea16ad1d643e` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:08:53.808880 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3285f8c1-65d1-48df-92aa-80535ed567e5 None admin] RBAC: Authorizing `identity:create_grant(project_id=b9d94f6a68f7470d9494f112175620dc, user_id=4c2d20fb54a54ecc99b47f237be4917c, role_id=57ec3ac460b749a5910aea16ad1d643e)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:08:53.811080 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3285f8c1-65d1-48df-92aa-80535ed567e5 None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:08:53.816176 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:08:53.825584 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 477/948] 10.222.0.18 () {58 vars in 1439 bytes} [Tue Sep 13 03:08:53 2022] PUT /identity/v3/projects/b9d94f6a68f7470d9494f112175620dc/users/4c2d20fb54a54ecc99b47f237be4917c/roles/57ec3ac460b749a5910aea16ad1d643e => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 03:08:53.826403 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3285f8c1-65d1-48df-92aa-80535ed567e5 None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:08:53.899650 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:08:53.902294 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b969034b-978d-4363-9bec-bf64a907ce3c None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:08:53.950071 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cd96d08-a4c5-43f9-80b3-0da3326826ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:08:53.951281 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd96d08-a4c5-43f9-80b3-0da3326826ee None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:08:53.952033 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd96d08-a4c5-43f9-80b3-0da3326826ee None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:08:53.952585 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd96d08-a4c5-43f9-80b3-0da3326826ee None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:08:53.953750 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cd96d08-a4c5-43f9-80b3-0da3326826ee None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:08:53.955828 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cd96d08-a4c5-43f9-80b3-0da3326826ee None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:08:53.959471 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3285f8c1-65d1-48df-92aa-80535ed567e5 None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:08:53.966726 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application [None req-0cd96d08-a4c5-43f9-80b3-0da3326826ee None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 13 03:08:53.966726 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 03:08:53.966726 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 13 03:08:53.966726 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 13 03:08:53.966726 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 13 03:08:53.966726 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 13 03:08:53.966726 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 13 03:08:53.966726 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 13 03:08:53.966726 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 13 03:08:53.966726 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Sep 13 03:08:53.966726 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 13 03:08:53.966726 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Sep 13 03:08:53.966726 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 03:08:53.966726 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 03:08:53.966726 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 03:08:53.966726 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 03:08:53.966726 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 03:08:53.967831 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 03:08:53.967831 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 03:08:53.967831 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 03:08:53.967831 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 03:08:53.967831 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 03:08:53.967831 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 03:08:53.967831 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 13 03:08:53.967831 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 13 03:08:53.967831 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 13 03:08:53.967831 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 13 03:08:53.967831 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 03:08:53.967831 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 03:08:53.967831 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 03:08:53.967831 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 03:08:53.967831 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 03:08:53.967831 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 03:08:53.968680 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 03:08:53.968680 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application with Lock( Sep 13 03:08:53.968680 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 03:08:53.968680 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._enter() Sep 13 03:08:53.968680 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 03:08:53.968680 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 03:08:53.968680 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 03:08:53.968680 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.creator() Sep 13 03:08:53.968680 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 03:08:53.968680 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application created_value = creator( Sep 13 03:08:53.968680 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 13 03:08:53.968680 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 13 03:08:53.968680 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 13 03:08:53.968680 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Sep 13 03:08:53.971682 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 472/949] 10.222.0.18 () {56 vars in 1126 bytes} [Tue Sep 13 03:08:53 2022] GET /identity/v3/services/object-store => generated 93 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 13 03:08:53.979924 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cd96d08-a4c5-43f9-80b3-0da3326826ee None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:08:54.005494 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b16182cc-cb45-482e-9d1c-fcdf255084ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:08:54.006509 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b16182cc-cb45-482e-9d1c-fcdf255084ec None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:08:54.007033 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b16182cc-cb45-482e-9d1c-fcdf255084ec None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:08:54.007588 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b16182cc-cb45-482e-9d1c-fcdf255084ec None admin] PATH_INFO: `/v3/services/compute` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:08:54.008401 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b16182cc-cb45-482e-9d1c-fcdf255084ec None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:08:54.010132 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b16182cc-cb45-482e-9d1c-fcdf255084ec None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:08:54.020533 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application [None req-b16182cc-cb45-482e-9d1c-fcdf255084ec None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Sep 13 03:08:54.020533 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 03:08:54.020533 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 13 03:08:54.020533 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 13 03:08:54.020533 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 13 03:08:54.020533 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 13 03:08:54.020533 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 13 03:08:54.020533 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 13 03:08:54.020533 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Sep 13 03:08:54.020533 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Sep 13 03:08:54.020533 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 13 03:08:54.020533 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Sep 13 03:08:54.020533 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 03:08:54.020533 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 03:08:54.020533 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 03:08:54.020533 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 03:08:54.020533 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 03:08:54.021615 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 03:08:54.021615 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 03:08:54.021615 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 03:08:54.021615 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 03:08:54.021615 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 03:08:54.021615 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 03:08:54.021615 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 13 03:08:54.021615 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 13 03:08:54.021615 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 13 03:08:54.021615 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 13 03:08:54.021615 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 03:08:54.021615 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 03:08:54.021615 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 03:08:54.021615 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 03:08:54.021615 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 03:08:54.021615 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 03:08:54.022475 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 03:08:54.022475 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application with Lock( Sep 13 03:08:54.022475 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 03:08:54.022475 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._enter() Sep 13 03:08:54.022475 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 03:08:54.022475 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 03:08:54.022475 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 03:08:54.022475 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.creator() Sep 13 03:08:54.022475 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 03:08:54.022475 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application created_value = creator( Sep 13 03:08:54.022475 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 13 03:08:54.022475 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 13 03:08:54.022475 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Sep 13 03:08:54.022475 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Sep 13 03:08:54.030439 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 478/950] 10.222.0.18 () {56 vars in 1111 bytes} [Tue Sep 13 03:08:53 2022] GET /identity/v3/services/compute => generated 88 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 03:08:54.031877 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b16182cc-cb45-482e-9d1c-fcdf255084ec None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:08:54.095131 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cd96d08-a4c5-43f9-80b3-0da3326826ee None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:08:54.096941 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:08:54.144082 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7570ff2-9718-4d76-8c20-bb528cb2ba72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:08:54.145071 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7570ff2-9718-4d76-8c20-bb528cb2ba72 None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:08:54.146078 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7570ff2-9718-4d76-8c20-bb528cb2ba72 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:08:54.146555 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7570ff2-9718-4d76-8c20-bb528cb2ba72 None admin] PATH_INFO: `/v3/services` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:08:54.147711 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7570ff2-9718-4d76-8c20-bb528cb2ba72 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 03:08:54.148501 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7570ff2-9718-4d76-8c20-bb528cb2ba72 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:08:54.150889 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7570ff2-9718-4d76-8c20-bb528cb2ba72 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:08:54.158527 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b16182cc-cb45-482e-9d1c-fcdf255084ec None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:08:54.159370 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 473/951] 10.222.0.18 () {56 vars in 1122 bytes} [Tue Sep 13 03:08:53 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 03:08:54.171249 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7570ff2-9718-4d76-8c20-bb528cb2ba72 None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:08:54.182310 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:08:54.201347 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed613fa1-c23b-4dd9-bbcd-96d565970a17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:08:54.202000 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed613fa1-c23b-4dd9-bbcd-96d565970a17 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:08:54.202273 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed613fa1-c23b-4dd9-bbcd-96d565970a17 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:08:54.202590 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed613fa1-c23b-4dd9-bbcd-96d565970a17 None admin] PATH_INFO: `/v3/services` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:08:54.203293 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed613fa1-c23b-4dd9-bbcd-96d565970a17 None admin] RBAC: Adding query filter params (name=compute) {{(pid=81400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 03:08:54.203595 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed613fa1-c23b-4dd9-bbcd-96d565970a17 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:08:54.205024 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed613fa1-c23b-4dd9-bbcd-96d565970a17 None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:08:54.210022 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 479/952] 10.222.0.18 () {56 vars in 1112 bytes} [Tue Sep 13 03:08:54 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 03:08:54.215551 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed613fa1-c23b-4dd9-bbcd-96d565970a17 None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:08:54.292385 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7570ff2-9718-4d76-8c20-bb528cb2ba72 None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:08:54.322642 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed613fa1-c23b-4dd9-bbcd-96d565970a17 None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:08:54.335541 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1592a87b-8023-40fe-83ee-c6f96a1e0f1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:08:54.336464 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1592a87b-8023-40fe-83ee-c6f96a1e0f1c None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:08:54.336881 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1592a87b-8023-40fe-83ee-c6f96a1e0f1c None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:08:54.337413 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1592a87b-8023-40fe-83ee-c6f96a1e0f1c None admin] PATH_INFO: `/v3/services` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:08:54.338173 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1592a87b-8023-40fe-83ee-c6f96a1e0f1c None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 03:08:54.338630 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1592a87b-8023-40fe-83ee-c6f96a1e0f1c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:08:54.340415 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1592a87b-8023-40fe-83ee-c6f96a1e0f1c None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:08:54.351614 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 474/953] 10.222.0.18 () {56 vars in 1122 bytes} [Tue Sep 13 03:08:54 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 03:08:54.353620 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1592a87b-8023-40fe-83ee-c6f96a1e0f1c None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:08:54.361117 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32aadf34-cd33-45e6-8410-fa7f26ce1978 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:08:54.361768 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32aadf34-cd33-45e6-8410-fa7f26ce1978 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:08:54.362043 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32aadf34-cd33-45e6-8410-fa7f26ce1978 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:08:54.362364 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32aadf34-cd33-45e6-8410-fa7f26ce1978 None admin] PATH_INFO: `/v3/services` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:08:54.363106 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32aadf34-cd33-45e6-8410-fa7f26ce1978 None admin] RBAC: Adding query filter params (type=compute) {{(pid=81400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 03:08:54.363406 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32aadf34-cd33-45e6-8410-fa7f26ce1978 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:08:54.365030 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32aadf34-cd33-45e6-8410-fa7f26ce1978 None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:08:54.374740 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 480/954] 10.222.0.18 () {56 vars in 1112 bytes} [Tue Sep 13 03:08:54 2022] GET /identity/v3/services?type=compute => generated 124 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 03:08:54.394201 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:08:54.476368 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1592a87b-8023-40fe-83ee-c6f96a1e0f1c None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:08:54.512378 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c698863-e44c-48cd-a031-2bab9fd494e5 None None] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:08:54.512710 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c698863-e44c-48cd-a031-2bab9fd494e5 None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:08:54.512931 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c698863-e44c-48cd-a031-2bab9fd494e5 None None] PATH_INFO: `/` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:08:54.516417 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 481/955] 10.222.0.18 () {54 vars in 927 bytes} [Tue Sep 13 03:08:54 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 13 03:08:54.520193 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b66291fa-b0b8-441d-8196-5522d700086e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:08:54.520795 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34fe6025-4beb-48ad-a056-e58fd5fe3f5c None None] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:08:54.521083 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34fe6025-4beb-48ad-a056-e58fd5fe3f5c None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:08:54.521359 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34fe6025-4beb-48ad-a056-e58fd5fe3f5c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:08:54.521864 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b66291fa-b0b8-441d-8196-5522d700086e None admin] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:08:54.522349 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b66291fa-b0b8-441d-8196-5522d700086e None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:08:54.522857 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b66291fa-b0b8-441d-8196-5522d700086e None admin] PATH_INFO: `/v3/endpoints` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:08:54.523726 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b66291fa-b0b8-441d-8196-5522d700086e None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:08:54.525448 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b66291fa-b0b8-441d-8196-5522d700086e None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:08:54.540273 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.auth.core [None req-34fe6025-4beb-48ad-a056-e58fd5fe3f5c None None] MFA Rules not processed for user `49f280ab086b4610855c4ffa378517d9`. Rule list: `[]` (Enabled: `True`). {{(pid=81400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 03:08:54.541601 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-34fe6025-4beb-48ad-a056-e58fd5fe3f5c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:08:54.553510 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 475/956] 10.222.0.18 () {60 vars in 1144 bytes} [Tue Sep 13 03:08:54 2022] POST /identity/v3/endpoints => generated 349 bytes in 201 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 03:08:54.595264 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:08:54.638774 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 482/957] 10.222.0.18 () {58 vars in 1024 bytes} [Tue Sep 13 03:08:54 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 13 03:08:54.908266 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b66291fa-b0b8-441d-8196-5522d700086e None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:08:54.997487 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:08:55.012970 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b66291fa-b0b8-441d-8196-5522d700086e None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:08:55.048777 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7b27719-53c0-4729-abc3-7d7e61fff281 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:08:55.049437 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b27719-53c0-4729-abc3-7d7e61fff281 None admin] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:08:55.049725 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b27719-53c0-4729-abc3-7d7e61fff281 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:08:55.049997 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b27719-53c0-4729-abc3-7d7e61fff281 None admin] PATH_INFO: `/v3/projects` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:08:55.050679 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7b27719-53c0-4729-abc3-7d7e61fff281 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:08:55.052119 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7b27719-53c0-4729-abc3-7d7e61fff281 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:08:55.055049 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: WARNING keystone.server.flask.common [None req-f7b27719-53c0-4729-abc3-7d7e61fff281 None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Sep 13 03:08:55.075053 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.sql.core [None req-f7b27719-53c0-4729-abc3-7d7e61fff281 None admin] Conflict project: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Sep 13 03:08:55.075053 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Sep 13 03:08:55.075053 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [parameters: {'id': '0de96c2a900247c2a9ec7939c0e217ed', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Sep 13 03:08:55.075053 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=81399) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Sep 13 03:08:55.082237 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application [None req-f7b27719-53c0-4729-abc3-7d7e61fff281 None admin] Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 0de96c2a900247c2a9ec7939c0e217ed.: keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 0de96c2a900247c2a9ec7939c0e217ed. Sep 13 03:08:55.082237 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 03:08:55.082237 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Sep 13 03:08:55.082237 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application self.dialect.do_execute( Sep 13 03:08:55.082237 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Sep 13 03:08:55.082237 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Sep 13 03:08:55.082237 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Sep 13 03:08:55.082237 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application result = self._query(query) Sep 13 03:08:55.082237 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Sep 13 03:08:55.082237 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application conn.query(q) Sep 13 03:08:55.082237 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Sep 13 03:08:55.082237 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Sep 13 03:08:55.082237 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Sep 13 03:08:55.083085 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application result.read() Sep 13 03:08:55.083085 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Sep 13 03:08:55.083085 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Sep 13 03:08:55.083085 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Sep 13 03:08:55.083085 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application packet.raise_for_error() Sep 13 03:08:55.083085 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Sep 13 03:08:55.083085 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Sep 13 03:08:55.083085 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Sep 13 03:08:55.083085 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Sep 13 03:08:55.083085 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Sep 13 03:08:55.083085 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Sep 13 03:08:55.083085 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Sep 13 03:08:55.083085 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Sep 13 03:08:55.083085 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 03:08:55.083085 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Sep 13 03:08:55.083085 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return method(*args, **kwargs) Sep 13 03:08:55.083085 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 241, in create_project Sep 13 03:08:55.083085 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return project_ref.to_dict() Sep 13 03:08:55.084058 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Sep 13 03:08:55.084058 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application next(self.gen) Sep 13 03:08:55.084058 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Sep 13 03:08:55.084058 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application yield resource Sep 13 03:08:55.084058 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Sep 13 03:08:55.084058 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application next(self.gen) Sep 13 03:08:55.084058 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Sep 13 03:08:55.084058 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application self.session.rollback() Sep 13 03:08:55.084058 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Sep 13 03:08:55.084058 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application self.force_reraise() Sep 13 03:08:55.084058 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Sep 13 03:08:55.084058 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application raise self.value Sep 13 03:08:55.084058 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Sep 13 03:08:55.084058 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Sep 13 03:08:55.084058 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Sep 13 03:08:55.084058 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application session.commit() Sep 13 03:08:55.084058 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Sep 13 03:08:55.085097 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Sep 13 03:08:55.085097 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Sep 13 03:08:55.085097 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application self._prepare_impl() Sep 13 03:08:55.085097 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Sep 13 03:08:55.085097 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application self.session.flush() Sep 13 03:08:55.085097 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Sep 13 03:08:55.085097 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application self._flush(objects) Sep 13 03:08:55.085097 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Sep 13 03:08:55.085097 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Sep 13 03:08:55.085097 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Sep 13 03:08:55.085097 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application compat.raise_( Sep 13 03:08:55.085097 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Sep 13 03:08:55.085097 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application raise exception Sep 13 03:08:55.085097 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Sep 13 03:08:55.085097 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application flush_context.execute() Sep 13 03:08:55.085097 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Sep 13 03:08:55.085097 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application rec.execute(self) Sep 13 03:08:55.086041 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Sep 13 03:08:55.086041 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Sep 13 03:08:55.086041 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Sep 13 03:08:55.086041 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application _emit_insert_statements( Sep 13 03:08:55.086041 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Sep 13 03:08:55.086041 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application c = connection._execute_20( Sep 13 03:08:55.086041 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Sep 13 03:08:55.086041 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Sep 13 03:08:55.086041 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Sep 13 03:08:55.086041 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Sep 13 03:08:55.086041 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Sep 13 03:08:55.086041 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application ret = self._execute_context( Sep 13 03:08:55.086041 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Sep 13 03:08:55.086041 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Sep 13 03:08:55.086041 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Sep 13 03:08:55.086889 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Sep 13 03:08:55.086889 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Sep 13 03:08:55.086889 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application raise exception Sep 13 03:08:55.086889 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Sep 13 03:08:55.086889 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application self.dialect.do_execute( Sep 13 03:08:55.086889 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Sep 13 03:08:55.086889 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Sep 13 03:08:55.086889 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Sep 13 03:08:55.086889 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application result = self._query(query) Sep 13 03:08:55.086889 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Sep 13 03:08:55.086889 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application conn.query(q) Sep 13 03:08:55.086889 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Sep 13 03:08:55.086889 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Sep 13 03:08:55.086889 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Sep 13 03:08:55.086889 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application result.read() Sep 13 03:08:55.086889 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Sep 13 03:08:55.086889 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Sep 13 03:08:55.089494 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Sep 13 03:08:55.089494 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application packet.raise_for_error() Sep 13 03:08:55.089494 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Sep 13 03:08:55.089494 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Sep 13 03:08:55.089494 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Sep 13 03:08:55.089494 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Sep 13 03:08:55.089494 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Sep 13 03:08:55.089494 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Sep 13 03:08:55.089494 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application [parameters: {'id': '0de96c2a900247c2a9ec7939c0e217ed', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Sep 13 03:08:55.089494 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Sep 13 03:08:55.089494 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Sep 13 03:08:55.089494 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 13 03:08:55.089494 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Sep 13 03:08:55.089494 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 03:08:55.089494 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 216, in create_project Sep 13 03:08:55.095186 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application ret = self.driver.create_project(project_id, project) Sep 13 03:08:55.095186 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Sep 13 03:08:55.095186 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Sep 13 03:08:55.095186 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - Duplicate entry found with name service at domain ID default. Sep 13 03:08:55.095186 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Sep 13 03:08:55.095186 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 13 03:08:55.095186 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Sep 13 03:08:55.095186 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 03:08:55.095186 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 03:08:55.095186 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 03:08:55.095186 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 03:08:55.095186 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 03:08:55.095186 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 03:08:55.095186 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 03:08:55.095186 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 03:08:55.095186 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 03:08:55.095186 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 03:08:55.096152 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 03:08:55.096152 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 176, in post Sep 13 03:08:55.096152 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application ref = PROVIDERS.resource_api.create_project( Sep 13 03:08:55.096152 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 03:08:55.096152 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 03:08:55.096152 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 218, in create_project Sep 13 03:08:55.096152 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application raise exception.Conflict( Sep 13 03:08:55.096152 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 0de96c2a900247c2a9ec7939c0e217ed. Sep 13 03:08:55.096152 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Sep 13 03:08:55.096152 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 476/958] 10.222.0.18 () {60 vars in 1140 bytes} [Tue Sep 13 03:08:54 2022] POST /identity/v3/projects => generated 262 bytes in 176 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Sep 13 03:08:55.096737 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34fe6025-4beb-48ad-a056-e58fd5fe3f5c None None] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:08:55.206404 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34fe6025-4beb-48ad-a056-e58fd5fe3f5c None None] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:08:55.244151 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3d27699-2058-43d1-aba6-d087ccb5318d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:08:55.245025 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3d27699-2058-43d1-aba6-d087ccb5318d None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:08:55.245454 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3d27699-2058-43d1-aba6-d087ccb5318d None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:08:55.245873 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3d27699-2058-43d1-aba6-d087ccb5318d None admin] PATH_INFO: `/v3/projects/service` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:08:55.253225 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3d27699-2058-43d1-aba6-d087ccb5318d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:08:55.254858 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3d27699-2058-43d1-aba6-d087ccb5318d None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:08:55.267123 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application [None req-c3d27699-2058-43d1-aba6-d087ccb5318d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 13 03:08:55.267123 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 03:08:55.267123 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 03:08:55.267123 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 03:08:55.267123 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 03:08:55.267123 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 03:08:55.267123 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 03:08:55.267123 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 03:08:55.267123 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 03:08:55.267123 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 03:08:55.267123 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 03:08:55.267123 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 03:08:55.267123 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 13 03:08:55.267123 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 13 03:08:55.267123 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 13 03:08:55.268238 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 13 03:08:55.268238 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 03:08:55.268238 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 03:08:55.268238 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 03:08:55.268238 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 03:08:55.268238 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 03:08:55.268238 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 03:08:55.268238 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 03:08:55.268238 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application with Lock( Sep 13 03:08:55.268238 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 03:08:55.268238 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._enter() Sep 13 03:08:55.268238 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 03:08:55.268238 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 03:08:55.268238 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 03:08:55.268238 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.creator() Sep 13 03:08:55.268238 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 03:08:55.268238 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application created_value = creator( Sep 13 03:08:55.269477 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 13 03:08:55.269477 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 13 03:08:55.269477 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 13 03:08:55.269477 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 13 03:08:55.269477 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 13 03:08:55.269477 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 13 03:08:55.269477 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 13 03:08:55.269477 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Sep 13 03:08:55.276027 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 483/959] 10.222.0.18 () {56 vars in 1111 bytes} [Tue Sep 13 03:08:55 2022] GET /identity/v3/projects/service => generated 88 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 03:08:55.277904 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7b27719-53c0-4729-abc3-7d7e61fff281 None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:08:55.385519 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7b27719-53c0-4729-abc3-7d7e61fff281 None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:08:55.420624 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ef97ae2-516a-491a-9fbe-0905947f9b05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:08:55.421434 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef97ae2-516a-491a-9fbe-0905947f9b05 None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:08:55.421862 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef97ae2-516a-491a-9fbe-0905947f9b05 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:08:55.422321 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef97ae2-516a-491a-9fbe-0905947f9b05 None admin] PATH_INFO: `/v3/projects` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:08:55.423190 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ef97ae2-516a-491a-9fbe-0905947f9b05 None admin] RBAC: Adding query filter params (name=service) {{(pid=81399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 03:08:55.423656 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ef97ae2-516a-491a-9fbe-0905947f9b05 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:08:55.425428 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ef97ae2-516a-491a-9fbe-0905947f9b05 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:08:55.446102 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 477/960] 10.222.0.18 () {56 vars in 1112 bytes} [Tue Sep 13 03:08:55 2022] GET /identity/v3/projects?name=service => generated 411 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 03:08:56.176323 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b5fd2e-3793-4e83-b671-412c7060b73a None None] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:08:56.176646 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b5fd2e-3793-4e83-b671-412c7060b73a None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:08:56.176863 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b5fd2e-3793-4e83-b671-412c7060b73a None None] PATH_INFO: `/` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:08:56.178465 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 484/961] 10.222.0.18 () {54 vars in 927 bytes} [Tue Sep 13 03:08:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 03:08:56.184097 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c45524-077f-4f23-a5d4-2bc1c56b3ed7 None None] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:08:56.184530 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c45524-077f-4f23-a5d4-2bc1c56b3ed7 None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:08:56.184702 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c45524-077f-4f23-a5d4-2bc1c56b3ed7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:08:56.191672 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:08:56.205547 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.auth.core [None req-60c45524-077f-4f23-a5d4-2bc1c56b3ed7 None None] MFA Rules not processed for user `49f280ab086b4610855c4ffa378517d9`. Rule list: `[]` (Enabled: `True`). {{(pid=81399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 03:08:56.207371 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-60c45524-077f-4f23-a5d4-2bc1c56b3ed7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:08:56.280803 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 478/962] 10.222.0.18 () {58 vars in 1024 bytes} [Tue Sep 13 03:08:56 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 13 03:08:56.550879 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81b5fd2e-3793-4e83-b671-412c7060b73a None None] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:08:56.585481 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:08:56.671156 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab4a2250-7e62-4140-8812-26f124934ae5 None None] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:08:56.671814 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab4a2250-7e62-4140-8812-26f124934ae5 None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:08:56.671924 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81b5fd2e-3793-4e83-b671-412c7060b73a None None] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:08:56.672392 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab4a2250-7e62-4140-8812-26f124934ae5 None None] PATH_INFO: `/` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:08:56.674189 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 479/963] 10.222.0.18 () {54 vars in 927 bytes} [Tue Sep 13 03:08:56 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 03:08:56.679732 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53f39c05-b533-4c1b-80f6-7325631ed911 None None] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:08:56.680281 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53f39c05-b533-4c1b-80f6-7325631ed911 None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:08:56.680773 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53f39c05-b533-4c1b-80f6-7325631ed911 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:08:56.704710 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.auth.core [None req-53f39c05-b533-4c1b-80f6-7325631ed911 None None] MFA Rules not processed for user `49f280ab086b4610855c4ffa378517d9`. Rule list: `[]` (Enabled: `True`). {{(pid=81399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 03:08:56.706175 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-53f39c05-b533-4c1b-80f6-7325631ed911 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:08:56.712995 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d22bdcfc-390c-4102-aa7c-859ae1164991 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:08:56.713631 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d22bdcfc-390c-4102-aa7c-859ae1164991 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:08:56.713909 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d22bdcfc-390c-4102-aa7c-859ae1164991 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:08:56.714640 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d22bdcfc-390c-4102-aa7c-859ae1164991 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:08:56.729689 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d22bdcfc-390c-4102-aa7c-859ae1164991 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 03:08:56.737240 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application [None req-d22bdcfc-390c-4102-aa7c-859ae1164991 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 13 03:08:56.737240 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 03:08:56.737240 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 03:08:56.737240 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 03:08:56.737240 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 03:08:56.737240 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 03:08:56.737240 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 03:08:56.737240 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 03:08:56.737240 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 03:08:56.737240 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 03:08:56.737240 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 03:08:56.737240 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 03:08:56.737240 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 03:08:56.737240 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 03:08:56.737240 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 03:08:56.737240 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 03:08:56.738177 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 03:08:56.738177 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 03:08:56.738177 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 03:08:56.738177 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 03:08:56.738177 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 03:08:56.738177 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 03:08:56.738177 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 03:08:56.738177 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 03:08:56.738177 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 03:08:56.738177 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 03:08:56.738177 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 03:08:56.738177 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application with Lock( Sep 13 03:08:56.738177 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 03:08:56.738177 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._enter() Sep 13 03:08:56.738177 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 03:08:56.738177 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 03:08:56.739068 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 03:08:56.739068 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.creator() Sep 13 03:08:56.739068 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 03:08:56.739068 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application created_value = creator( Sep 13 03:08:56.739068 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 03:08:56.739068 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 03:08:56.739068 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 03:08:56.739068 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 03:08:56.739068 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 03:08:56.739068 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 03:08:56.739068 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 13 03:08:56.739068 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Sep 13 03:08:56.739742 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 485/964] 10.222.0.18 () {56 vars in 1120 bytes} [Tue Sep 13 03:08:56 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 03:08:56.747936 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d22bdcfc-390c-4102-aa7c-859ae1164991 None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:08:56.784880 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 480/965] 10.222.0.18 () {58 vars in 1024 bytes} [Tue Sep 13 03:08:56 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 13 03:08:56.786121 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ba0782-1fea-4785-9c89-a3daece47121 None None] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:08:56.786401 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ba0782-1fea-4785-9c89-a3daece47121 None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:08:56.786703 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ba0782-1fea-4785-9c89-a3daece47121 None None] PATH_INFO: `/` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:08:56.788069 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 481/966] 10.222.0.18 () {54 vars in 927 bytes} [Tue Sep 13 03:08:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 03:08:56.793876 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-021c94a2-4fd0-4efc-a59d-5c40fad7ef70 None None] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:08:56.794157 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-021c94a2-4fd0-4efc-a59d-5c40fad7ef70 None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:08:56.794429 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-021c94a2-4fd0-4efc-a59d-5c40fad7ef70 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:08:56.812583 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.auth.core [None req-021c94a2-4fd0-4efc-a59d-5c40fad7ef70 None None] MFA Rules not processed for user `49f280ab086b4610855c4ffa378517d9`. Rule list: `[]` (Enabled: `True`). {{(pid=81399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 03:08:56.813674 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-021c94a2-4fd0-4efc-a59d-5c40fad7ef70 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:08:56.862103 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:08:56.874970 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d22bdcfc-390c-4102-aa7c-859ae1164991 None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:08:56.881581 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:08:56.898760 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 482/967] 10.222.0.18 () {58 vars in 1024 bytes} [Tue Sep 13 03:08:56 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 13 03:08:56.921770 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2000d910-8b50-4fb3-9f87-041665582ffe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:08:56.922434 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2000d910-8b50-4fb3-9f87-041665582ffe None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:08:56.922742 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2000d910-8b50-4fb3-9f87-041665582ffe None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:08:56.923064 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2000d910-8b50-4fb3-9f87-041665582ffe None admin] PATH_INFO: `/v3/roles` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:08:56.923723 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2000d910-8b50-4fb3-9f87-041665582ffe None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 03:08:56.924076 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2000d910-8b50-4fb3-9f87-041665582ffe None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:08:56.925439 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2000d910-8b50-4fb3-9f87-041665582ffe None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:08:56.934109 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 486/968] 10.222.0.18 () {56 vars in 1115 bytes} [Tue Sep 13 03:08:56 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 03:08:56.942343 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-021c94a2-4fd0-4efc-a59d-5c40fad7ef70 None None] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:08:57.045509 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-021c94a2-4fd0-4efc-a59d-5c40fad7ef70 None None] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:08:57.079941 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2000d910-8b50-4fb3-9f87-041665582ffe None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:08:57.081784 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18f62dc8-4d2e-4d8f-b3ad-9b4483194672 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:08:57.082422 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f62dc8-4d2e-4d8f-b3ad-9b4483194672 None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:08:57.082727 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f62dc8-4d2e-4d8f-b3ad-9b4483194672 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:08:57.083000 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f62dc8-4d2e-4d8f-b3ad-9b4483194672 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:08:57.097110 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18f62dc8-4d2e-4d8f-b3ad-9b4483194672 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:08:57.098802 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18f62dc8-4d2e-4d8f-b3ad-9b4483194672 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:08:57.118692 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application [None req-18f62dc8-4d2e-4d8f-b3ad-9b4483194672 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 03:08:57.118692 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 03:08:57.118692 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 03:08:57.118692 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 03:08:57.118692 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 03:08:57.118692 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 03:08:57.118692 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 03:08:57.118692 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 03:08:57.118692 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 03:08:57.118692 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 03:08:57.118692 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 03:08:57.118692 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 03:08:57.118692 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 13 03:08:57.118692 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 13 03:08:57.118692 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 13 03:08:57.118692 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 13 03:08:57.119754 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 03:08:57.119754 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 03:08:57.119754 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 03:08:57.119754 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 03:08:57.119754 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 03:08:57.119754 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 03:08:57.119754 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 03:08:57.119754 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application with Lock( Sep 13 03:08:57.119754 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 03:08:57.119754 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._enter() Sep 13 03:08:57.119754 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 03:08:57.119754 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 03:08:57.119754 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 03:08:57.119754 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.creator() Sep 13 03:08:57.119754 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 03:08:57.119754 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application created_value = creator( Sep 13 03:08:57.119754 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 13 03:08:57.120672 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 13 03:08:57.120672 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 03:08:57.120672 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Sep 13 03:08:57.124801 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 483/969] 10.222.0.18 () {56 vars in 1108 bytes} [Tue Sep 13 03:08:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 03:08:57.129283 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18f62dc8-4d2e-4d8f-b3ad-9b4483194672 None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:08:57.188165 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:08:57.199242 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2000d910-8b50-4fb3-9f87-041665582ffe None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:08:57.240464 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-443a0682-ff9a-45bb-a3e8-c8a034fd1e3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:08:57.241480 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-443a0682-ff9a-45bb-a3e8-c8a034fd1e3d None admin] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:08:57.241760 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-443a0682-ff9a-45bb-a3e8-c8a034fd1e3d None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:08:57.242312 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-443a0682-ff9a-45bb-a3e8-c8a034fd1e3d None admin] PATH_INFO: `/v3/services` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:08:57.242997 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-443a0682-ff9a-45bb-a3e8-c8a034fd1e3d None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:08:57.243966 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18f62dc8-4d2e-4d8f-b3ad-9b4483194672 None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:08:57.244360 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-443a0682-ff9a-45bb-a3e8-c8a034fd1e3d None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:08:57.261868 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 487/970] 10.222.0.18 () {60 vars in 1141 bytes} [Tue Sep 13 03:08:57 2022] POST /identity/v3/services => generated 242 bytes in 182 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 03:08:57.262778 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-443a0682-ff9a-45bb-a3e8-c8a034fd1e3d None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:08:57.291652 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:08:57.297312 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecc9bd48-7c70-4154-bbe4-c770a882ef53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:08:57.298101 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc9bd48-7c70-4154-bbe4-c770a882ef53 None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:08:57.298373 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc9bd48-7c70-4154-bbe4-c770a882ef53 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:08:57.298718 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc9bd48-7c70-4154-bbe4-c770a882ef53 None admin] PATH_INFO: `/v3/domains` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:08:57.299465 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecc9bd48-7c70-4154-bbe4-c770a882ef53 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 03:08:57.299848 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecc9bd48-7c70-4154-bbe4-c770a882ef53 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:08:57.301415 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecc9bd48-7c70-4154-bbe4-c770a882ef53 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:08:57.316385 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 484/971] 10.222.0.18 () {56 vars in 1109 bytes} [Tue Sep 13 03:08:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 03:08:57.322289 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecc9bd48-7c70-4154-bbe4-c770a882ef53 None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:08:57.416116 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-443a0682-ff9a-45bb-a3e8-c8a034fd1e3d None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:08:57.450781 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:08:57.456669 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecc9bd48-7c70-4154-bbe4-c770a882ef53 None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:08:57.462683 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6b315c5-f0ca-4cef-b0c4-c42d4b666b86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:08:57.463323 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b315c5-f0ca-4cef-b0c4-c42d4b666b86 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:08:57.463653 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b315c5-f0ca-4cef-b0c4-c42d4b666b86 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:08:57.463926 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b315c5-f0ca-4cef-b0c4-c42d4b666b86 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:08:57.464507 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6b315c5-f0ca-4cef-b0c4-c42d4b666b86 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:08:57.466340 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6b315c5-f0ca-4cef-b0c4-c42d4b666b86 None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:08:57.477364 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application [None req-d6b315c5-f0ca-4cef-b0c4-c42d4b666b86 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 13 03:08:57.477364 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 03:08:57.477364 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 13 03:08:57.477364 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 13 03:08:57.477364 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 13 03:08:57.477364 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 13 03:08:57.477364 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 13 03:08:57.477364 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 13 03:08:57.477364 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 13 03:08:57.477364 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Sep 13 03:08:57.477364 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 13 03:08:57.477364 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Sep 13 03:08:57.477364 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 03:08:57.477364 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 03:08:57.477364 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 03:08:57.477364 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 03:08:57.477364 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 03:08:57.478435 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 03:08:57.478435 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 03:08:57.478435 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 03:08:57.478435 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 03:08:57.478435 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 03:08:57.478435 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 03:08:57.478435 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 13 03:08:57.478435 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 13 03:08:57.478435 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 13 03:08:57.478435 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 13 03:08:57.478435 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 03:08:57.478435 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 03:08:57.478435 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 03:08:57.478435 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 03:08:57.478435 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 03:08:57.478435 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 03:08:57.479421 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 03:08:57.479421 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application with Lock( Sep 13 03:08:57.479421 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 03:08:57.479421 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._enter() Sep 13 03:08:57.479421 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 03:08:57.479421 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 03:08:57.479421 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 03:08:57.479421 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.creator() Sep 13 03:08:57.479421 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 03:08:57.479421 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application created_value = creator( Sep 13 03:08:57.479421 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 13 03:08:57.479421 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 13 03:08:57.479421 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 13 03:08:57.479421 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Sep 13 03:08:57.480195 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 488/972] 10.222.0.18 () {56 vars in 1126 bytes} [Tue Sep 13 03:08:57 2022] GET /identity/v3/services/object-store => generated 93 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 03:08:57.485459 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6b315c5-f0ca-4cef-b0c4-c42d4b666b86 None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:08:57.501325 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cde37747-d736-4af5-b799-ac7fe65f7837 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:08:57.501977 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cde37747-d736-4af5-b799-ac7fe65f7837 None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:08:57.502254 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cde37747-d736-4af5-b799-ac7fe65f7837 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:08:57.502526 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cde37747-d736-4af5-b799-ac7fe65f7837 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:08:57.510707 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cde37747-d736-4af5-b799-ac7fe65f7837 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:08:57.512464 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cde37747-d736-4af5-b799-ac7fe65f7837 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:08:57.533692 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application [None req-cde37747-d736-4af5-b799-ac7fe65f7837 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 13 03:08:57.533692 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 03:08:57.533692 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 03:08:57.533692 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 03:08:57.533692 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 03:08:57.533692 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 03:08:57.533692 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 03:08:57.533692 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 03:08:57.533692 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 03:08:57.533692 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 03:08:57.533692 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 03:08:57.533692 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 03:08:57.533692 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 13 03:08:57.533692 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 13 03:08:57.533692 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 13 03:08:57.533692 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 13 03:08:57.534745 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 03:08:57.534745 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 03:08:57.534745 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 13 03:08:57.534745 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 13 03:08:57.534745 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 13 03:08:57.534745 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 13 03:08:57.534745 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 03:08:57.534745 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 03:08:57.534745 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 03:08:57.534745 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 03:08:57.534745 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 03:08:57.534745 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application with Lock( Sep 13 03:08:57.534745 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 03:08:57.534745 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._enter() Sep 13 03:08:57.534745 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 03:08:57.534745 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 03:08:57.534745 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 03:08:57.534745 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.creator() Sep 13 03:08:57.535723 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 03:08:57.535723 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application created_value = creator( Sep 13 03:08:57.535723 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 13 03:08:57.535723 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 13 03:08:57.535723 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 13 03:08:57.535723 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 13 03:08:57.535723 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 13 03:08:57.535723 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 13 03:08:57.535723 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 13 03:08:57.535723 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Sep 13 03:08:57.535723 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 485/973] 10.222.0.18 () {56 vars in 1117 bytes} [Tue Sep 13 03:08:57 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 03:08:57.542580 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cde37747-d736-4af5-b799-ac7fe65f7837 None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:08:57.595915 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:08:57.626437 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6b315c5-f0ca-4cef-b0c4-c42d4b666b86 None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:08:57.668346 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cde37747-d736-4af5-b799-ac7fe65f7837 None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:08:57.670531 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d6458a8-5b3d-4bf2-8a7f-7fa593c59b5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:08:57.671432 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d6458a8-5b3d-4bf2-8a7f-7fa593c59b5b None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:08:57.671705 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d6458a8-5b3d-4bf2-8a7f-7fa593c59b5b None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:08:57.671967 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d6458a8-5b3d-4bf2-8a7f-7fa593c59b5b None admin] PATH_INFO: `/v3/services` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:08:57.673530 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d6458a8-5b3d-4bf2-8a7f-7fa593c59b5b None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 03:08:57.673838 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d6458a8-5b3d-4bf2-8a7f-7fa593c59b5b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:08:57.675290 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d6458a8-5b3d-4bf2-8a7f-7fa593c59b5b None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:08:57.681879 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 489/974] 10.222.0.18 () {56 vars in 1122 bytes} [Tue Sep 13 03:08:57 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 03:08:57.687659 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36ab8214-a87b-4197-925f-c40e87bc55bf None None] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:08:57.687949 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36ab8214-a87b-4197-925f-c40e87bc55bf None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:08:57.688225 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36ab8214-a87b-4197-925f-c40e87bc55bf None None] PATH_INFO: `/` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:08:57.689620 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 490/975] 10.222.0.18 () {54 vars in 927 bytes} [Tue Sep 13 03:08:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 13 03:08:57.691896 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36ab8214-a87b-4197-925f-c40e87bc55bf None None] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:08:57.714460 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fdd28c5-b4b3-45ac-a6dc-363063def064 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:08:57.715122 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fdd28c5-b4b3-45ac-a6dc-363063def064 None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:08:57.715402 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fdd28c5-b4b3-45ac-a6dc-363063def064 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:08:57.715678 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fdd28c5-b4b3-45ac-a6dc-363063def064 None admin] PATH_INFO: `/v3/users` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:08:57.716391 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fdd28c5-b4b3-45ac-a6dc-363063def064 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 03:08:57.716728 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fdd28c5-b4b3-45ac-a6dc-363063def064 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:08:57.718153 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fdd28c5-b4b3-45ac-a6dc-363063def064 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:08:57.738234 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 486/976] 10.222.0.18 () {56 vars in 1149 bytes} [Tue Sep 13 03:08:57 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 03:08:57.739690 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6f28636-6036-4634-a618-63e714a566a4 None None] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:08:57.740077 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6f28636-6036-4634-a618-63e714a566a4 None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:08:57.740452 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6f28636-6036-4634-a618-63e714a566a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:08:57.759431 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.auth.core [None req-b6f28636-6036-4634-a618-63e714a566a4 None None] MFA Rules not processed for user `49f280ab086b4610855c4ffa378517d9`. Rule list: `[]` (Enabled: `True`). {{(pid=81399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 03:08:57.760813 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-b6f28636-6036-4634-a618-63e714a566a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:08:57.774669 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:08:57.808771 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36ab8214-a87b-4197-925f-c40e87bc55bf None None] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:08:57.833405 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 487/977] 10.222.0.18 () {58 vars in 1024 bytes} [Tue Sep 13 03:08:57 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 13 03:08:57.835860 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6f28636-6036-4634-a618-63e714a566a4 None None] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:08:57.852588 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb565702-3ddf-48c6-beed-c00426fcea02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:08:57.853513 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb565702-3ddf-48c6-beed-c00426fcea02 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:08:57.854021 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb565702-3ddf-48c6-beed-c00426fcea02 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:08:57.854520 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb565702-3ddf-48c6-beed-c00426fcea02 None admin] PATH_INFO: `/v3/services` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:08:57.855434 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb565702-3ddf-48c6-beed-c00426fcea02 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 03:08:57.855934 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb565702-3ddf-48c6-beed-c00426fcea02 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:08:57.858150 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb565702-3ddf-48c6-beed-c00426fcea02 None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:08:57.863837 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 491/978] 10.222.0.18 () {56 vars in 1122 bytes} [Tue Sep 13 03:08:57 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 03:08:57.869150 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb565702-3ddf-48c6-beed-c00426fcea02 None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:08:57.881369 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:08:57.956667 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6f28636-6036-4634-a618-63e714a566a4 None None] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:08:57.997269 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-074045d4-c42d-4583-b0df-c784c62a2d22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:08:57.997929 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-074045d4-c42d-4583-b0df-c784c62a2d22 None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:08:57.998206 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-074045d4-c42d-4583-b0df-c784c62a2d22 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:08:57.998473 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-074045d4-c42d-4583-b0df-c784c62a2d22 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:08:58.014896 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb565702-3ddf-48c6-beed-c00426fcea02 None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:08:58.018121 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:08:58.061516 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b96858c4-7dca-4170-89bd-fef63d12b3cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:08:58.062485 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b96858c4-7dca-4170-89bd-fef63d12b3cd None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:08:58.062931 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b96858c4-7dca-4170-89bd-fef63d12b3cd None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:08:58.063378 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b96858c4-7dca-4170-89bd-fef63d12b3cd None admin] PATH_INFO: `/v3/endpoints` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:08:58.064220 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b96858c4-7dca-4170-89bd-fef63d12b3cd None admin] RBAC: Adding query filter params (interface=admin, service_id=a7a456d8462644079858efd21c4a1488, region_id=RegionOne) {{(pid=81400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 03:08:58.064668 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b96858c4-7dca-4170-89bd-fef63d12b3cd None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:08:58.066170 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b96858c4-7dca-4170-89bd-fef63d12b3cd None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:08:58.077362 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 492/979] 10.222.0.18 () {56 vars in 1249 bytes} [Tue Sep 13 03:08:57 2022] GET /identity/v3/endpoints?service_id=a7a456d8462644079858efd21c4a1488&interface=admin®ion_id=RegionOne => generated 193 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 03:08:58.078492 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b96858c4-7dca-4170-89bd-fef63d12b3cd None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:08:58.095517 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-074045d4-c42d-4583-b0df-c784c62a2d22 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:08:58.097052 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-074045d4-c42d-4583-b0df-c784c62a2d22 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:08:58.161238 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:08:58.198898 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b96858c4-7dca-4170-89bd-fef63d12b3cd None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:08:58.199590 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 488/980] 10.222.0.18 () {58 vars in 1282 bytes} [Tue Sep 13 03:08:57 2022] GET /identity/v3/auth/tokens => generated 1570 bytes in 366 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 13 03:08:58.200244 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-074045d4-c42d-4583-b0df-c784c62a2d22 None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:08:58.248364 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68e70036-24ee-4578-bae8-9f5416cfdb45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:08:58.249526 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e70036-24ee-4578-bae8-9f5416cfdb45 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:08:58.249988 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e70036-24ee-4578-bae8-9f5416cfdb45 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:08:58.250348 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:08:58.250565 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e70036-24ee-4578-bae8-9f5416cfdb45 None admin] PATH_INFO: `/v3/services` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:08:58.251677 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68e70036-24ee-4578-bae8-9f5416cfdb45 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:08:58.252994 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68e70036-24ee-4578-bae8-9f5416cfdb45 None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:08:58.260771 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 493/981] 10.222.0.18 () {56 vars in 1087 bytes} [Tue Sep 13 03:08:58 2022] GET /identity/v3/services => generated 1727 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 13 03:08:58.261681 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68e70036-24ee-4578-bae8-9f5416cfdb45 None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:08:58.326878 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-074045d4-c42d-4583-b0df-c784c62a2d22 None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:08:58.373963 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef94d904-8027-4862-9a63-a515ed0a453b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:08:58.374602 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef94d904-8027-4862-9a63-a515ed0a453b None admin] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:08:58.374920 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef94d904-8027-4862-9a63-a515ed0a453b None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:08:58.375188 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef94d904-8027-4862-9a63-a515ed0a453b None admin] PATH_INFO: `/v3/users` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:08:58.376280 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef94d904-8027-4862-9a63-a515ed0a453b None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:08:58.378112 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef94d904-8027-4862-9a63-a515ed0a453b None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:08:58.397125 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.sql.core [None req-ef94d904-8027-4862-9a63-a515ed0a453b None admin] Conflict user: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Sep 13 03:08:58.397125 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Sep 13 03:08:58.397125 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [parameters: {'user_id': 'f1e0b0ccbfe943ab8c30ba509e12c289', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Sep 13 03:08:58.397125 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=81399) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Sep 13 03:08:58.402491 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68e70036-24ee-4578-bae8-9f5416cfdb45 None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:08:58.405113 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application [None req-ef94d904-8027-4862-9a63-a515ed0a453b None admin] Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default.: keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Sep 13 03:08:58.405113 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 03:08:58.405113 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Sep 13 03:08:58.405113 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application self.dialect.do_execute( Sep 13 03:08:58.405113 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Sep 13 03:08:58.405113 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Sep 13 03:08:58.405113 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Sep 13 03:08:58.405113 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application result = self._query(query) Sep 13 03:08:58.405113 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Sep 13 03:08:58.405113 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application conn.query(q) Sep 13 03:08:58.405113 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Sep 13 03:08:58.405113 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Sep 13 03:08:58.405113 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Sep 13 03:08:58.405113 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application result.read() Sep 13 03:08:58.405113 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Sep 13 03:08:58.406172 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Sep 13 03:08:58.406172 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Sep 13 03:08:58.406172 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application packet.raise_for_error() Sep 13 03:08:58.406172 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Sep 13 03:08:58.406172 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Sep 13 03:08:58.406172 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Sep 13 03:08:58.406172 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Sep 13 03:08:58.406172 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Sep 13 03:08:58.406172 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Sep 13 03:08:58.406172 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Sep 13 03:08:58.406172 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Sep 13 03:08:58.406172 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 03:08:58.406172 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Sep 13 03:08:58.406172 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return method(*args, **kwargs) Sep 13 03:08:58.406172 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 141, in create_user Sep 13 03:08:58.406172 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return base.filter_user(user_ref.to_dict()) Sep 13 03:08:58.406172 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Sep 13 03:08:58.406172 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application next(self.gen) Sep 13 03:08:58.407777 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Sep 13 03:08:58.407777 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application yield resource Sep 13 03:08:58.407777 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Sep 13 03:08:58.407777 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application next(self.gen) Sep 13 03:08:58.407777 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Sep 13 03:08:58.407777 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application self.session.rollback() Sep 13 03:08:58.407777 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Sep 13 03:08:58.407777 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application self.force_reraise() Sep 13 03:08:58.407777 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Sep 13 03:08:58.407777 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application raise self.value Sep 13 03:08:58.407777 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Sep 13 03:08:58.407777 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Sep 13 03:08:58.407777 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Sep 13 03:08:58.407777 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application session.commit() Sep 13 03:08:58.407777 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Sep 13 03:08:58.407777 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Sep 13 03:08:58.407777 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Sep 13 03:08:58.408723 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application self._prepare_impl() Sep 13 03:08:58.408723 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Sep 13 03:08:58.408723 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application self.session.flush() Sep 13 03:08:58.408723 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Sep 13 03:08:58.408723 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application self._flush(objects) Sep 13 03:08:58.408723 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Sep 13 03:08:58.408723 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Sep 13 03:08:58.408723 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Sep 13 03:08:58.408723 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application compat.raise_( Sep 13 03:08:58.408723 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Sep 13 03:08:58.408723 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application raise exception Sep 13 03:08:58.408723 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Sep 13 03:08:58.408723 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application flush_context.execute() Sep 13 03:08:58.408723 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Sep 13 03:08:58.408723 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application rec.execute(self) Sep 13 03:08:58.408723 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Sep 13 03:08:58.408723 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Sep 13 03:08:58.409676 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Sep 13 03:08:58.409676 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application _emit_insert_statements( Sep 13 03:08:58.409676 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1238, in _emit_insert_statements Sep 13 03:08:58.409676 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application result = connection._execute_20( Sep 13 03:08:58.409676 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Sep 13 03:08:58.409676 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Sep 13 03:08:58.409676 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Sep 13 03:08:58.409676 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Sep 13 03:08:58.409676 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Sep 13 03:08:58.409676 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application ret = self._execute_context( Sep 13 03:08:58.409676 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Sep 13 03:08:58.409676 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Sep 13 03:08:58.409676 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Sep 13 03:08:58.409676 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Sep 13 03:08:58.409676 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Sep 13 03:08:58.409676 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application raise exception Sep 13 03:08:58.410647 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Sep 13 03:08:58.410647 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application self.dialect.do_execute( Sep 13 03:08:58.410647 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Sep 13 03:08:58.410647 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Sep 13 03:08:58.410647 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Sep 13 03:08:58.410647 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application result = self._query(query) Sep 13 03:08:58.410647 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Sep 13 03:08:58.410647 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application conn.query(q) Sep 13 03:08:58.410647 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Sep 13 03:08:58.410647 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Sep 13 03:08:58.410647 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Sep 13 03:08:58.410647 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application result.read() Sep 13 03:08:58.410647 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Sep 13 03:08:58.410647 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Sep 13 03:08:58.410647 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Sep 13 03:08:58.410647 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application packet.raise_for_error() Sep 13 03:08:58.413164 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Sep 13 03:08:58.413164 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Sep 13 03:08:58.413164 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Sep 13 03:08:58.413164 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Sep 13 03:08:58.413164 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Sep 13 03:08:58.413164 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Sep 13 03:08:58.413164 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application [parameters: {'user_id': 'f1e0b0ccbfe943ab8c30ba509e12c289', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Sep 13 03:08:58.413164 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Sep 13 03:08:58.413164 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Sep 13 03:08:58.413164 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 13 03:08:58.413164 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Sep 13 03:08:58.413164 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 03:08:58.413164 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 03:08:58.413164 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 03:08:58.413164 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 03:08:58.415315 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 03:08:58.415315 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 03:08:58.415315 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 03:08:58.415315 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 03:08:58.415315 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 03:08:58.415315 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 03:08:58.415315 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 03:08:58.415315 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 244, in post Sep 13 03:08:58.415315 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.create_user( Sep 13 03:08:58.415315 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 03:08:58.415315 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 03:08:58.415315 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 13 03:08:58.415315 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 13 03:08:58.415315 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 13 03:08:58.415315 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 13 03:08:58.415315 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1000, in create_user Sep 13 03:08:58.415315 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application ref = self._create_user_with_federated_objects(user, driver) Sep 13 03:08:58.419037 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 966, in _create_user_with_federated_objects Sep 13 03:08:58.419037 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application user = driver.create_user(user['id'], user) Sep 13 03:08:58.419037 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Sep 13 03:08:58.419037 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Sep 13 03:08:58.419037 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Sep 13 03:08:58.419037 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Sep 13 03:08:58.419037 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 489/982] 10.222.0.18 () {60 vars in 1131 bytes} [Tue Sep 13 03:08:58 2022] POST /identity/v3/users => generated 160 bytes in 212 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Sep 13 03:08:58.419037 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef94d904-8027-4862-9a63-a515ed0a453b None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:08:58.462258 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c788e99-5b8a-4998-a7b5-22f6f46e8182 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:08:58.463102 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c788e99-5b8a-4998-a7b5-22f6f46e8182 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:08:58.463380 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c788e99-5b8a-4998-a7b5-22f6f46e8182 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:08:58.463649 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c788e99-5b8a-4998-a7b5-22f6f46e8182 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:08:58.465620 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:08:58.478033 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c788e99-5b8a-4998-a7b5-22f6f46e8182 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:08:58.479511 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c788e99-5b8a-4998-a7b5-22f6f46e8182 None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:08:58.495336 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application [None req-2c788e99-5b8a-4998-a7b5-22f6f46e8182 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 03:08:58.495336 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 03:08:58.495336 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 03:08:58.495336 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 03:08:58.495336 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 03:08:58.495336 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 03:08:58.495336 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 03:08:58.495336 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 03:08:58.495336 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 03:08:58.495336 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 03:08:58.495336 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 03:08:58.495336 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 03:08:58.495336 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 13 03:08:58.495336 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 13 03:08:58.495336 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 13 03:08:58.495336 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 13 03:08:58.496305 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 03:08:58.496305 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 03:08:58.496305 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 03:08:58.496305 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 03:08:58.496305 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 03:08:58.496305 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 03:08:58.496305 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 03:08:58.496305 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application with Lock( Sep 13 03:08:58.496305 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 03:08:58.496305 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._enter() Sep 13 03:08:58.496305 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 03:08:58.496305 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 03:08:58.496305 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 03:08:58.496305 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.creator() Sep 13 03:08:58.496305 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 03:08:58.496305 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application created_value = creator( Sep 13 03:08:58.496305 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 13 03:08:58.497220 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 13 03:08:58.497220 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 03:08:58.497220 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Sep 13 03:08:58.497220 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 494/983] 10.222.0.18 () {56 vars in 1108 bytes} [Tue Sep 13 03:08:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 235 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 03:08:58.503114 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c788e99-5b8a-4998-a7b5-22f6f46e8182 None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:08:58.542538 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef94d904-8027-4862-9a63-a515ed0a453b None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:08:58.555064 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:08:58.583554 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff6e03be-af29-4ac3-a11e-288be08570f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:08:58.584195 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff6e03be-af29-4ac3-a11e-288be08570f2 None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:08:58.584529 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff6e03be-af29-4ac3-a11e-288be08570f2 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:08:58.584802 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff6e03be-af29-4ac3-a11e-288be08570f2 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:08:58.593174 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff6e03be-af29-4ac3-a11e-288be08570f2 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:08:58.594846 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff6e03be-af29-4ac3-a11e-288be08570f2 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:08:58.607283 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application [None req-ff6e03be-af29-4ac3-a11e-288be08570f2 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Sep 13 03:08:58.607283 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 03:08:58.607283 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 03:08:58.607283 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 03:08:58.607283 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 03:08:58.607283 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 03:08:58.607283 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 03:08:58.607283 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 03:08:58.607283 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 03:08:58.607283 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 03:08:58.607283 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 03:08:58.607283 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 03:08:58.607283 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 13 03:08:58.607283 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 13 03:08:58.607283 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 13 03:08:58.607283 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 13 03:08:58.608272 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 03:08:58.608272 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 03:08:58.608272 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 13 03:08:58.608272 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 13 03:08:58.608272 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 13 03:08:58.608272 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 13 03:08:58.608272 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 03:08:58.608272 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 03:08:58.608272 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 03:08:58.608272 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 03:08:58.608272 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 03:08:58.608272 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application with Lock( Sep 13 03:08:58.608272 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 03:08:58.608272 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._enter() Sep 13 03:08:58.608272 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 03:08:58.608272 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 03:08:58.608272 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 03:08:58.608272 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.creator() Sep 13 03:08:58.609236 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 03:08:58.609236 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application created_value = creator( Sep 13 03:08:58.609236 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 13 03:08:58.609236 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 13 03:08:58.609236 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 13 03:08:58.609236 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 13 03:08:58.609236 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 13 03:08:58.609236 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 13 03:08:58.609236 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Sep 13 03:08:58.609236 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Sep 13 03:08:58.609236 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 490/984] 10.222.0.18 () {56 vars in 1099 bytes} [Tue Sep 13 03:08:58 2022] GET /identity/v3/users/cinder => generated 84 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 03:08:58.618840 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff6e03be-af29-4ac3-a11e-288be08570f2 None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:08:58.632827 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c788e99-5b8a-4998-a7b5-22f6f46e8182 None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:08:58.680216 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5316cc15-b52c-469b-8f50-d60960c280f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:08:58.681257 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5316cc15-b52c-469b-8f50-d60960c280f1 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:08:58.682079 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5316cc15-b52c-469b-8f50-d60960c280f1 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:08:58.682447 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5316cc15-b52c-469b-8f50-d60960c280f1 None admin] PATH_INFO: `/v3/domains` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:08:58.683456 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5316cc15-b52c-469b-8f50-d60960c280f1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 03:08:58.684135 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5316cc15-b52c-469b-8f50-d60960c280f1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:08:58.686866 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5316cc15-b52c-469b-8f50-d60960c280f1 None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:08:58.703047 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 495/985] 10.222.0.18 () {56 vars in 1109 bytes} [Tue Sep 13 03:08:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 03:08:58.709616 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5316cc15-b52c-469b-8f50-d60960c280f1 None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:08:58.735790 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff6e03be-af29-4ac3-a11e-288be08570f2 None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:08:58.760261 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:08:58.780587 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bca0566b-2caa-4e86-9458-b646cc1a3211 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:08:58.781340 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca0566b-2caa-4e86-9458-b646cc1a3211 None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:08:58.781720 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca0566b-2caa-4e86-9458-b646cc1a3211 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:08:58.782090 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca0566b-2caa-4e86-9458-b646cc1a3211 None admin] PATH_INFO: `/v3/users` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:08:58.783403 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bca0566b-2caa-4e86-9458-b646cc1a3211 None admin] RBAC: Adding query filter params (name=cinder) {{(pid=81399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 03:08:58.783710 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bca0566b-2caa-4e86-9458-b646cc1a3211 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:08:58.785129 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bca0566b-2caa-4e86-9458-b646cc1a3211 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:08:58.826693 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 491/986] 10.222.0.18 () {56 vars in 1101 bytes} [Tue Sep 13 03:08:58 2022] GET /identity/v3/users?name=cinder => generated 354 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 03:08:58.831795 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5316cc15-b52c-469b-8f50-d60960c280f1 None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:08:58.872263 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff222447-8c1a-456f-8a7c-acf11e386945 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:08:58.873769 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff222447-8c1a-456f-8a7c-acf11e386945 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:08:58.874228 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff222447-8c1a-456f-8a7c-acf11e386945 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:08:58.874717 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff222447-8c1a-456f-8a7c-acf11e386945 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:08:58.882680 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff222447-8c1a-456f-8a7c-acf11e386945 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:08:58.884462 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff222447-8c1a-456f-8a7c-acf11e386945 None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:08:58.898917 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application [None req-ff222447-8c1a-456f-8a7c-acf11e386945 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 13 03:08:58.898917 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 03:08:58.898917 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 03:08:58.898917 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 03:08:58.898917 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 03:08:58.898917 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 03:08:58.898917 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 03:08:58.898917 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 03:08:58.898917 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 03:08:58.898917 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 03:08:58.898917 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 03:08:58.898917 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 03:08:58.898917 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 13 03:08:58.898917 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 13 03:08:58.898917 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 13 03:08:58.899929 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 13 03:08:58.899929 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 03:08:58.899929 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 03:08:58.899929 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 03:08:58.899929 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 03:08:58.899929 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 03:08:58.899929 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 03:08:58.899929 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 03:08:58.899929 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application with Lock( Sep 13 03:08:58.899929 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 03:08:58.899929 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._enter() Sep 13 03:08:58.899929 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 03:08:58.899929 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 03:08:58.899929 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 03:08:58.899929 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.creator() Sep 13 03:08:58.899929 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 03:08:58.899929 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application created_value = creator( Sep 13 03:08:58.900868 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 13 03:08:58.900868 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 13 03:08:58.900868 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 13 03:08:58.900868 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 13 03:08:58.900868 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 13 03:08:58.900868 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 13 03:08:58.900868 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 13 03:08:58.900868 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Sep 13 03:08:58.903120 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 496/987] 10.222.0.18 () {56 vars in 1111 bytes} [Tue Sep 13 03:08:58 2022] GET /identity/v3/projects/service => generated 88 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 03:08:58.909395 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bca0566b-2caa-4e86-9458-b646cc1a3211 None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:08:58.942065 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:08:59.026982 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bca0566b-2caa-4e86-9458-b646cc1a3211 None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:08:59.063414 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dd3f39c-ce23-442b-bba6-d12843d3f40f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:08:59.064231 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd3f39c-ce23-442b-bba6-d12843d3f40f None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:08:59.064599 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd3f39c-ce23-442b-bba6-d12843d3f40f None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:08:59.064965 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd3f39c-ce23-442b-bba6-d12843d3f40f None admin] PATH_INFO: `/v3/projects` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:08:59.066167 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dd3f39c-ce23-442b-bba6-d12843d3f40f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 03:08:59.066564 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dd3f39c-ce23-442b-bba6-d12843d3f40f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:08:59.068128 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dd3f39c-ce23-442b-bba6-d12843d3f40f None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:08:59.083004 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 492/988] 10.222.0.18 () {56 vars in 1148 bytes} [Tue Sep 13 03:08:58 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 03:08:59.088931 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff222447-8c1a-456f-8a7c-acf11e386945 None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:08:59.197423 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff222447-8c1a-456f-8a7c-acf11e386945 None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:08:59.204685 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:08:59.232754 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2973538e-3d57-4a3b-89fd-7445eb81edb7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:08:59.233352 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2973538e-3d57-4a3b-89fd-7445eb81edb7 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:08:59.233628 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2973538e-3d57-4a3b-89fd-7445eb81edb7 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:08:59.233897 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2973538e-3d57-4a3b-89fd-7445eb81edb7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:08:59.234637 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2973538e-3d57-4a3b-89fd-7445eb81edb7 None admin] RBAC: Adding query filter params (role.id=57ec3ac460b749a5910aea16ad1d643e, scope.project.id=b9d94f6a68f7470d9494f112175620dc, user.id=4c2d20fb54a54ecc99b47f237be4917c) {{(pid=81400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 03:08:59.234939 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2973538e-3d57-4a3b-89fd-7445eb81edb7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:08:59.236361 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2973538e-3d57-4a3b-89fd-7445eb81edb7 None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:08:59.250693 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 497/989] 10.222.0.18 () {56 vars in 1374 bytes} [Tue Sep 13 03:08:59 2022] GET /identity/v3/role_assignments?user.id=4c2d20fb54a54ecc99b47f237be4917c&scope.project.id=b9d94f6a68f7470d9494f112175620dc&role.id=57ec3ac460b749a5910aea16ad1d643e => generated 608 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 03:08:59.521384 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-747bbb52-100c-40d7-9ece-b6f1b4ec6510 None None] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:08:59.521884 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-747bbb52-100c-40d7-9ece-b6f1b4ec6510 None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:08:59.521884 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-747bbb52-100c-40d7-9ece-b6f1b4ec6510 None None] PATH_INFO: `/` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:08:59.523483 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 493/990] 10.222.0.18 () {54 vars in 927 bytes} [Tue Sep 13 03:08:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 03:08:59.528481 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d89df6f-f6c7-4fe7-99d3-e8eec3db2eba None None] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:08:59.528766 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d89df6f-f6c7-4fe7-99d3-e8eec3db2eba None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:08:59.529527 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d89df6f-f6c7-4fe7-99d3-e8eec3db2eba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:08:59.548325 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.auth.core [None req-4d89df6f-f6c7-4fe7-99d3-e8eec3db2eba None None] MFA Rules not processed for user `49f280ab086b4610855c4ffa378517d9`. Rule list: `[]` (Enabled: `True`). {{(pid=81400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 03:08:59.549515 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-4d89df6f-f6c7-4fe7-99d3-e8eec3db2eba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:08:59.618310 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 498/991] 10.222.0.18 () {58 vars in 1024 bytes} [Tue Sep 13 03:08:59 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 13 03:08:59.890342 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-747bbb52-100c-40d7-9ece-b6f1b4ec6510 None None] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:08:59.981806 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:08:59.995711 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-747bbb52-100c-40d7-9ece-b6f1b4ec6510 None None] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:09:00.028956 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d0722e1-9886-409d-aa1f-712bb156f487 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:09:00.030087 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d0722e1-9886-409d-aa1f-712bb156f487 None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:09:00.030366 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d0722e1-9886-409d-aa1f-712bb156f487 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:09:00.030656 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d0722e1-9886-409d-aa1f-712bb156f487 None admin] PATH_INFO: `/v3/services/compute` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:09:00.031234 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d0722e1-9886-409d-aa1f-712bb156f487 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:09:00.032587 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d0722e1-9886-409d-aa1f-712bb156f487 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:09:00.041349 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application [None req-6d0722e1-9886-409d-aa1f-712bb156f487 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Sep 13 03:09:00.041349 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 03:09:00.041349 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 13 03:09:00.041349 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 13 03:09:00.041349 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 13 03:09:00.041349 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 13 03:09:00.041349 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 13 03:09:00.041349 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 13 03:09:00.041349 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Sep 13 03:09:00.041349 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Sep 13 03:09:00.041349 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 13 03:09:00.041349 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Sep 13 03:09:00.041349 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 03:09:00.041349 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 03:09:00.041349 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 03:09:00.041349 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 03:09:00.041349 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 03:09:00.042348 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 03:09:00.042348 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 03:09:00.042348 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 03:09:00.042348 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 03:09:00.042348 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 03:09:00.042348 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 03:09:00.042348 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 13 03:09:00.042348 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 13 03:09:00.042348 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 13 03:09:00.042348 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 13 03:09:00.042348 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 03:09:00.042348 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 03:09:00.042348 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 03:09:00.042348 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 03:09:00.042348 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 03:09:00.042348 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 03:09:00.043244 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 03:09:00.043244 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application with Lock( Sep 13 03:09:00.043244 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 03:09:00.043244 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._enter() Sep 13 03:09:00.043244 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 03:09:00.043244 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 03:09:00.043244 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 03:09:00.043244 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.creator() Sep 13 03:09:00.043244 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 03:09:00.043244 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application created_value = creator( Sep 13 03:09:00.043244 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 13 03:09:00.043244 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 13 03:09:00.043244 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Sep 13 03:09:00.043244 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Sep 13 03:09:00.043244 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 494/992] 10.222.0.18 () {56 vars in 1111 bytes} [Tue Sep 13 03:08:59 2022] GET /identity/v3/services/compute => generated 88 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 03:09:00.050064 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d89df6f-f6c7-4fe7-99d3-e8eec3db2eba None None] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:09:00.153173 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:09:00.157176 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d89df6f-f6c7-4fe7-99d3-e8eec3db2eba None None] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:09:00.192976 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-779fd538-7f61-4b36-b91a-468804a63a33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:09:00.193583 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-779fd538-7f61-4b36-b91a-468804a63a33 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:09:00.193858 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-779fd538-7f61-4b36-b91a-468804a63a33 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:09:00.194127 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-779fd538-7f61-4b36-b91a-468804a63a33 None admin] PATH_INFO: `/v3/services` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:09:00.194778 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-779fd538-7f61-4b36-b91a-468804a63a33 None admin] RBAC: Adding query filter params (name=compute) {{(pid=81400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 03:09:00.195080 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-779fd538-7f61-4b36-b91a-468804a63a33 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:09:00.196439 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-779fd538-7f61-4b36-b91a-468804a63a33 None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:09:00.202045 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 499/993] 10.222.0.18 () {56 vars in 1112 bytes} [Tue Sep 13 03:09:00 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 03:09:00.207431 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d0722e1-9886-409d-aa1f-712bb156f487 None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:09:00.308580 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d0722e1-9886-409d-aa1f-712bb156f487 None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:09:00.342772 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd0f9011-491e-4267-94f9-323f1d162b9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:09:00.343668 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd0f9011-491e-4267-94f9-323f1d162b9c None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:09:00.344148 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd0f9011-491e-4267-94f9-323f1d162b9c None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:09:00.344637 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd0f9011-491e-4267-94f9-323f1d162b9c None admin] PATH_INFO: `/v3/services` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:09:00.345450 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd0f9011-491e-4267-94f9-323f1d162b9c None admin] RBAC: Adding query filter params (type=compute) {{(pid=81399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 03:09:00.345950 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd0f9011-491e-4267-94f9-323f1d162b9c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:09:00.347516 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd0f9011-491e-4267-94f9-323f1d162b9c None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:09:00.352936 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 495/994] 10.222.0.18 () {56 vars in 1112 bytes} [Tue Sep 13 03:09:00 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 03:09:00.358461 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-779fd538-7f61-4b36-b91a-468804a63a33 None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:09:00.464700 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-779fd538-7f61-4b36-b91a-468804a63a33 None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:09:00.500607 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dd53fdb-878f-42bf-bf0f-a77b60ce0b71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:09:00.501258 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd53fdb-878f-42bf-bf0f-a77b60ce0b71 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:09:00.501535 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd53fdb-878f-42bf-bf0f-a77b60ce0b71 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:09:00.501804 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd53fdb-878f-42bf-bf0f-a77b60ce0b71 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:09:00.502492 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dd53fdb-878f-42bf-bf0f-a77b60ce0b71 None admin] RBAC: Adding query filter params (interface=public, service_id=69e9364d6ff74d7d8a317016fb9a9851, region_id=RegionOne) {{(pid=81400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 03:09:00.502823 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dd53fdb-878f-42bf-bf0f-a77b60ce0b71 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:09:00.504181 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dd53fdb-878f-42bf-bf0f-a77b60ce0b71 None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:09:00.510673 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 500/995] 10.222.0.18 () {56 vars in 1251 bytes} [Tue Sep 13 03:09:00 2022] GET /identity/v3/endpoints?service_id=69e9364d6ff74d7d8a317016fb9a9851&interface=public®ion_id=RegionOne => generated 194 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 03:09:00.515783 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd0f9011-491e-4267-94f9-323f1d162b9c None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:09:00.534446 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:09:00.562138 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b2dccf-2050-4ac4-8f69-2a04f941662f None None] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:09:00.562422 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b2dccf-2050-4ac4-8f69-2a04f941662f None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:09:00.562702 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b2dccf-2050-4ac4-8f69-2a04f941662f None None] PATH_INFO: `/` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:09:00.564159 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 501/996] 10.222.0.18 () {54 vars in 927 bytes} [Tue Sep 13 03:09:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 03:09:00.568815 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3bd9a92-79a4-45bf-91fa-93542d81a4f5 None None] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:09:00.569094 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3bd9a92-79a4-45bf-91fa-93542d81a4f5 None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:09:00.569365 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3bd9a92-79a4-45bf-91fa-93542d81a4f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:09:00.587004 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.auth.core [None req-e3bd9a92-79a4-45bf-91fa-93542d81a4f5 None None] MFA Rules not processed for user `49f280ab086b4610855c4ffa378517d9`. Rule list: `[]` (Enabled: `True`). {{(pid=81400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 03:09:00.588177 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-e3bd9a92-79a4-45bf-91fa-93542d81a4f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:09:00.612177 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:09:00.618693 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd0f9011-491e-4267-94f9-323f1d162b9c None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:09:00.652849 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcf85c86-ed34-4b19-a07a-219a1dcbf1a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:09:00.653465 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcf85c86-ed34-4b19-a07a-219a1dcbf1a7 None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:09:00.653726 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcf85c86-ed34-4b19-a07a-219a1dcbf1a7 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:09:00.654029 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcf85c86-ed34-4b19-a07a-219a1dcbf1a7 None admin] PATH_INFO: `/v3/services` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:09:00.654635 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcf85c86-ed34-4b19-a07a-219a1dcbf1a7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:09:00.655996 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcf85c86-ed34-4b19-a07a-219a1dcbf1a7 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:09:00.657935 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 502/997] 10.222.0.18 () {58 vars in 1024 bytes} [Tue Sep 13 03:09:00 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 13 03:09:00.674658 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 496/998] 10.222.0.18 () {56 vars in 1087 bytes} [Tue Sep 13 03:09:00 2022] GET /identity/v3/services => generated 1727 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 13 03:09:00.928498 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3bd9a92-79a4-45bf-91fa-93542d81a4f5 None None] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:09:01.030079 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3bd9a92-79a4-45bf-91fa-93542d81a4f5 None None] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:09:01.071036 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcc10dd1-7556-46f9-994e-ce16f90f0168 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:09:01.071490 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcc10dd1-7556-46f9-994e-ce16f90f0168 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:09:01.071774 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcc10dd1-7556-46f9-994e-ce16f90f0168 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:09:01.072045 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcc10dd1-7556-46f9-994e-ce16f90f0168 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:09:01.072629 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcc10dd1-7556-46f9-994e-ce16f90f0168 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:09:01.073989 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcc10dd1-7556-46f9-994e-ce16f90f0168 None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:09:01.083454 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application [None req-dcc10dd1-7556-46f9-994e-ce16f90f0168 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 13 03:09:01.083454 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 03:09:01.083454 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 13 03:09:01.083454 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 13 03:09:01.083454 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 13 03:09:01.083454 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 13 03:09:01.083454 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 13 03:09:01.083454 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 13 03:09:01.083454 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 13 03:09:01.083454 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Sep 13 03:09:01.083454 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 13 03:09:01.083454 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Sep 13 03:09:01.083454 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 03:09:01.083454 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 03:09:01.083454 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 03:09:01.083454 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 03:09:01.083454 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 03:09:01.084453 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 03:09:01.084453 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 03:09:01.084453 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 03:09:01.084453 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 03:09:01.084453 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 03:09:01.084453 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 03:09:01.084453 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 13 03:09:01.084453 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 13 03:09:01.084453 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 13 03:09:01.084453 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 13 03:09:01.084453 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 03:09:01.084453 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 03:09:01.084453 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 03:09:01.084453 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 03:09:01.084453 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 03:09:01.084453 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 03:09:01.085315 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 03:09:01.085315 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application with Lock( Sep 13 03:09:01.085315 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 03:09:01.085315 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._enter() Sep 13 03:09:01.085315 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 03:09:01.085315 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 03:09:01.085315 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 03:09:01.085315 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.creator() Sep 13 03:09:01.085315 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 03:09:01.085315 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application created_value = creator( Sep 13 03:09:01.085315 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 13 03:09:01.085315 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 13 03:09:01.085315 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 13 03:09:01.085315 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Sep 13 03:09:01.085315 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 503/999] 10.222.0.18 () {56 vars in 1126 bytes} [Tue Sep 13 03:09:00 2022] GET /identity/v3/services/object-store => generated 93 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 03:09:01.092500 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcf85c86-ed34-4b19-a07a-219a1dcbf1a7 None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:09:01.175926 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:09:01.195725 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcf85c86-ed34-4b19-a07a-219a1dcbf1a7 None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:09:01.229063 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-614f395c-99d0-4fbc-a2f1-8c6be948456d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:09:01.229669 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-614f395c-99d0-4fbc-a2f1-8c6be948456d None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:09:01.229940 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-614f395c-99d0-4fbc-a2f1-8c6be948456d None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:09:01.230208 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-614f395c-99d0-4fbc-a2f1-8c6be948456d None admin] PATH_INFO: `/v3/services` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:09:01.230849 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-614f395c-99d0-4fbc-a2f1-8c6be948456d None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 03:09:01.231149 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-614f395c-99d0-4fbc-a2f1-8c6be948456d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:09:01.233100 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-614f395c-99d0-4fbc-a2f1-8c6be948456d None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:09:01.237920 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 497/1000] 10.222.0.18 () {56 vars in 1122 bytes} [Tue Sep 13 03:09:01 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 03:09:01.243811 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcc10dd1-7556-46f9-994e-ce16f90f0168 None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:09:01.269660 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:09:01.351098 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcc10dd1-7556-46f9-994e-ce16f90f0168 None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:09:01.384263 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e7bde69-645d-49aa-8f22-9eec55c53cc7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:09:01.384866 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e7bde69-645d-49aa-8f22-9eec55c53cc7 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:09:01.385141 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e7bde69-645d-49aa-8f22-9eec55c53cc7 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:09:01.385407 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e7bde69-645d-49aa-8f22-9eec55c53cc7 None admin] PATH_INFO: `/v3/services` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:09:01.386025 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e7bde69-645d-49aa-8f22-9eec55c53cc7 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 03:09:01.386324 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e7bde69-645d-49aa-8f22-9eec55c53cc7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:09:01.387719 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e7bde69-645d-49aa-8f22-9eec55c53cc7 None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:09:01.397805 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 504/1001] 10.222.0.18 () {56 vars in 1122 bytes} [Tue Sep 13 03:09:01 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 03:09:01.399086 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-614f395c-99d0-4fbc-a2f1-8c6be948456d None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:09:01.438282 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f25e7d-5244-4546-831f-0427ccf57442 None None] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:09:01.438906 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f25e7d-5244-4546-831f-0427ccf57442 None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:09:01.439355 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f25e7d-5244-4546-831f-0427ccf57442 None None] PATH_INFO: `/` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:09:01.441759 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 505/1002] 10.222.0.18 () {54 vars in 927 bytes} [Tue Sep 13 03:09:01 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 03:09:01.446639 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87a9474e-4ae5-4d1b-a296-8b145fadcf85 None None] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:09:01.447360 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87a9474e-4ae5-4d1b-a296-8b145fadcf85 None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:09:01.448292 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87a9474e-4ae5-4d1b-a296-8b145fadcf85 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:09:01.473056 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.auth.core [None req-87a9474e-4ae5-4d1b-a296-8b145fadcf85 None None] MFA Rules not processed for user `49f280ab086b4610855c4ffa378517d9`. Rule list: `[]` (Enabled: `True`). {{(pid=81400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 03:09:01.475349 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-87a9474e-4ae5-4d1b-a296-8b145fadcf85 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:09:01.506721 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-614f395c-99d0-4fbc-a2f1-8c6be948456d None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:09:01.544962 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45d3feb6-37fe-4a37-a9dc-cf4b85f18381 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:09:01.545603 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45d3feb6-37fe-4a37-a9dc-cf4b85f18381 None admin] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:09:01.545848 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45d3feb6-37fe-4a37-a9dc-cf4b85f18381 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:09:01.546121 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45d3feb6-37fe-4a37-a9dc-cf4b85f18381 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:09:01.546816 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45d3feb6-37fe-4a37-a9dc-cf4b85f18381 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:09:01.548178 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45d3feb6-37fe-4a37-a9dc-cf4b85f18381 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:09:01.550775 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 506/1003] 10.222.0.18 () {58 vars in 1024 bytes} [Tue Sep 13 03:09:01 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 13 03:09:01.566190 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 498/1004] 10.222.0.18 () {60 vars in 1144 bytes} [Tue Sep 13 03:09:01 2022] POST /identity/v3/endpoints => generated 325 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 03:09:01.819963 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87a9474e-4ae5-4d1b-a296-8b145fadcf85 None None] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:09:01.838152 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:09:01.924320 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87a9474e-4ae5-4d1b-a296-8b145fadcf85 None None] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:09:01.965643 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aebe072-140d-4060-8932-945c7a094c25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:09:01.966268 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aebe072-140d-4060-8932-945c7a094c25 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:09:01.966587 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aebe072-140d-4060-8932-945c7a094c25 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:09:01.966919 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aebe072-140d-4060-8932-945c7a094c25 None admin] PATH_INFO: `/v3/services/image` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:09:01.967494 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aebe072-140d-4060-8932-945c7a094c25 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:09:01.968848 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aebe072-140d-4060-8932-945c7a094c25 None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:09:01.977560 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application [None req-0aebe072-140d-4060-8932-945c7a094c25 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Sep 13 03:09:01.977560 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 03:09:01.977560 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 13 03:09:01.977560 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 13 03:09:01.977560 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 13 03:09:01.977560 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 13 03:09:01.977560 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 13 03:09:01.977560 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 13 03:09:01.977560 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Sep 13 03:09:01.977560 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Sep 13 03:09:01.977560 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 13 03:09:01.977560 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Sep 13 03:09:01.977560 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 03:09:01.977560 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 03:09:01.977560 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 03:09:01.977560 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 03:09:01.977560 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 03:09:01.978559 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 03:09:01.978559 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 03:09:01.978559 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 03:09:01.978559 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 03:09:01.978559 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 03:09:01.978559 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 03:09:01.978559 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 13 03:09:01.978559 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 13 03:09:01.978559 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 13 03:09:01.978559 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 13 03:09:01.978559 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 03:09:01.978559 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 03:09:01.978559 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 03:09:01.978559 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 03:09:01.978559 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 03:09:01.978559 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 03:09:01.979451 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 03:09:01.979451 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application with Lock( Sep 13 03:09:01.979451 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 03:09:01.979451 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._enter() Sep 13 03:09:01.979451 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 03:09:01.979451 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 03:09:01.979451 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 03:09:01.979451 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.creator() Sep 13 03:09:01.979451 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 03:09:01.979451 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application created_value = creator( Sep 13 03:09:01.979451 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 13 03:09:01.979451 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 13 03:09:01.979451 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Sep 13 03:09:01.979451 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Sep 13 03:09:01.982682 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 507/1005] 10.222.0.18 () {56 vars in 1105 bytes} [Tue Sep 13 03:09:01 2022] GET /identity/v3/services/image => generated 86 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 03:09:01.984659 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45d3feb6-37fe-4a37-a9dc-cf4b85f18381 None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:09:01.988406 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:09:02.094351 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45d3feb6-37fe-4a37-a9dc-cf4b85f18381 None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:09:02.134663 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3e18f17-241e-4e72-bd51-9b05b53736e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:09:02.135579 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e18f17-241e-4e72-bd51-9b05b53736e7 None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:09:02.136053 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e18f17-241e-4e72-bd51-9b05b53736e7 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:09:02.136533 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e18f17-241e-4e72-bd51-9b05b53736e7 None admin] PATH_INFO: `/v3/services` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:09:02.137355 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3e18f17-241e-4e72-bd51-9b05b53736e7 None admin] RBAC: Adding query filter params (name=image) {{(pid=81399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 03:09:02.137846 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3e18f17-241e-4e72-bd51-9b05b53736e7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:09:02.139459 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3e18f17-241e-4e72-bd51-9b05b53736e7 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:09:02.144654 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 499/1006] 10.222.0.18 () {56 vars in 1108 bytes} [Tue Sep 13 03:09:01 2022] GET /identity/v3/services?name=image => generated 122 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 03:09:02.150051 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aebe072-140d-4060-8932-945c7a094c25 None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:09:02.228523 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:09:02.254015 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aebe072-140d-4060-8932-945c7a094c25 None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:09:02.287003 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65003801-195b-41af-b7e7-e5dacb94405e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:09:02.287614 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65003801-195b-41af-b7e7-e5dacb94405e None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:09:02.287902 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65003801-195b-41af-b7e7-e5dacb94405e None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:09:02.288162 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65003801-195b-41af-b7e7-e5dacb94405e None admin] PATH_INFO: `/v3/services` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:09:02.288784 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65003801-195b-41af-b7e7-e5dacb94405e None admin] RBAC: Adding query filter params (type=image) {{(pid=81400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 03:09:02.289083 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65003801-195b-41af-b7e7-e5dacb94405e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:09:02.290441 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65003801-195b-41af-b7e7-e5dacb94405e None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:09:02.301214 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 508/1007] 10.222.0.18 () {56 vars in 1108 bytes} [Tue Sep 13 03:09:02 2022] GET /identity/v3/services?type=image => generated 122 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 03:09:02.814265 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2358284a-953a-4b93-8ab4-e843fff9edd1 None None] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:09:02.814832 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2358284a-953a-4b93-8ab4-e843fff9edd1 None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:09:02.814832 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2358284a-953a-4b93-8ab4-e843fff9edd1 None None] PATH_INFO: `/` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:09:02.816317 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 500/1008] 10.222.0.18 () {54 vars in 927 bytes} [Tue Sep 13 03:09:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 03:09:02.821336 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51e493eb-cc2e-4ed8-b531-5928e456d359 None None] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:09:02.821619 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51e493eb-cc2e-4ed8-b531-5928e456d359 None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:09:02.821890 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51e493eb-cc2e-4ed8-b531-5928e456d359 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:09:02.840478 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.auth.core [None req-51e493eb-cc2e-4ed8-b531-5928e456d359 None None] MFA Rules not processed for user `49f280ab086b4610855c4ffa378517d9`. Rule list: `[]` (Enabled: `True`). {{(pid=81400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 03:09:02.842214 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-51e493eb-cc2e-4ed8-b531-5928e456d359 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:09:02.909605 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 509/1009] 10.222.0.18 () {58 vars in 1024 bytes} [Tue Sep 13 03:09:02 2022] POST /identity/v3/auth/tokens => generated 1717 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 13 03:09:03.181167 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2358284a-953a-4b93-8ab4-e843fff9edd1 None None] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:09:03.251655 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:09:03.287206 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2358284a-953a-4b93-8ab4-e843fff9edd1 None None] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:09:03.320558 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-277f702b-4ed7-4500-95dd-0d7bc78ddc0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:09:03.321108 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-277f702b-4ed7-4500-95dd-0d7bc78ddc0a None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:09:03.321385 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-277f702b-4ed7-4500-95dd-0d7bc78ddc0a None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:09:03.321653 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-277f702b-4ed7-4500-95dd-0d7bc78ddc0a None admin] PATH_INFO: `/v3/services/compute` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:09:03.322223 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-277f702b-4ed7-4500-95dd-0d7bc78ddc0a None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:09:03.323599 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-277f702b-4ed7-4500-95dd-0d7bc78ddc0a None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:09:03.332949 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application [None req-277f702b-4ed7-4500-95dd-0d7bc78ddc0a None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Sep 13 03:09:03.332949 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 03:09:03.332949 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 13 03:09:03.332949 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 13 03:09:03.332949 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 13 03:09:03.332949 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 13 03:09:03.332949 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 13 03:09:03.332949 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 13 03:09:03.332949 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Sep 13 03:09:03.332949 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Sep 13 03:09:03.332949 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 13 03:09:03.332949 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Sep 13 03:09:03.332949 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 03:09:03.332949 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 03:09:03.332949 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 03:09:03.332949 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 03:09:03.332949 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 03:09:03.333930 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 03:09:03.333930 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 03:09:03.333930 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 03:09:03.333930 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 03:09:03.333930 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 03:09:03.333930 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 03:09:03.333930 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 13 03:09:03.333930 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 13 03:09:03.333930 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 13 03:09:03.333930 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 13 03:09:03.333930 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 03:09:03.333930 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 03:09:03.333930 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 03:09:03.333930 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 03:09:03.333930 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 03:09:03.333930 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 03:09:03.334822 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 03:09:03.334822 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application with Lock( Sep 13 03:09:03.334822 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 03:09:03.334822 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._enter() Sep 13 03:09:03.334822 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 03:09:03.334822 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 03:09:03.334822 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 03:09:03.334822 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.creator() Sep 13 03:09:03.334822 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 03:09:03.334822 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application created_value = creator( Sep 13 03:09:03.334822 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 13 03:09:03.334822 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 13 03:09:03.334822 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Sep 13 03:09:03.334822 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Sep 13 03:09:03.338992 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 501/1010] 10.222.0.18 () {56 vars in 1111 bytes} [Tue Sep 13 03:09:03 2022] GET /identity/v3/services/compute => generated 88 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 03:09:03.340195 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51e493eb-cc2e-4ed8-b531-5928e456d359 None None] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:09:03.408303 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:09:03.439865 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51e493eb-cc2e-4ed8-b531-5928e456d359 None None] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:09:03.474040 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-183bae9b-459a-4649-b514-3400109b17ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:09:03.474955 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-183bae9b-459a-4649-b514-3400109b17ae None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:09:03.475411 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-183bae9b-459a-4649-b514-3400109b17ae None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:09:03.475854 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-183bae9b-459a-4649-b514-3400109b17ae None admin] PATH_INFO: `/v3/services` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:09:03.476646 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-183bae9b-459a-4649-b514-3400109b17ae None admin] RBAC: Adding query filter params (name=compute) {{(pid=81400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 03:09:03.477146 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-183bae9b-459a-4649-b514-3400109b17ae None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:09:03.478746 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-183bae9b-459a-4649-b514-3400109b17ae None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:09:03.488157 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 510/1011] 10.222.0.18 () {56 vars in 1112 bytes} [Tue Sep 13 03:09:03 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 03:09:03.489783 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-277f702b-4ed7-4500-95dd-0d7bc78ddc0a None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:09:03.590226 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-277f702b-4ed7-4500-95dd-0d7bc78ddc0a None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:09:03.620694 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:09:03.627731 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-975a41fa-c303-4ddf-a3ad-4c4f794e2c30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:09:03.628365 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-975a41fa-c303-4ddf-a3ad-4c4f794e2c30 None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:09:03.628743 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-975a41fa-c303-4ddf-a3ad-4c4f794e2c30 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:09:03.629009 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-975a41fa-c303-4ddf-a3ad-4c4f794e2c30 None admin] PATH_INFO: `/v3/services` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:09:03.629647 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-975a41fa-c303-4ddf-a3ad-4c4f794e2c30 None admin] RBAC: Adding query filter params (type=compute) {{(pid=81399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 03:09:03.629944 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-975a41fa-c303-4ddf-a3ad-4c4f794e2c30 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:09:03.631365 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-975a41fa-c303-4ddf-a3ad-4c4f794e2c30 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:09:03.641944 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 502/1012] 10.222.0.18 () {56 vars in 1112 bytes} [Tue Sep 13 03:09:03 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 03:09:03.642930 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-183bae9b-459a-4649-b514-3400109b17ae None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:09:03.738564 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd00100b-6bf1-4b96-894d-485e6843a233 None None] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:09:03.738906 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd00100b-6bf1-4b96-894d-485e6843a233 None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:09:03.739141 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd00100b-6bf1-4b96-894d-485e6843a233 None None] PATH_INFO: `/` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:09:03.742646 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 503/1013] 10.222.0.18 () {54 vars in 927 bytes} [Tue Sep 13 03:09:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 13 03:09:03.745407 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d953f97-06e0-4d72-9b19-d7fb3ffbc66f None None] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:09:03.745694 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d953f97-06e0-4d72-9b19-d7fb3ffbc66f None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:09:03.745968 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d953f97-06e0-4d72-9b19-d7fb3ffbc66f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:09:03.749087 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-183bae9b-459a-4649-b514-3400109b17ae None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:09:03.765687 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.auth.core [None req-3d953f97-06e0-4d72-9b19-d7fb3ffbc66f None None] MFA Rules not processed for user `49f280ab086b4610855c4ffa378517d9`. Rule list: `[]` (Enabled: `True`). {{(pid=81399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 03:09:03.767202 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-3d953f97-06e0-4d72-9b19-d7fb3ffbc66f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:09:03.788556 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d266b5e-7be7-4921-bcb1-032409b4cded None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:09:03.789189 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d266b5e-7be7-4921-bcb1-032409b4cded None admin] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:09:03.789466 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d266b5e-7be7-4921-bcb1-032409b4cded None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:09:03.789734 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d266b5e-7be7-4921-bcb1-032409b4cded None admin] PATH_INFO: `/v3/endpoints` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:09:03.790402 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d266b5e-7be7-4921-bcb1-032409b4cded None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:09:03.791793 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d266b5e-7be7-4921-bcb1-032409b4cded None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:09:03.796735 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:09:03.822643 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 511/1014] 10.222.0.18 () {60 vars in 1144 bytes} [Tue Sep 13 03:09:03 2022] POST /identity/v3/endpoints => generated 334 bytes in 170 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 13 03:09:03.863084 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 504/1015] 10.222.0.18 () {58 vars in 1024 bytes} [Tue Sep 13 03:09:03 2022] POST /identity/v3/auth/tokens => generated 1873 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 13 03:09:04.132486 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d266b5e-7be7-4921-bcb1-032409b4cded None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:09:04.237388 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d266b5e-7be7-4921-bcb1-032409b4cded None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:09:04.272970 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbb097b6-1877-420d-a466-66bceb911504 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:09:04.273599 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb097b6-1877-420d-a466-66bceb911504 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:09:04.273877 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb097b6-1877-420d-a466-66bceb911504 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:09:04.274146 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb097b6-1877-420d-a466-66bceb911504 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:09:04.275263 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbb097b6-1877-420d-a466-66bceb911504 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:09:04.276689 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbb097b6-1877-420d-a466-66bceb911504 None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:09:04.278416 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 512/1016] 10.222.0.18 () {56 vars in 1108 bytes} [Tue Sep 13 03:09:04 2022] GET /identity/v3/domains/default => generated 199 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 03:09:04.284926 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d953f97-06e0-4d72-9b19-d7fb3ffbc66f None None] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:09:04.390026 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d953f97-06e0-4d72-9b19-d7fb3ffbc66f None None] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:09:04.414107 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:09:04.426782 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fda315b3-89fa-4ad8-88a4-64977f079512 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:09:04.427396 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda315b3-89fa-4ad8-88a4-64977f079512 None admin] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:09:04.427674 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda315b3-89fa-4ad8-88a4-64977f079512 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:09:04.427945 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda315b3-89fa-4ad8-88a4-64977f079512 None admin] PATH_INFO: `/v3/projects` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:09:04.428601 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fda315b3-89fa-4ad8-88a4-64977f079512 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:09:04.430286 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fda315b3-89fa-4ad8-88a4-64977f079512 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:09:04.462643 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 505/1017] 10.222.0.18 () {60 vars in 1141 bytes} [Tue Sep 13 03:09:04 2022] POST /identity/v3/projects => generated 311 bytes in 173 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 03:09:04.474701 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc43f031-a0cd-4da0-996d-3592ec2c28d7 None None] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:09:04.475273 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc43f031-a0cd-4da0-996d-3592ec2c28d7 None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:09:04.475273 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc43f031-a0cd-4da0-996d-3592ec2c28d7 None None] PATH_INFO: `/` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:09:04.476678 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 513/1018] 10.222.0.18 () {54 vars in 927 bytes} [Tue Sep 13 03:09:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 03:09:04.482817 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc6dea3-39f6-4021-a025-c628ad6a2301 None None] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:09:04.483105 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc6dea3-39f6-4021-a025-c628ad6a2301 None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:09:04.483387 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc6dea3-39f6-4021-a025-c628ad6a2301 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:09:04.502503 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.auth.core [None req-fdc6dea3-39f6-4021-a025-c628ad6a2301 None None] MFA Rules not processed for user `49f280ab086b4610855c4ffa378517d9`. Rule list: `[]` (Enabled: `True`). {{(pid=81399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 03:09:04.503992 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-fdc6dea3-39f6-4021-a025-c628ad6a2301 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:09:04.574877 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 506/1019] 10.222.0.18 () {58 vars in 1024 bytes} [Tue Sep 13 03:09:04 2022] POST /identity/v3/auth/tokens => generated 1873 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 13 03:09:04.844365 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc43f031-a0cd-4da0-996d-3592ec2c28d7 None None] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:09:04.904084 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:09:04.949389 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc43f031-a0cd-4da0-996d-3592ec2c28d7 None None] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:09:04.982883 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49e6b140-737f-43fc-850b-11134687361f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:09:04.983486 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49e6b140-737f-43fc-850b-11134687361f None admin] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:09:04.983764 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49e6b140-737f-43fc-850b-11134687361f None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:09:04.984024 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49e6b140-737f-43fc-850b-11134687361f None admin] PATH_INFO: `/v3/services` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:09:04.984680 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49e6b140-737f-43fc-850b-11134687361f None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:09:04.986110 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49e6b140-737f-43fc-850b-11134687361f None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:09:05.001748 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 514/1020] 10.222.0.18 () {60 vars in 1141 bytes} [Tue Sep 13 03:09:04 2022] POST /identity/v3/services => generated 242 bytes in 158 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 03:09:06.082896 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d95e840e-a22f-4930-ad02-6ac8c0d6357a None None] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:09:06.083377 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d95e840e-a22f-4930-ad02-6ac8c0d6357a None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:09:06.083470 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d95e840e-a22f-4930-ad02-6ac8c0d6357a None None] PATH_INFO: `/` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:09:06.084798 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 507/1021] 10.222.0.18 () {54 vars in 927 bytes} [Tue Sep 13 03:09:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 03:09:06.090125 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-decd48e3-687e-422f-8aef-d5476f69973d None None] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:09:06.090545 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-decd48e3-687e-422f-8aef-d5476f69973d None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:09:06.090762 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-decd48e3-687e-422f-8aef-d5476f69973d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:09:06.109177 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.auth.core [None req-decd48e3-687e-422f-8aef-d5476f69973d None None] MFA Rules not processed for user `49f280ab086b4610855c4ffa378517d9`. Rule list: `[]` (Enabled: `True`). {{(pid=81400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 03:09:06.110344 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-decd48e3-687e-422f-8aef-d5476f69973d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:09:06.182802 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 515/1022] 10.222.0.18 () {58 vars in 1024 bytes} [Tue Sep 13 03:09:06 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 13 03:09:06.446256 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d95e840e-a22f-4930-ad02-6ac8c0d6357a None None] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:09:06.547744 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d95e840e-a22f-4930-ad02-6ac8c0d6357a None None] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:09:06.550513 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:09:06.586296 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef82cabc-a8b9-4c24-9d06-200a4c749f83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:09:06.586950 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef82cabc-a8b9-4c24-9d06-200a4c749f83 None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:09:06.587222 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef82cabc-a8b9-4c24-9d06-200a4c749f83 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:09:06.587486 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef82cabc-a8b9-4c24-9d06-200a4c749f83 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:09:06.597461 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef82cabc-a8b9-4c24-9d06-200a4c749f83 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 03:09:06.602281 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application [None req-ef82cabc-a8b9-4c24-9d06-200a4c749f83 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 13 03:09:06.602281 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 03:09:06.602281 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 03:09:06.602281 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 03:09:06.602281 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 03:09:06.602281 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 03:09:06.602281 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 03:09:06.602281 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 03:09:06.602281 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 03:09:06.602281 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 03:09:06.602281 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 03:09:06.602281 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 03:09:06.602281 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 03:09:06.602281 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 03:09:06.602281 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 03:09:06.602281 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 03:09:06.603238 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 03:09:06.603238 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 03:09:06.603238 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 03:09:06.603238 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 03:09:06.603238 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 03:09:06.603238 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 03:09:06.603238 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 03:09:06.603238 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 03:09:06.603238 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 03:09:06.603238 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 03:09:06.603238 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 03:09:06.603238 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application with Lock( Sep 13 03:09:06.603238 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 03:09:06.603238 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._enter() Sep 13 03:09:06.603238 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 03:09:06.603238 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 03:09:06.604111 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 03:09:06.604111 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.creator() Sep 13 03:09:06.604111 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 03:09:06.604111 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application created_value = creator( Sep 13 03:09:06.604111 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 03:09:06.604111 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 03:09:06.604111 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 03:09:06.604111 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 03:09:06.604111 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 03:09:06.604111 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 03:09:06.604111 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 13 03:09:06.604111 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Sep 13 03:09:06.605513 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 508/1023] 10.222.0.18 () {56 vars in 1120 bytes} [Tue Sep 13 03:09:06 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 03:09:06.609946 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-decd48e3-687e-422f-8aef-d5476f69973d None None] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:09:06.654919 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:09:06.662247 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4acfa38-b3bd-4bdf-a56f-9e0c9c119b22 None None] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:09:06.662955 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4acfa38-b3bd-4bdf-a56f-9e0c9c119b22 None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:09:06.663231 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4acfa38-b3bd-4bdf-a56f-9e0c9c119b22 None None] PATH_INFO: `/` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:09:06.664673 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 509/1024] 10.222.0.18 () {54 vars in 927 bytes} [Tue Sep 13 03:09:06 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 13 03:09:06.670265 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8efa8bf-3195-4c94-8617-9b637b639548 None None] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:09:06.670546 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8efa8bf-3195-4c94-8617-9b637b639548 None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:09:06.670846 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8efa8bf-3195-4c94-8617-9b637b639548 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:09:06.691365 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.auth.core [None req-e8efa8bf-3195-4c94-8617-9b637b639548 None None] MFA Rules not processed for user `49f280ab086b4610855c4ffa378517d9`. Rule list: `[]` (Enabled: `True`). {{(pid=81399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 03:09:06.692563 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-e8efa8bf-3195-4c94-8617-9b637b639548 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:09:06.714837 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-decd48e3-687e-422f-8aef-d5476f69973d None None] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:09:06.751242 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21f92a4b-3040-4675-919e-32b77ad22ff8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:09:06.751839 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f92a4b-3040-4675-919e-32b77ad22ff8 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:09:06.752114 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f92a4b-3040-4675-919e-32b77ad22ff8 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:09:06.752517 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f92a4b-3040-4675-919e-32b77ad22ff8 None admin] PATH_INFO: `/v3/roles` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:09:06.753613 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21f92a4b-3040-4675-919e-32b77ad22ff8 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 03:09:06.754159 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21f92a4b-3040-4675-919e-32b77ad22ff8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:09:06.756732 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21f92a4b-3040-4675-919e-32b77ad22ff8 None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:09:06.766216 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 516/1025] 10.222.0.18 () {56 vars in 1115 bytes} [Tue Sep 13 03:09:06 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 03:09:06.769076 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 510/1026] 10.222.0.18 () {58 vars in 1024 bytes} [Tue Sep 13 03:09:06 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 13 03:09:06.771728 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21f92a4b-3040-4675-919e-32b77ad22ff8 None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:09:06.871166 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21f92a4b-3040-4675-919e-32b77ad22ff8 None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:09:06.878543 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:09:06.906182 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1371ed4-9125-4ffa-8169-e7608165c342 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:09:06.906870 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1371ed4-9125-4ffa-8169-e7608165c342 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:09:06.907144 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1371ed4-9125-4ffa-8169-e7608165c342 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:09:06.907462 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1371ed4-9125-4ffa-8169-e7608165c342 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:09:06.917789 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1371ed4-9125-4ffa-8169-e7608165c342 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:09:06.919297 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1371ed4-9125-4ffa-8169-e7608165c342 None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:09:06.933976 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application [None req-b1371ed4-9125-4ffa-8169-e7608165c342 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 03:09:06.933976 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 03:09:06.933976 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 03:09:06.933976 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 03:09:06.933976 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 03:09:06.933976 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 03:09:06.933976 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 03:09:06.933976 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 03:09:06.933976 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 03:09:06.933976 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 03:09:06.933976 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 03:09:06.933976 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 03:09:06.933976 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 13 03:09:06.933976 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 13 03:09:06.933976 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 13 03:09:06.933976 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 13 03:09:06.935272 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 03:09:06.935272 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 03:09:06.935272 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 03:09:06.935272 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 03:09:06.935272 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 03:09:06.935272 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 03:09:06.935272 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 03:09:06.935272 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application with Lock( Sep 13 03:09:06.935272 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 03:09:06.935272 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._enter() Sep 13 03:09:06.935272 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 03:09:06.935272 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 03:09:06.935272 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 03:09:06.935272 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.creator() Sep 13 03:09:06.935272 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 03:09:06.935272 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application created_value = creator( Sep 13 03:09:06.935272 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 13 03:09:06.936509 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 13 03:09:06.936509 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 03:09:06.936509 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Sep 13 03:09:06.938950 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 517/1027] 10.222.0.18 () {56 vars in 1108 bytes} [Tue Sep 13 03:09:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 13 03:09:06.940743 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8efa8bf-3195-4c94-8617-9b637b639548 None None] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:09:07.030880 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:09:07.038532 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8efa8bf-3195-4c94-8617-9b637b639548 None None] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:09:07.040897 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1371ed4-9125-4ffa-8169-e7608165c342 None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:09:07.081083 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17f45a35-6eae-4655-9b87-3c9e2f74b79f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:09:07.081720 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f45a35-6eae-4655-9b87-3c9e2f74b79f None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:09:07.081999 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f45a35-6eae-4655-9b87-3c9e2f74b79f None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:09:07.082271 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f45a35-6eae-4655-9b87-3c9e2f74b79f None admin] PATH_INFO: `/v3/domains` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:09:07.082976 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17f45a35-6eae-4655-9b87-3c9e2f74b79f None admin] RBAC: Adding query filter params (name=Default) {{(pid=81399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 03:09:07.083551 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17f45a35-6eae-4655-9b87-3c9e2f74b79f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:09:07.084682 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17f45a35-6eae-4655-9b87-3c9e2f74b79f None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:09:07.102535 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 511/1028] 10.222.0.18 () {56 vars in 1109 bytes} [Tue Sep 13 03:09:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 03:09:07.103743 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17f45a35-6eae-4655-9b87-3c9e2f74b79f None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:09:07.154547 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1371ed4-9125-4ffa-8169-e7608165c342 None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:09:07.189389 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dea93746-a616-423b-b5a7-2d666d141ebb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:09:07.190020 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dea93746-a616-423b-b5a7-2d666d141ebb None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:09:07.190299 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dea93746-a616-423b-b5a7-2d666d141ebb None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:09:07.190570 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dea93746-a616-423b-b5a7-2d666d141ebb None admin] PATH_INFO: `/v3/domains/default` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:09:07.192973 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dea93746-a616-423b-b5a7-2d666d141ebb None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:09:07.194400 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dea93746-a616-423b-b5a7-2d666d141ebb None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:09:07.196197 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 518/1029] 10.222.0.18 () {56 vars in 1108 bytes} [Tue Sep 13 03:09:07 2022] GET /identity/v3/domains/default => generated 199 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 03:09:07.201468 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dea93746-a616-423b-b5a7-2d666d141ebb None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:09:07.214014 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17f45a35-6eae-4655-9b87-3c9e2f74b79f None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:09:07.230315 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:09:07.247902 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4a861de-25bf-4757-9c8c-a79177f8b0ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:09:07.248531 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a861de-25bf-4757-9c8c-a79177f8b0ed None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:09:07.248812 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a861de-25bf-4757-9c8c-a79177f8b0ed None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:09:07.249502 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a861de-25bf-4757-9c8c-a79177f8b0ed None admin] PATH_INFO: `/v3/users/nova` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:09:07.256879 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4a861de-25bf-4757-9c8c-a79177f8b0ed None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:09:07.258314 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4a861de-25bf-4757-9c8c-a79177f8b0ed None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:09:07.270065 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application [None req-a4a861de-25bf-4757-9c8c-a79177f8b0ed None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 13 03:09:07.270065 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 03:09:07.270065 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 03:09:07.270065 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 03:09:07.270065 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 03:09:07.270065 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 03:09:07.270065 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 03:09:07.270065 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 03:09:07.270065 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 03:09:07.270065 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 03:09:07.270065 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 03:09:07.270065 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 03:09:07.270065 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 13 03:09:07.270065 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 13 03:09:07.270065 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 13 03:09:07.270065 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 13 03:09:07.271122 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 03:09:07.271122 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 03:09:07.271122 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 13 03:09:07.271122 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 13 03:09:07.271122 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 13 03:09:07.271122 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 13 03:09:07.271122 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 03:09:07.271122 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 03:09:07.271122 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 03:09:07.271122 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 03:09:07.271122 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 03:09:07.271122 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application with Lock( Sep 13 03:09:07.271122 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 03:09:07.271122 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._enter() Sep 13 03:09:07.271122 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 03:09:07.271122 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 03:09:07.271122 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 03:09:07.271122 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.creator() Sep 13 03:09:07.272093 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 03:09:07.272093 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application created_value = creator( Sep 13 03:09:07.272093 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 13 03:09:07.272093 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 13 03:09:07.272093 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 13 03:09:07.272093 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 13 03:09:07.272093 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 13 03:09:07.272093 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 13 03:09:07.272093 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 13 03:09:07.272093 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Sep 13 03:09:07.272093 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 512/1030] 10.222.0.18 () {56 vars in 1093 bytes} [Tue Sep 13 03:09:07 2022] GET /identity/v3/users/nova => generated 82 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 13 03:09:07.272714 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ed1ace-26fa-4e79-8b9b-b1d542fdbe71 None None] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:09:07.272714 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ed1ace-26fa-4e79-8b9b-b1d542fdbe71 None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:09:07.273319 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ed1ace-26fa-4e79-8b9b-b1d542fdbe71 None None] PATH_INFO: `/` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:09:07.274463 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 513/1031] 10.222.0.18 () {54 vars in 927 bytes} [Tue Sep 13 03:09:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 03:09:07.276769 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63ed1ace-26fa-4e79-8b9b-b1d542fdbe71 None None] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:09:07.292373 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:09:07.308962 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dea93746-a616-423b-b5a7-2d666d141ebb None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:09:07.342200 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc758257-1ad9-4c45-8187-636b221eac35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:09:07.342896 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc758257-1ad9-4c45-8187-636b221eac35 None admin] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:09:07.343176 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc758257-1ad9-4c45-8187-636b221eac35 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:09:07.343443 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc758257-1ad9-4c45-8187-636b221eac35 None admin] PATH_INFO: `/v3/users` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:09:07.344080 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc758257-1ad9-4c45-8187-636b221eac35 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:09:07.345505 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc758257-1ad9-4c45-8187-636b221eac35 None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:09:07.365674 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 519/1032] 10.222.0.18 () {60 vars in 1132 bytes} [Tue Sep 13 03:09:07 2022] POST /identity/v3/users => generated 285 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 13 03:09:07.372798 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec5256f-c5ed-4fa4-b2a3-451cc09600e8 None None] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:09:07.373092 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec5256f-c5ed-4fa4-b2a3-451cc09600e8 None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:09:07.373362 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec5256f-c5ed-4fa4-b2a3-451cc09600e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:09:07.378897 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63ed1ace-26fa-4e79-8b9b-b1d542fdbe71 None None] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:09:07.394141 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.auth.core [None req-7ec5256f-c5ed-4fa4-b2a3-451cc09600e8 None None] MFA Rules not processed for user `49f280ab086b4610855c4ffa378517d9`. Rule list: `[]` (Enabled: `True`). {{(pid=81400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 03:09:07.395532 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-7ec5256f-c5ed-4fa4-b2a3-451cc09600e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:09:07.416671 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57d79340-b69b-46cf-bf3a-660822838f97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:09:07.417308 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d79340-b69b-46cf-bf3a-660822838f97 None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:09:07.417569 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d79340-b69b-46cf-bf3a-660822838f97 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:09:07.417837 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d79340-b69b-46cf-bf3a-660822838f97 None admin] PATH_INFO: `/v3/users` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:09:07.418566 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57d79340-b69b-46cf-bf3a-660822838f97 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 03:09:07.418907 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57d79340-b69b-46cf-bf3a-660822838f97 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:09:07.420418 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57d79340-b69b-46cf-bf3a-660822838f97 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:09:07.442338 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 514/1033] 10.222.0.18 () {56 vars in 1133 bytes} [Tue Sep 13 03:09:07 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 03:09:07.448207 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57d79340-b69b-46cf-bf3a-660822838f97 None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:09:07.478096 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 520/1034] 10.222.0.18 () {58 vars in 1024 bytes} [Tue Sep 13 03:09:07 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 13 03:09:07.533710 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:09:07.564850 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57d79340-b69b-46cf-bf3a-660822838f97 None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:09:07.599523 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28beef6a-bddf-4bf6-847f-038d498fded1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:09:07.600136 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28beef6a-bddf-4bf6-847f-038d498fded1 None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:09:07.600408 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28beef6a-bddf-4bf6-847f-038d498fded1 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:09:07.600674 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28beef6a-bddf-4bf6-847f-038d498fded1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:09:07.668708 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28beef6a-bddf-4bf6-847f-038d498fded1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:09:07.670168 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28beef6a-bddf-4bf6-847f-038d498fded1 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:09:07.736746 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:09:07.754017 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ec5256f-c5ed-4fa4-b2a3-451cc09600e8 None None] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:09:07.768806 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:09:07.769580 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 515/1035] 10.222.0.18 () {58 vars in 1282 bytes} [Tue Sep 13 03:09:07 2022] GET /identity/v3/auth/tokens => generated 1969 bytes in 322 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Sep 13 03:09:07.776149 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28beef6a-bddf-4bf6-847f-038d498fded1 None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:09:07.860718 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ec5256f-c5ed-4fa4-b2a3-451cc09600e8 None None] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:09:07.879493 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28beef6a-bddf-4bf6-847f-038d498fded1 None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:09:07.896265 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1937eeb4-e3e3-48b2-968f-951719d357ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:09:07.896897 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1937eeb4-e3e3-48b2-968f-951719d357ef None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:09:07.897174 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1937eeb4-e3e3-48b2-968f-951719d357ef None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:09:07.897443 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1937eeb4-e3e3-48b2-968f-951719d357ef None admin] PATH_INFO: `/v3/services/image` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:09:07.898029 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1937eeb4-e3e3-48b2-968f-951719d357ef None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:09:07.899568 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1937eeb4-e3e3-48b2-968f-951719d357ef None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:09:07.908642 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application [None req-1937eeb4-e3e3-48b2-968f-951719d357ef None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Sep 13 03:09:07.908642 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 03:09:07.908642 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 13 03:09:07.908642 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 13 03:09:07.908642 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 13 03:09:07.908642 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 13 03:09:07.908642 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 13 03:09:07.908642 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 13 03:09:07.908642 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Sep 13 03:09:07.908642 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Sep 13 03:09:07.908642 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 13 03:09:07.908642 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Sep 13 03:09:07.908642 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 03:09:07.908642 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 03:09:07.908642 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 03:09:07.908642 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 03:09:07.908642 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 03:09:07.909691 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 03:09:07.909691 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 03:09:07.909691 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 03:09:07.909691 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 03:09:07.909691 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 03:09:07.909691 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 03:09:07.909691 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 13 03:09:07.909691 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 13 03:09:07.909691 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 13 03:09:07.909691 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 13 03:09:07.909691 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 03:09:07.909691 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 03:09:07.909691 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 03:09:07.909691 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 03:09:07.909691 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 03:09:07.909691 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 03:09:07.910560 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 03:09:07.910560 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application with Lock( Sep 13 03:09:07.910560 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 03:09:07.910560 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._enter() Sep 13 03:09:07.910560 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 03:09:07.910560 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 03:09:07.910560 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 03:09:07.910560 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.creator() Sep 13 03:09:07.910560 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 03:09:07.910560 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application created_value = creator( Sep 13 03:09:07.910560 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 13 03:09:07.910560 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 13 03:09:07.910560 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Sep 13 03:09:07.910560 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Sep 13 03:09:07.910560 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 521/1036] 10.222.0.18 () {56 vars in 1105 bytes} [Tue Sep 13 03:09:07 2022] GET /identity/v3/services/image => generated 86 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 13 03:09:07.915892 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1937eeb4-e3e3-48b2-968f-951719d357ef None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:09:07.920432 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef87d584-754f-4a26-b850-7b1b364e4b78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:09:07.921195 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef87d584-754f-4a26-b850-7b1b364e4b78 None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:09:07.921470 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef87d584-754f-4a26-b850-7b1b364e4b78 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:09:07.921735 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef87d584-754f-4a26-b850-7b1b364e4b78 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:09:07.932266 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef87d584-754f-4a26-b850-7b1b364e4b78 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:09:07.933710 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef87d584-754f-4a26-b850-7b1b364e4b78 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:09:07.949718 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application [None req-ef87d584-754f-4a26-b850-7b1b364e4b78 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 03:09:07.949718 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 03:09:07.949718 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 03:09:07.949718 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 03:09:07.949718 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 03:09:07.949718 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 03:09:07.949718 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 03:09:07.949718 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 03:09:07.949718 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 03:09:07.949718 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 03:09:07.949718 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 03:09:07.949718 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 03:09:07.949718 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 13 03:09:07.949718 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 13 03:09:07.949718 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 13 03:09:07.949718 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 13 03:09:07.951397 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 03:09:07.951397 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 03:09:07.951397 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 03:09:07.951397 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 03:09:07.951397 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 03:09:07.951397 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 03:09:07.951397 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 03:09:07.951397 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application with Lock( Sep 13 03:09:07.951397 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 03:09:07.951397 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._enter() Sep 13 03:09:07.951397 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 03:09:07.951397 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 03:09:07.951397 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 03:09:07.951397 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.creator() Sep 13 03:09:07.951397 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 03:09:07.951397 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application created_value = creator( Sep 13 03:09:07.951397 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 13 03:09:07.952835 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 13 03:09:07.952835 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 03:09:07.952835 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Sep 13 03:09:07.952835 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 516/1037] 10.222.0.18 () {56 vars in 1108 bytes} [Tue Sep 13 03:09:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 13 03:09:07.956786 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef87d584-754f-4a26-b850-7b1b364e4b78 None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:09:07.983773 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:09:08.011468 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:09:08.036711 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1937eeb4-e3e3-48b2-968f-951719d357ef None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:09:08.067315 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef87d584-754f-4a26-b850-7b1b364e4b78 None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:09:08.080055 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a331325-2409-4538-9f32-a997a40cd1c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:09:08.081039 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a331325-2409-4538-9f32-a997a40cd1c5 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:09:08.081822 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a331325-2409-4538-9f32-a997a40cd1c5 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:09:08.082375 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a331325-2409-4538-9f32-a997a40cd1c5 None admin] PATH_INFO: `/v3/services` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:09:08.083336 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a331325-2409-4538-9f32-a997a40cd1c5 None admin] RBAC: Adding query filter params (name=image) {{(pid=81400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 03:09:08.083932 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a331325-2409-4538-9f32-a997a40cd1c5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:09:08.085723 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a331325-2409-4538-9f32-a997a40cd1c5 None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:09:08.093879 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 522/1038] 10.222.0.18 () {56 vars in 1108 bytes} [Tue Sep 13 03:09:07 2022] GET /identity/v3/services?name=image => generated 122 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 03:09:08.097583 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a331325-2409-4538-9f32-a997a40cd1c5 None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:09:08.106523 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d7c4ed0-f081-4273-87de-50ae6ebee3e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:09:08.107695 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d7c4ed0-f081-4273-87de-50ae6ebee3e3 None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:09:08.108027 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d7c4ed0-f081-4273-87de-50ae6ebee3e3 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:09:08.108299 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d7c4ed0-f081-4273-87de-50ae6ebee3e3 None admin] PATH_INFO: `/v3/domains` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:09:08.108963 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d7c4ed0-f081-4273-87de-50ae6ebee3e3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 03:09:08.109563 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d7c4ed0-f081-4273-87de-50ae6ebee3e3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:09:08.110810 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d7c4ed0-f081-4273-87de-50ae6ebee3e3 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:09:08.124655 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 517/1039] 10.222.0.18 () {56 vars in 1109 bytes} [Tue Sep 13 03:09:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 03:09:08.129982 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d7c4ed0-f081-4273-87de-50ae6ebee3e3 None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:09:08.206861 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a331325-2409-4538-9f32-a997a40cd1c5 None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:09:08.216980 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:09:08.233689 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d7c4ed0-f081-4273-87de-50ae6ebee3e3 None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:09:08.243027 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b719679-d896-4905-9bfb-a255261a5d6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:09:08.243634 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b719679-d896-4905-9bfb-a255261a5d6d None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:09:08.243913 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b719679-d896-4905-9bfb-a255261a5d6d None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:09:08.244182 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b719679-d896-4905-9bfb-a255261a5d6d None admin] PATH_INFO: `/v3/services` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:09:08.244806 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b719679-d896-4905-9bfb-a255261a5d6d None admin] RBAC: Adding query filter params (type=image) {{(pid=81400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 03:09:08.245104 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b719679-d896-4905-9bfb-a255261a5d6d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:09:08.246464 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b719679-d896-4905-9bfb-a255261a5d6d None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:09:08.252320 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 523/1040] 10.222.0.18 () {56 vars in 1108 bytes} [Tue Sep 13 03:09:08 2022] GET /identity/v3/services?type=image => generated 350 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 03:09:08.257886 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b719679-d896-4905-9bfb-a255261a5d6d None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:09:08.261381 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:09:08.270659 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-323efd54-67d6-43dc-abdd-6a93dd73c24e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:09:08.271300 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-323efd54-67d6-43dc-abdd-6a93dd73c24e None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:09:08.271579 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-323efd54-67d6-43dc-abdd-6a93dd73c24e None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:09:08.271854 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-323efd54-67d6-43dc-abdd-6a93dd73c24e None admin] PATH_INFO: `/v3/projects/service` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:09:08.278796 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-323efd54-67d6-43dc-abdd-6a93dd73c24e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:09:08.280225 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-323efd54-67d6-43dc-abdd-6a93dd73c24e None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:09:08.291561 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application [None req-323efd54-67d6-43dc-abdd-6a93dd73c24e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 13 03:09:08.291561 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 03:09:08.291561 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 03:09:08.291561 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 03:09:08.291561 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 03:09:08.291561 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 03:09:08.291561 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 03:09:08.291561 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 03:09:08.291561 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 03:09:08.291561 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 03:09:08.291561 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 03:09:08.291561 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 03:09:08.291561 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 13 03:09:08.291561 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 13 03:09:08.291561 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 13 03:09:08.292492 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 13 03:09:08.292492 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 03:09:08.292492 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 03:09:08.292492 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 03:09:08.292492 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 03:09:08.292492 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 03:09:08.292492 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 03:09:08.292492 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 03:09:08.292492 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application with Lock( Sep 13 03:09:08.292492 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 03:09:08.292492 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._enter() Sep 13 03:09:08.292492 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 03:09:08.292492 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 03:09:08.292492 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 03:09:08.292492 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.creator() Sep 13 03:09:08.292492 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 03:09:08.292492 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application created_value = creator( Sep 13 03:09:08.299182 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 13 03:09:08.299182 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 13 03:09:08.299182 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 13 03:09:08.299182 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 13 03:09:08.299182 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 13 03:09:08.299182 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 13 03:09:08.299182 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 13 03:09:08.299182 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Sep 13 03:09:08.299182 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 518/1041] 10.222.0.18 () {56 vars in 1111 bytes} [Tue Sep 13 03:09:08 2022] GET /identity/v3/projects/service => generated 88 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 13 03:09:08.299716 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-323efd54-67d6-43dc-abdd-6a93dd73c24e None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:09:08.368847 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b719679-d896-4905-9bfb-a255261a5d6d None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:09:08.402137 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52dfd2de-0f06-4dd3-a3ef-0ff36117aa97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:09:08.402861 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52dfd2de-0f06-4dd3-a3ef-0ff36117aa97 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:09:08.403140 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52dfd2de-0f06-4dd3-a3ef-0ff36117aa97 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:09:08.403408 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52dfd2de-0f06-4dd3-a3ef-0ff36117aa97 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:09:08.404802 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52dfd2de-0f06-4dd3-a3ef-0ff36117aa97 None admin] RBAC: Adding query filter params (interface=public, service_id=470490b55d48474c9bb1542b868c81f1, region_id=RegionOne) {{(pid=81400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 03:09:08.404904 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-323efd54-67d6-43dc-abdd-6a93dd73c24e None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:09:08.405107 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52dfd2de-0f06-4dd3-a3ef-0ff36117aa97 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:09:08.406464 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52dfd2de-0f06-4dd3-a3ef-0ff36117aa97 None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:09:08.411654 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 524/1042] 10.222.0.18 () {56 vars in 1251 bytes} [Tue Sep 13 03:09:08 2022] GET /identity/v3/endpoints?service_id=470490b55d48474c9bb1542b868c81f1&interface=public®ion_id=RegionOne => generated 194 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 03:09:08.417013 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52dfd2de-0f06-4dd3-a3ef-0ff36117aa97 None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:09:08.439731 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f1c0875-e872-43cd-961e-d9ad851e2c27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:09:08.440350 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f1c0875-e872-43cd-961e-d9ad851e2c27 None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:09:08.440634 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f1c0875-e872-43cd-961e-d9ad851e2c27 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:09:08.440884 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f1c0875-e872-43cd-961e-d9ad851e2c27 None admin] PATH_INFO: `/v3/projects` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:09:08.441563 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f1c0875-e872-43cd-961e-d9ad851e2c27 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 03:09:08.441881 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f1c0875-e872-43cd-961e-d9ad851e2c27 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:09:08.443340 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f1c0875-e872-43cd-961e-d9ad851e2c27 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:09:08.460440 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 519/1043] 10.222.0.18 () {56 vars in 1148 bytes} [Tue Sep 13 03:09:08 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 03:09:08.461447 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f1c0875-e872-43cd-961e-d9ad851e2c27 None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:09:08.464913 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:09:08.483814 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:09:08.517708 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52dfd2de-0f06-4dd3-a3ef-0ff36117aa97 None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:09:08.553429 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa7ff185-8902-4cb6-b553-93237e29eeff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:09:08.554312 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa7ff185-8902-4cb6-b553-93237e29eeff None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:09:08.554569 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa7ff185-8902-4cb6-b553-93237e29eeff None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:09:08.554869 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa7ff185-8902-4cb6-b553-93237e29eeff None admin] PATH_INFO: `/v3/services` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:09:08.555468 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa7ff185-8902-4cb6-b553-93237e29eeff None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:09:08.556820 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa7ff185-8902-4cb6-b553-93237e29eeff None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:09:08.563493 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 525/1044] 10.222.0.18 () {56 vars in 1087 bytes} [Tue Sep 13 03:09:08 2022] GET /identity/v3/services => generated 1957 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 13 03:09:08.569464 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f1c0875-e872-43cd-961e-d9ad851e2c27 None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:09:08.608709 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d4bf20b-8e4a-4f6f-b97b-dffdc4ab6f6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:09:08.609321 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d4bf20b-8e4a-4f6f-b97b-dffdc4ab6f6f None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:09:08.609596 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d4bf20b-8e4a-4f6f-b97b-dffdc4ab6f6f None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:09:08.609865 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d4bf20b-8e4a-4f6f-b97b-dffdc4ab6f6f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:09:08.610588 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d4bf20b-8e4a-4f6f-b97b-dffdc4ab6f6f None admin] RBAC: Adding query filter params (role.id=57ec3ac460b749a5910aea16ad1d643e, scope.project.id=b9d94f6a68f7470d9494f112175620dc, user.id=c7377f5bee4a49959182e1bd34bc97a4) {{(pid=81399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 03:09:08.610924 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d4bf20b-8e4a-4f6f-b97b-dffdc4ab6f6f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:09:08.612380 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d4bf20b-8e4a-4f6f-b97b-dffdc4ab6f6f None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:09:08.626758 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 520/1045] 10.222.0.18 () {56 vars in 1374 bytes} [Tue Sep 13 03:09:08 2022] GET /identity/v3/role_assignments?user.id=c7377f5bee4a49959182e1bd34bc97a4&scope.project.id=b9d94f6a68f7470d9494f112175620dc&role.id=57ec3ac460b749a5910aea16ad1d643e => generated 259 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 03:09:09.596029 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe858d2-1949-46da-a412-eba8d2b8dc06 None None] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:09:09.596551 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe858d2-1949-46da-a412-eba8d2b8dc06 None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:09:09.596551 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe858d2-1949-46da-a412-eba8d2b8dc06 None None] PATH_INFO: `/` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:09:09.598051 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 526/1046] 10.222.0.18 () {54 vars in 927 bytes} [Tue Sep 13 03:09:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 03:09:09.603186 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a91770b3-eb49-4e33-b0e3-1e6d1378b8f9 None None] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:09:09.603558 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a91770b3-eb49-4e33-b0e3-1e6d1378b8f9 None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:09:09.603718 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a91770b3-eb49-4e33-b0e3-1e6d1378b8f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:09:09.622167 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.auth.core [None req-a91770b3-eb49-4e33-b0e3-1e6d1378b8f9 None None] MFA Rules not processed for user `49f280ab086b4610855c4ffa378517d9`. Rule list: `[]` (Enabled: `True`). {{(pid=81399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 03:09:09.623928 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-a91770b3-eb49-4e33-b0e3-1e6d1378b8f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:09:09.683984 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:09:09.691393 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 521/1047] 10.222.0.18 () {58 vars in 1024 bytes} [Tue Sep 13 03:09:09 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 13 03:09:09.963982 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fe858d2-1949-46da-a412-eba8d2b8dc06 None None] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:09:10.073844 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fe858d2-1949-46da-a412-eba8d2b8dc06 None None] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:09:10.102944 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:09:10.110081 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69c76b0f-fccb-4508-8b06-60eaa24743b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:09:10.110736 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c76b0f-fccb-4508-8b06-60eaa24743b6 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:09:10.111012 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c76b0f-fccb-4508-8b06-60eaa24743b6 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:09:10.111275 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c76b0f-fccb-4508-8b06-60eaa24743b6 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:09:10.121890 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69c76b0f-fccb-4508-8b06-60eaa24743b6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 03:09:10.126680 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application [None req-69c76b0f-fccb-4508-8b06-60eaa24743b6 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 13 03:09:10.126680 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 03:09:10.126680 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 03:09:10.126680 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 03:09:10.126680 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 03:09:10.126680 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 03:09:10.126680 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 03:09:10.126680 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 03:09:10.126680 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 03:09:10.126680 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 03:09:10.126680 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 03:09:10.126680 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 03:09:10.126680 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 03:09:10.126680 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 03:09:10.126680 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 03:09:10.126680 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 03:09:10.127637 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 03:09:10.127637 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 03:09:10.127637 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 03:09:10.127637 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 03:09:10.127637 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 03:09:10.127637 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 03:09:10.127637 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 03:09:10.127637 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 03:09:10.127637 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 03:09:10.127637 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 03:09:10.127637 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 03:09:10.127637 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application with Lock( Sep 13 03:09:10.127637 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 03:09:10.127637 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._enter() Sep 13 03:09:10.127637 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 03:09:10.127637 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 03:09:10.128544 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 03:09:10.128544 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.creator() Sep 13 03:09:10.128544 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 03:09:10.128544 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application created_value = creator( Sep 13 03:09:10.128544 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 03:09:10.128544 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 03:09:10.128544 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 03:09:10.128544 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 03:09:10.128544 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 03:09:10.128544 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 03:09:10.128544 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 13 03:09:10.128544 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Sep 13 03:09:10.129919 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 527/1048] 10.222.0.18 () {56 vars in 1096 bytes} [Tue Sep 13 03:09:09 2022] GET /identity/v3/roles/admin => generated 83 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 03:09:10.133680 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a91770b3-eb49-4e33-b0e3-1e6d1378b8f9 None None] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:09:10.234403 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a91770b3-eb49-4e33-b0e3-1e6d1378b8f9 None None] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:09:10.268507 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4af0f22f-3378-439a-8519-c264e85f838b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:09:10.269122 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4af0f22f-3378-439a-8519-c264e85f838b None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:09:10.269400 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4af0f22f-3378-439a-8519-c264e85f838b None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:09:10.269671 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4af0f22f-3378-439a-8519-c264e85f838b None admin] PATH_INFO: `/v3/roles` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:09:10.270327 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4af0f22f-3378-439a-8519-c264e85f838b None admin] RBAC: Adding query filter params (name=admin) {{(pid=81399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 03:09:10.270639 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4af0f22f-3378-439a-8519-c264e85f838b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:09:10.271992 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4af0f22f-3378-439a-8519-c264e85f838b None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:09:10.285579 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 522/1049] 10.222.0.18 () {56 vars in 1099 bytes} [Tue Sep 13 03:09:10 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 03:09:10.286858 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69c76b0f-fccb-4508-8b06-60eaa24743b6 None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:09:10.388958 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69c76b0f-fccb-4508-8b06-60eaa24743b6 None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:09:10.422268 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19837519-b35e-4f7c-b3a8-08b72fed50eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:09:10.422911 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19837519-b35e-4f7c-b3a8-08b72fed50eb None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:09:10.423185 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19837519-b35e-4f7c-b3a8-08b72fed50eb None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:09:10.423449 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19837519-b35e-4f7c-b3a8-08b72fed50eb None admin] PATH_INFO: `/v3/users/2534727f8f3f4cd5b92c726c21c9b4e4` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:09:10.436862 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19837519-b35e-4f7c-b3a8-08b72fed50eb None admin] RBAC: Authorizing `identity:get_user(user_id=2534727f8f3f4cd5b92c726c21c9b4e4)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:09:10.438296 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19837519-b35e-4f7c-b3a8-08b72fed50eb None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:09:10.440124 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 528/1050] 10.222.0.18 () {56 vars in 1177 bytes} [Tue Sep 13 03:09:10 2022] GET /identity/v3/users/2534727f8f3f4cd5b92c726c21c9b4e4 => generated 285 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 03:09:10.445593 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4af0f22f-3378-439a-8519-c264e85f838b None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:09:10.503903 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:09:10.547772 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4af0f22f-3378-439a-8519-c264e85f838b None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:09:10.580603 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6efdd538-9c49-4972-8d10-8964c05b0758 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:09:10.581204 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6efdd538-9c49-4972-8d10-8964c05b0758 None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:09:10.581474 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6efdd538-9c49-4972-8d10-8964c05b0758 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:09:10.581738 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6efdd538-9c49-4972-8d10-8964c05b0758 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:09:10.652313 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6efdd538-9c49-4972-8d10-8964c05b0758 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:09:10.654454 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6efdd538-9c49-4972-8d10-8964c05b0758 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:09:10.715127 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:09:10.725551 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-288593d8-3d1a-422f-b89a-1caa64be82cd None None] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:09:10.726027 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-288593d8-3d1a-422f-b89a-1caa64be82cd None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:09:10.726119 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-288593d8-3d1a-422f-b89a-1caa64be82cd None None] PATH_INFO: `/` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:09:10.727701 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 529/1051] 10.222.0.18 () {54 vars in 927 bytes} [Tue Sep 13 03:09:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 13 03:09:10.732294 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39641291-2837-4324-928b-f9bbaebc409e None None] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:09:10.732572 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39641291-2837-4324-928b-f9bbaebc409e None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:09:10.732844 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39641291-2837-4324-928b-f9bbaebc409e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:09:10.751476 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.auth.core [None req-39641291-2837-4324-928b-f9bbaebc409e None None] MFA Rules not processed for user `49f280ab086b4610855c4ffa378517d9`. Rule list: `[]` (Enabled: `True`). {{(pid=81400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 03:09:10.752656 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-39641291-2837-4324-928b-f9bbaebc409e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:09:10.761279 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 523/1052] 10.222.0.18 () {58 vars in 1282 bytes} [Tue Sep 13 03:09:10 2022] GET /identity/v3/auth/tokens => generated 1969 bytes in 316 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 13 03:09:10.767345 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6efdd538-9c49-4972-8d10-8964c05b0758 None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:09:10.799467 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:09:10.821218 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 530/1053] 10.222.0.18 () {58 vars in 1024 bytes} [Tue Sep 13 03:09:10 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 13 03:09:10.822511 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8df1d4e-86c3-4005-a7b6-9877da1d9e5f None None] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:09:10.822830 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8df1d4e-86c3-4005-a7b6-9877da1d9e5f None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:09:10.823165 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8df1d4e-86c3-4005-a7b6-9877da1d9e5f None None] PATH_INFO: `/` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:09:10.824539 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 531/1054] 10.222.0.18 () {54 vars in 927 bytes} [Tue Sep 13 03:09:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 03:09:10.830212 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43c66a14-2f54-4ec1-ac29-90c406f8cfe2 None None] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:09:10.830494 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43c66a14-2f54-4ec1-ac29-90c406f8cfe2 None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:09:10.830795 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43c66a14-2f54-4ec1-ac29-90c406f8cfe2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:09:10.851028 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.auth.core [None req-43c66a14-2f54-4ec1-ac29-90c406f8cfe2 None None] MFA Rules not processed for user `49f280ab086b4610855c4ffa378517d9`. Rule list: `[]` (Enabled: `True`). {{(pid=81400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 03:09:10.852202 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-43c66a14-2f54-4ec1-ac29-90c406f8cfe2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:09:10.869316 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6efdd538-9c49-4972-8d10-8964c05b0758 None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:09:10.906245 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eae7323-3e5a-4cea-b247-e09c58cb9efc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:09:10.906907 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eae7323-3e5a-4cea-b247-e09c58cb9efc None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:09:10.907191 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eae7323-3e5a-4cea-b247-e09c58cb9efc None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:09:10.907464 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eae7323-3e5a-4cea-b247-e09c58cb9efc None admin] PATH_INFO: `/v3/projects/dc806d4e65dc49168d7393b2a0663d93` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:09:10.908570 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eae7323-3e5a-4cea-b247-e09c58cb9efc None admin] RBAC: Authorizing `identity:get_project(project_id=dc806d4e65dc49168d7393b2a0663d93)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:09:10.909995 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eae7323-3e5a-4cea-b247-e09c58cb9efc None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:09:10.911669 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 524/1055] 10.222.0.18 () {56 vars in 1186 bytes} [Tue Sep 13 03:09:10 2022] GET /identity/v3/projects/dc806d4e65dc49168d7393b2a0663d93 => generated 311 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 03:09:10.917654 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eae7323-3e5a-4cea-b247-e09c58cb9efc None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:09:10.929874 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 532/1056] 10.222.0.18 () {58 vars in 1024 bytes} [Tue Sep 13 03:09:10 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 13 03:09:10.946049 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:09:11.019201 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eae7323-3e5a-4cea-b247-e09c58cb9efc None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:09:11.052064 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a9b00a6-a42e-4e48-865e-5b455045c55e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:09:11.052661 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a9b00a6-a42e-4e48-865e-5b455045c55e None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:09:11.052941 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a9b00a6-a42e-4e48-865e-5b455045c55e None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:09:11.053213 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a9b00a6-a42e-4e48-865e-5b455045c55e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:09:11.054185 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a9b00a6-a42e-4e48-865e-5b455045c55e None admin] RBAC: Adding query filter params (role.id=425d4b481c7846e0bc07e7e805d5b15f, scope.project.id=dc806d4e65dc49168d7393b2a0663d93, user.id=2534727f8f3f4cd5b92c726c21c9b4e4) {{(pid=81399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 03:09:11.054506 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a9b00a6-a42e-4e48-865e-5b455045c55e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:09:11.055934 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a9b00a6-a42e-4e48-865e-5b455045c55e None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:09:11.063469 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 525/1057] 10.222.0.18 () {56 vars in 1374 bytes} [Tue Sep 13 03:09:10 2022] GET /identity/v3/role_assignments?user.id=2534727f8f3f4cd5b92c726c21c9b4e4&scope.project.id=dc806d4e65dc49168d7393b2a0663d93&role.id=425d4b481c7846e0bc07e7e805d5b15f => generated 259 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 03:09:11.089461 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43c66a14-2f54-4ec1-ac29-90c406f8cfe2 None None] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:09:11.197858 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43c66a14-2f54-4ec1-ac29-90c406f8cfe2 None None] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:09:11.202112 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a9b00a6-a42e-4e48-865e-5b455045c55e None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:09:11.211135 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:09:11.240834 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5330666-760b-4ef0-80a5-0c5fd7820bff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:09:11.241459 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5330666-760b-4ef0-80a5-0c5fd7820bff None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:09:11.241729 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5330666-760b-4ef0-80a5-0c5fd7820bff None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:09:11.241989 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5330666-760b-4ef0-80a5-0c5fd7820bff None admin] PATH_INFO: `/v3/services/image` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:09:11.242555 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5330666-760b-4ef0-80a5-0c5fd7820bff None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:09:11.243945 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5330666-760b-4ef0-80a5-0c5fd7820bff None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:09:11.260391 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application [None req-e5330666-760b-4ef0-80a5-0c5fd7820bff None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Sep 13 03:09:11.260391 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 03:09:11.260391 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 13 03:09:11.260391 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 13 03:09:11.260391 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 13 03:09:11.260391 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 13 03:09:11.260391 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 13 03:09:11.260391 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 13 03:09:11.260391 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Sep 13 03:09:11.260391 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Sep 13 03:09:11.260391 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 13 03:09:11.260391 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Sep 13 03:09:11.260391 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 03:09:11.260391 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 03:09:11.260391 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 03:09:11.260391 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 03:09:11.260391 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 03:09:11.261465 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 03:09:11.261465 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 03:09:11.261465 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 03:09:11.261465 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 03:09:11.261465 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 03:09:11.261465 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 03:09:11.261465 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 13 03:09:11.261465 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 13 03:09:11.261465 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 13 03:09:11.261465 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 13 03:09:11.261465 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 03:09:11.261465 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 03:09:11.261465 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 03:09:11.261465 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 03:09:11.261465 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 03:09:11.261465 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 03:09:11.262332 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 03:09:11.262332 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application with Lock( Sep 13 03:09:11.262332 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 03:09:11.262332 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._enter() Sep 13 03:09:11.262332 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 03:09:11.262332 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 03:09:11.262332 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 03:09:11.262332 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.creator() Sep 13 03:09:11.262332 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 03:09:11.262332 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application created_value = creator( Sep 13 03:09:11.262332 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 13 03:09:11.262332 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 13 03:09:11.262332 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Sep 13 03:09:11.262332 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Sep 13 03:09:11.262332 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 533/1058] 10.222.0.18 () {56 vars in 1105 bytes} [Tue Sep 13 03:09:11 2022] GET /identity/v3/services/image => generated 86 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 03:09:11.267541 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5330666-760b-4ef0-80a5-0c5fd7820bff None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:09:11.303702 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:09:11.313429 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a9b00a6-a42e-4e48-865e-5b455045c55e None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:09:11.348127 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d88a294-ca7b-4607-8e9e-4968c085c6a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:09:11.348754 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d88a294-ca7b-4607-8e9e-4968c085c6a1 None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:09:11.349030 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d88a294-ca7b-4607-8e9e-4968c085c6a1 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:09:11.349300 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d88a294-ca7b-4607-8e9e-4968c085c6a1 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:09:11.360588 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d88a294-ca7b-4607-8e9e-4968c085c6a1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 03:09:11.365376 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application [None req-0d88a294-ca7b-4607-8e9e-4968c085c6a1 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 13 03:09:11.365376 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 03:09:11.365376 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 03:09:11.365376 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 03:09:11.365376 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 03:09:11.365376 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 03:09:11.365376 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 03:09:11.365376 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 03:09:11.365376 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 03:09:11.365376 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 03:09:11.365376 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 03:09:11.365376 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 03:09:11.365376 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 03:09:11.365376 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 03:09:11.365376 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 03:09:11.365376 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 03:09:11.366336 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 03:09:11.366336 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 03:09:11.366336 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 03:09:11.366336 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 03:09:11.366336 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 03:09:11.366336 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 03:09:11.366336 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 03:09:11.366336 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 03:09:11.366336 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 03:09:11.366336 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 03:09:11.366336 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 03:09:11.366336 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application with Lock( Sep 13 03:09:11.366336 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 03:09:11.366336 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._enter() Sep 13 03:09:11.366336 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 03:09:11.366336 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 03:09:11.367550 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 03:09:11.367550 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.creator() Sep 13 03:09:11.367550 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 03:09:11.367550 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application created_value = creator( Sep 13 03:09:11.367550 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 03:09:11.367550 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 03:09:11.367550 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 03:09:11.367550 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 03:09:11.367550 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 03:09:11.367550 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 03:09:11.367550 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 13 03:09:11.367550 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Sep 13 03:09:11.367550 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 526/1059] 10.222.0.18 () {56 vars in 1120 bytes} [Tue Sep 13 03:09:11 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 13 03:09:11.372221 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d88a294-ca7b-4607-8e9e-4968c085c6a1 None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:09:11.378256 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5330666-760b-4ef0-80a5-0c5fd7820bff None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:09:11.414832 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff7fc522-76e0-4943-bf2b-f858b03f1f3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:09:11.415434 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff7fc522-76e0-4943-bf2b-f858b03f1f3e None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:09:11.415708 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff7fc522-76e0-4943-bf2b-f858b03f1f3e None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:09:11.416181 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff7fc522-76e0-4943-bf2b-f858b03f1f3e None admin] PATH_INFO: `/v3/services` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:09:11.416608 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff7fc522-76e0-4943-bf2b-f858b03f1f3e None admin] RBAC: Adding query filter params (name=image) {{(pid=81400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 03:09:11.416904 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff7fc522-76e0-4943-bf2b-f858b03f1f3e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:09:11.418417 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff7fc522-76e0-4943-bf2b-f858b03f1f3e None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:09:11.426790 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 534/1060] 10.222.0.18 () {56 vars in 1108 bytes} [Tue Sep 13 03:09:11 2022] GET /identity/v3/services?name=image => generated 122 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 03:09:11.429693 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff7fc522-76e0-4943-bf2b-f858b03f1f3e None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:09:11.470780 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:09:11.483023 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d88a294-ca7b-4607-8e9e-4968c085c6a1 None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:09:11.517679 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eabcffa-49f0-4de7-9df9-6ef2bdbd44d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:09:11.518401 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eabcffa-49f0-4de7-9df9-6ef2bdbd44d0 None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:09:11.518703 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eabcffa-49f0-4de7-9df9-6ef2bdbd44d0 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:09:11.518969 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eabcffa-49f0-4de7-9df9-6ef2bdbd44d0 None admin] PATH_INFO: `/v3/roles` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:09:11.519616 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eabcffa-49f0-4de7-9df9-6ef2bdbd44d0 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 03:09:11.519909 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eabcffa-49f0-4de7-9df9-6ef2bdbd44d0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:09:11.521375 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eabcffa-49f0-4de7-9df9-6ef2bdbd44d0 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:09:11.537977 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 527/1061] 10.222.0.18 () {56 vars in 1115 bytes} [Tue Sep 13 03:09:11 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 03:09:11.539099 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eabcffa-49f0-4de7-9df9-6ef2bdbd44d0 None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:09:11.542264 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff7fc522-76e0-4943-bf2b-f858b03f1f3e None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:09:11.555869 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:09:11.578756 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7cfea77-c6a3-4a8c-9c51-4e6647b1ac5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:09:11.579367 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7cfea77-c6a3-4a8c-9c51-4e6647b1ac5d None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:09:11.579645 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7cfea77-c6a3-4a8c-9c51-4e6647b1ac5d None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:09:11.579919 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7cfea77-c6a3-4a8c-9c51-4e6647b1ac5d None admin] PATH_INFO: `/v3/services` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:09:11.580544 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7cfea77-c6a3-4a8c-9c51-4e6647b1ac5d None admin] RBAC: Adding query filter params (type=image) {{(pid=81400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 03:09:11.580843 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7cfea77-c6a3-4a8c-9c51-4e6647b1ac5d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:09:11.582209 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7cfea77-c6a3-4a8c-9c51-4e6647b1ac5d None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:09:11.593197 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 535/1062] 10.222.0.18 () {56 vars in 1108 bytes} [Tue Sep 13 03:09:11 2022] GET /identity/v3/services?type=image => generated 350 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 03:09:11.593978 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7cfea77-c6a3-4a8c-9c51-4e6647b1ac5d None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:09:11.645494 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eabcffa-49f0-4de7-9df9-6ef2bdbd44d0 None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:09:11.682094 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1720df3c-a01f-4b13-a103-91bb7cdf0434 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:09:11.682753 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1720df3c-a01f-4b13-a103-91bb7cdf0434 None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:09:11.683112 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1720df3c-a01f-4b13-a103-91bb7cdf0434 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:09:11.683290 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1720df3c-a01f-4b13-a103-91bb7cdf0434 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:09:11.694469 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1720df3c-a01f-4b13-a103-91bb7cdf0434 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:09:11.695919 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1720df3c-a01f-4b13-a103-91bb7cdf0434 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:09:11.704337 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7cfea77-c6a3-4a8c-9c51-4e6647b1ac5d None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:09:11.706023 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:09:11.710313 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application [None req-1720df3c-a01f-4b13-a103-91bb7cdf0434 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 03:09:11.710313 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 03:09:11.710313 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 03:09:11.710313 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 03:09:11.710313 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 03:09:11.710313 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 03:09:11.710313 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 03:09:11.710313 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 03:09:11.710313 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 03:09:11.710313 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 03:09:11.710313 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 03:09:11.710313 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 03:09:11.710313 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 13 03:09:11.710313 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 13 03:09:11.710313 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 13 03:09:11.710313 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 13 03:09:11.711362 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 03:09:11.711362 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 03:09:11.711362 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 03:09:11.711362 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 03:09:11.711362 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 03:09:11.711362 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 03:09:11.711362 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 03:09:11.711362 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application with Lock( Sep 13 03:09:11.711362 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 03:09:11.711362 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._enter() Sep 13 03:09:11.711362 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 03:09:11.711362 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 03:09:11.711362 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 03:09:11.711362 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.creator() Sep 13 03:09:11.711362 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 03:09:11.711362 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application created_value = creator( Sep 13 03:09:11.711362 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 13 03:09:11.712395 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 13 03:09:11.712395 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 03:09:11.712395 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Sep 13 03:09:11.717212 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 528/1063] 10.222.0.18 () {56 vars in 1108 bytes} [Tue Sep 13 03:09:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 03:09:11.717894 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1720df3c-a01f-4b13-a103-91bb7cdf0434 None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:09:11.745261 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba47ed72-7028-4a98-b639-0632cf5e05ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:09:11.745939 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba47ed72-7028-4a98-b639-0632cf5e05ec None admin] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:09:11.746136 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba47ed72-7028-4a98-b639-0632cf5e05ec None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:09:11.746394 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba47ed72-7028-4a98-b639-0632cf5e05ec None admin] PATH_INFO: `/v3/endpoints` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:09:11.747079 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba47ed72-7028-4a98-b639-0632cf5e05ec None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:09:11.748467 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba47ed72-7028-4a98-b639-0632cf5e05ec None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:09:11.771267 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 536/1064] 10.222.0.18 () {60 vars in 1144 bytes} [Tue Sep 13 03:09:11 2022] POST /identity/v3/endpoints => generated 327 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 13 03:09:11.791016 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:09:11.832325 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1720df3c-a01f-4b13-a103-91bb7cdf0434 None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:09:11.875534 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f3d4995-71f9-4281-a2b2-668b243dc95c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:09:11.876158 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f3d4995-71f9-4281-a2b2-668b243dc95c None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:09:11.876436 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f3d4995-71f9-4281-a2b2-668b243dc95c None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:09:11.876712 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f3d4995-71f9-4281-a2b2-668b243dc95c None admin] PATH_INFO: `/v3/domains` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:09:11.877355 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f3d4995-71f9-4281-a2b2-668b243dc95c None admin] RBAC: Adding query filter params (name=Default) {{(pid=81399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 03:09:11.877658 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f3d4995-71f9-4281-a2b2-668b243dc95c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:09:11.879077 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f3d4995-71f9-4281-a2b2-668b243dc95c None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:09:11.898127 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 529/1065] 10.222.0.18 () {56 vars in 1109 bytes} [Tue Sep 13 03:09:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 03:09:11.899800 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba47ed72-7028-4a98-b639-0632cf5e05ec None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:09:12.016528 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba47ed72-7028-4a98-b639-0632cf5e05ec None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:09:12.051487 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59054f78-9f66-42db-9d8d-32dcfd53d741 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:09:12.052111 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59054f78-9f66-42db-9d8d-32dcfd53d741 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:09:12.052388 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59054f78-9f66-42db-9d8d-32dcfd53d741 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:09:12.052658 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59054f78-9f66-42db-9d8d-32dcfd53d741 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:09:12.061878 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59054f78-9f66-42db-9d8d-32dcfd53d741 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:09:12.063359 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59054f78-9f66-42db-9d8d-32dcfd53d741 None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:09:12.079129 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application [None req-59054f78-9f66-42db-9d8d-32dcfd53d741 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 13 03:09:12.079129 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 03:09:12.079129 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 03:09:12.079129 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 03:09:12.079129 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 03:09:12.079129 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 03:09:12.079129 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 03:09:12.079129 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 03:09:12.079129 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 03:09:12.079129 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 03:09:12.079129 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 03:09:12.079129 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 03:09:12.079129 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 13 03:09:12.079129 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 13 03:09:12.079129 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 13 03:09:12.079129 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 13 03:09:12.080110 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 03:09:12.080110 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 03:09:12.080110 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 13 03:09:12.080110 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 13 03:09:12.080110 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 13 03:09:12.080110 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 13 03:09:12.080110 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 03:09:12.080110 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 03:09:12.080110 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 03:09:12.080110 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 03:09:12.080110 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 03:09:12.080110 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application with Lock( Sep 13 03:09:12.080110 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 03:09:12.080110 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._enter() Sep 13 03:09:12.080110 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 03:09:12.080110 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 03:09:12.080110 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 03:09:12.080110 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.creator() Sep 13 03:09:12.081117 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 03:09:12.081117 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application created_value = creator( Sep 13 03:09:12.081117 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 13 03:09:12.081117 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 13 03:09:12.081117 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 13 03:09:12.081117 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 13 03:09:12.081117 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 13 03:09:12.081117 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 13 03:09:12.081117 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 13 03:09:12.081117 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Sep 13 03:09:12.081117 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 537/1066] 10.222.0.18 () {56 vars in 1093 bytes} [Tue Sep 13 03:09:11 2022] GET /identity/v3/users/nova => generated 82 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 03:09:12.087543 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f3d4995-71f9-4281-a2b2-668b243dc95c None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:09:12.187681 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f3d4995-71f9-4281-a2b2-668b243dc95c None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:09:12.209686 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:09:12.222425 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a39070d6-1039-4cc8-b303-1d77b4f74690 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:09:12.223058 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a39070d6-1039-4cc8-b303-1d77b4f74690 None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:09:12.223340 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a39070d6-1039-4cc8-b303-1d77b4f74690 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:09:12.223608 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a39070d6-1039-4cc8-b303-1d77b4f74690 None admin] PATH_INFO: `/v3/users` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:09:12.224320 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a39070d6-1039-4cc8-b303-1d77b4f74690 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 03:09:12.224620 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a39070d6-1039-4cc8-b303-1d77b4f74690 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:09:12.226027 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a39070d6-1039-4cc8-b303-1d77b4f74690 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:09:12.243561 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 530/1067] 10.222.0.18 () {56 vars in 1133 bytes} [Tue Sep 13 03:09:12 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 03:09:12.249108 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59054f78-9f66-42db-9d8d-32dcfd53d741 None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:09:12.268562 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:09:12.359463 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59054f78-9f66-42db-9d8d-32dcfd53d741 None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:09:12.392300 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4045610d-e182-4f45-81dd-e5bf4cd2e389 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:09:12.392920 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4045610d-e182-4f45-81dd-e5bf4cd2e389 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:09:12.393196 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4045610d-e182-4f45-81dd-e5bf4cd2e389 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:09:12.393465 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4045610d-e182-4f45-81dd-e5bf4cd2e389 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:09:12.404354 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4045610d-e182-4f45-81dd-e5bf4cd2e389 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:09:12.405784 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4045610d-e182-4f45-81dd-e5bf4cd2e389 None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:09:12.420175 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application [None req-4045610d-e182-4f45-81dd-e5bf4cd2e389 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 03:09:12.420175 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 03:09:12.420175 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 03:09:12.420175 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 03:09:12.420175 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 03:09:12.420175 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 03:09:12.420175 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 03:09:12.420175 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 03:09:12.420175 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 03:09:12.420175 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 03:09:12.420175 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 03:09:12.420175 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 03:09:12.420175 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 13 03:09:12.420175 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 13 03:09:12.420175 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 13 03:09:12.420175 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 13 03:09:12.421268 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 03:09:12.421268 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 03:09:12.421268 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 03:09:12.421268 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 03:09:12.421268 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 03:09:12.421268 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 03:09:12.421268 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 03:09:12.421268 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application with Lock( Sep 13 03:09:12.421268 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 03:09:12.421268 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._enter() Sep 13 03:09:12.421268 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 03:09:12.421268 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 03:09:12.421268 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 03:09:12.421268 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.creator() Sep 13 03:09:12.421268 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 03:09:12.421268 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application created_value = creator( Sep 13 03:09:12.421268 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 13 03:09:12.422313 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 13 03:09:12.422313 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 03:09:12.422313 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Sep 13 03:09:12.422313 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 538/1068] 10.222.0.18 () {56 vars in 1108 bytes} [Tue Sep 13 03:09:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 03:09:12.426918 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a39070d6-1039-4cc8-b303-1d77b4f74690 None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:09:12.524830 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a39070d6-1039-4cc8-b303-1d77b4f74690 None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:09:12.560507 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f696cc95-822c-4017-811b-32b55409da78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:09:12.561120 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f696cc95-822c-4017-811b-32b55409da78 None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:09:12.561399 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f696cc95-822c-4017-811b-32b55409da78 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:09:12.561670 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f696cc95-822c-4017-811b-32b55409da78 None admin] PATH_INFO: `/v3/domains` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:09:12.562305 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f696cc95-822c-4017-811b-32b55409da78 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 03:09:12.562620 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f696cc95-822c-4017-811b-32b55409da78 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:09:12.563990 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f696cc95-822c-4017-811b-32b55409da78 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:09:12.580367 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 531/1069] 10.222.0.18 () {56 vars in 1109 bytes} [Tue Sep 13 03:09:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 03:09:12.581495 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4045610d-e182-4f45-81dd-e5bf4cd2e389 None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:09:12.650922 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:09:12.683556 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4045610d-e182-4f45-81dd-e5bf4cd2e389 None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:09:12.719563 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae8347b4-3092-4664-84ed-922f7563f05c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:09:12.720493 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae8347b4-3092-4664-84ed-922f7563f05c None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:09:12.720777 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae8347b4-3092-4664-84ed-922f7563f05c None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:09:12.721023 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae8347b4-3092-4664-84ed-922f7563f05c None admin] PATH_INFO: `/v3/projects/service` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:09:12.728273 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae8347b4-3092-4664-84ed-922f7563f05c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:09:12.729728 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae8347b4-3092-4664-84ed-922f7563f05c None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:09:12.741027 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application [None req-ae8347b4-3092-4664-84ed-922f7563f05c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 13 03:09:12.741027 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 03:09:12.741027 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 03:09:12.741027 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 03:09:12.741027 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 03:09:12.741027 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 03:09:12.741027 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 03:09:12.741027 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 03:09:12.741027 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 03:09:12.741027 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 03:09:12.741027 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 03:09:12.741027 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 03:09:12.741027 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 13 03:09:12.741027 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 13 03:09:12.741027 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 13 03:09:12.741920 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 13 03:09:12.741920 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 03:09:12.741920 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 03:09:12.741920 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 03:09:12.741920 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 03:09:12.741920 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 03:09:12.741920 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 03:09:12.741920 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 03:09:12.741920 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application with Lock( Sep 13 03:09:12.741920 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 03:09:12.741920 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._enter() Sep 13 03:09:12.741920 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 03:09:12.741920 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 03:09:12.741920 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 03:09:12.741920 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.creator() Sep 13 03:09:12.741920 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 03:09:12.741920 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application created_value = creator( Sep 13 03:09:12.742894 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 13 03:09:12.742894 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 13 03:09:12.742894 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 13 03:09:12.742894 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 13 03:09:12.742894 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 13 03:09:12.742894 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 13 03:09:12.742894 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 13 03:09:12.742894 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Sep 13 03:09:12.742894 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 539/1070] 10.222.0.18 () {56 vars in 1111 bytes} [Tue Sep 13 03:09:12 2022] GET /identity/v3/projects/service => generated 88 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 03:09:12.749159 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f696cc95-822c-4017-811b-32b55409da78 None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:09:12.791646 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:09:12.849826 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f696cc95-822c-4017-811b-32b55409da78 None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:09:12.882079 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7fc5e1e-0b95-4353-b3e8-8c07991c2942 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:09:12.882709 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7fc5e1e-0b95-4353-b3e8-8c07991c2942 None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:09:12.882988 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7fc5e1e-0b95-4353-b3e8-8c07991c2942 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:09:12.883256 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7fc5e1e-0b95-4353-b3e8-8c07991c2942 None admin] PATH_INFO: `/v3/projects` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:09:12.883935 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7fc5e1e-0b95-4353-b3e8-8c07991c2942 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 03:09:12.884237 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7fc5e1e-0b95-4353-b3e8-8c07991c2942 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:09:12.885658 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7fc5e1e-0b95-4353-b3e8-8c07991c2942 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:09:12.902683 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 532/1071] 10.222.0.18 () {56 vars in 1148 bytes} [Tue Sep 13 03:09:12 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 03:09:12.904359 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae8347b4-3092-4664-84ed-922f7563f05c None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:09:13.004233 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae8347b4-3092-4664-84ed-922f7563f05c None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:09:13.035403 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:09:13.042045 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c59b04e-4a19-41d2-9b65-438ca44579a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:09:13.042724 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c59b04e-4a19-41d2-9b65-438ca44579a5 None admin] REQUEST_METHOD: `PUT` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:09:13.043002 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c59b04e-4a19-41d2-9b65-438ca44579a5 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:09:13.043271 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c59b04e-4a19-41d2-9b65-438ca44579a5 None admin] PATH_INFO: `/v3/projects/b9d94f6a68f7470d9494f112175620dc/users/c7377f5bee4a49959182e1bd34bc97a4/roles/57ec3ac460b749a5910aea16ad1d643e` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:09:13.045410 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c59b04e-4a19-41d2-9b65-438ca44579a5 None admin] RBAC: Authorizing `identity:create_grant(project_id=b9d94f6a68f7470d9494f112175620dc, user_id=c7377f5bee4a49959182e1bd34bc97a4, role_id=57ec3ac460b749a5910aea16ad1d643e)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:09:13.046886 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c59b04e-4a19-41d2-9b65-438ca44579a5 None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:09:13.074646 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 540/1072] 10.222.0.18 () {58 vars in 1439 bytes} [Tue Sep 13 03:09:12 2022] PUT /identity/v3/projects/b9d94f6a68f7470d9494f112175620dc/users/c7377f5bee4a49959182e1bd34bc97a4/roles/57ec3ac460b749a5910aea16ad1d643e => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 13 03:09:13.205296 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5253c24f-a85a-4d69-a9e8-ebe54cb66a3a None None] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:09:13.205577 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5253c24f-a85a-4d69-a9e8-ebe54cb66a3a None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:09:13.205850 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5253c24f-a85a-4d69-a9e8-ebe54cb66a3a None None] PATH_INFO: `/` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:09:13.207255 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 533/1073] 10.222.0.18 () {54 vars in 927 bytes} [Tue Sep 13 03:09:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 03:09:13.212809 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-388ae0e6-f435-482e-9b89-da0de2831579 None None] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:09:13.213088 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-388ae0e6-f435-482e-9b89-da0de2831579 None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:09:13.213362 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-388ae0e6-f435-482e-9b89-da0de2831579 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:09:13.233141 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.auth.core [None req-388ae0e6-f435-482e-9b89-da0de2831579 None None] MFA Rules not processed for user `49f280ab086b4610855c4ffa378517d9`. Rule list: `[]` (Enabled: `True`). {{(pid=81400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 03:09:13.234254 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-388ae0e6-f435-482e-9b89-da0de2831579 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:09:13.302428 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 541/1074] 10.222.0.18 () {58 vars in 1024 bytes} [Tue Sep 13 03:09:13 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 13 03:09:13.566774 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5253c24f-a85a-4d69-a9e8-ebe54cb66a3a None None] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:09:13.665652 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:09:13.671366 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5253c24f-a85a-4d69-a9e8-ebe54cb66a3a None None] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:09:13.704474 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29b0a62a-5135-4897-a744-ab910ee23e47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:09:13.705106 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b0a62a-5135-4897-a744-ab910ee23e47 None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:09:13.705383 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b0a62a-5135-4897-a744-ab910ee23e47 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:09:13.705650 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b0a62a-5135-4897-a744-ab910ee23e47 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:09:13.715890 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29b0a62a-5135-4897-a744-ab910ee23e47 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 03:09:13.721330 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application [None req-29b0a62a-5135-4897-a744-ab910ee23e47 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 13 03:09:13.721330 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 03:09:13.721330 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 03:09:13.721330 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 03:09:13.721330 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 03:09:13.721330 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 03:09:13.721330 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 03:09:13.721330 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 03:09:13.721330 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 03:09:13.721330 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 03:09:13.721330 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 03:09:13.721330 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 03:09:13.721330 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 03:09:13.721330 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 03:09:13.721330 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 03:09:13.721330 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 03:09:13.722346 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 03:09:13.722346 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 03:09:13.722346 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 03:09:13.722346 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 03:09:13.722346 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 03:09:13.722346 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 03:09:13.722346 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 03:09:13.722346 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 03:09:13.722346 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 03:09:13.722346 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 03:09:13.722346 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 03:09:13.722346 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application with Lock( Sep 13 03:09:13.722346 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 03:09:13.722346 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._enter() Sep 13 03:09:13.722346 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 03:09:13.722346 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 03:09:13.727530 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 03:09:13.727530 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.creator() Sep 13 03:09:13.727530 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 03:09:13.727530 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application created_value = creator( Sep 13 03:09:13.727530 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 03:09:13.727530 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 03:09:13.727530 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 03:09:13.727530 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 03:09:13.727530 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 03:09:13.727530 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 03:09:13.727530 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 13 03:09:13.727530 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Sep 13 03:09:13.727530 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 534/1075] 10.222.0.18 () {56 vars in 1096 bytes} [Tue Sep 13 03:09:13 2022] GET /identity/v3/roles/admin => generated 83 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 13 03:09:13.730114 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-388ae0e6-f435-482e-9b89-da0de2831579 None None] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:09:13.831126 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-388ae0e6-f435-482e-9b89-da0de2831579 None None] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:09:13.852185 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:09:13.869286 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d91a183-ab04-4d8b-ac05-892c20033ef1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:09:13.869924 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d91a183-ab04-4d8b-ac05-892c20033ef1 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:09:13.870204 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d91a183-ab04-4d8b-ac05-892c20033ef1 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:09:13.870474 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d91a183-ab04-4d8b-ac05-892c20033ef1 None admin] PATH_INFO: `/v3/roles` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:09:13.871188 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d91a183-ab04-4d8b-ac05-892c20033ef1 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 03:09:13.871496 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d91a183-ab04-4d8b-ac05-892c20033ef1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:09:13.872957 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d91a183-ab04-4d8b-ac05-892c20033ef1 None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:09:13.887619 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 542/1076] 10.222.0.18 () {56 vars in 1099 bytes} [Tue Sep 13 03:09:13 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 03:09:13.889356 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29b0a62a-5135-4897-a744-ab910ee23e47 None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:09:13.995597 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93566a39-51e4-4ad5-b135-e6744316d4ff None None] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:09:13.996320 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93566a39-51e4-4ad5-b135-e6744316d4ff None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:09:13.996320 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93566a39-51e4-4ad5-b135-e6744316d4ff None None] PATH_INFO: `/` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:09:14.001491 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 543/1077] 10.222.0.18 () {54 vars in 927 bytes} [Tue Sep 13 03:09:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 13 03:09:14.005443 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29b0a62a-5135-4897-a744-ab910ee23e47 None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:09:14.006351 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de04ba38-193b-4c63-8161-3a5c03003737 None None] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:09:14.006650 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de04ba38-193b-4c63-8161-3a5c03003737 None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:09:14.007117 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de04ba38-193b-4c63-8161-3a5c03003737 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:09:14.027230 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.auth.core [None req-de04ba38-193b-4c63-8161-3a5c03003737 None None] MFA Rules not processed for user `49f280ab086b4610855c4ffa378517d9`. Rule list: `[]` (Enabled: `True`). {{(pid=81400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 03:09:14.028418 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-de04ba38-193b-4c63-8161-3a5c03003737 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:09:14.042766 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-139aec13-72ae-48fb-8846-a1bbd87b7967 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:09:14.043401 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-139aec13-72ae-48fb-8846-a1bbd87b7967 None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:09:14.043672 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-139aec13-72ae-48fb-8846-a1bbd87b7967 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:09:14.043939 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-139aec13-72ae-48fb-8846-a1bbd87b7967 None admin] PATH_INFO: `/v3/users/2534727f8f3f4cd5b92c726c21c9b4e4` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:09:14.045084 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-139aec13-72ae-48fb-8846-a1bbd87b7967 None admin] RBAC: Authorizing `identity:get_user(user_id=2534727f8f3f4cd5b92c726c21c9b4e4)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:09:14.046506 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-139aec13-72ae-48fb-8846-a1bbd87b7967 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:09:14.048455 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 535/1078] 10.222.0.18 () {56 vars in 1177 bytes} [Tue Sep 13 03:09:13 2022] GET /identity/v3/users/2534727f8f3f4cd5b92c726c21c9b4e4 => generated 285 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 03:09:14.053740 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-139aec13-72ae-48fb-8846-a1bbd87b7967 None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:09:14.090129 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:09:14.114494 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 544/1079] 10.222.0.18 () {58 vars in 1024 bytes} [Tue Sep 13 03:09:14 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 13 03:09:14.166075 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-139aec13-72ae-48fb-8846-a1bbd87b7967 None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:09:14.199711 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9a6c218-c0cd-4555-a206-0befb910b1e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:09:14.200840 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a6c218-c0cd-4555-a206-0befb910b1e7 None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:09:14.201120 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a6c218-c0cd-4555-a206-0befb910b1e7 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:09:14.201390 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a6c218-c0cd-4555-a206-0befb910b1e7 None admin] PATH_INFO: `/v3/projects/dc806d4e65dc49168d7393b2a0663d93` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:09:14.202493 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9a6c218-c0cd-4555-a206-0befb910b1e7 None admin] RBAC: Authorizing `identity:get_project(project_id=dc806d4e65dc49168d7393b2a0663d93)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:09:14.203937 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9a6c218-c0cd-4555-a206-0befb910b1e7 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:09:14.205784 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 536/1080] 10.222.0.18 () {56 vars in 1186 bytes} [Tue Sep 13 03:09:14 2022] GET /identity/v3/projects/dc806d4e65dc49168d7393b2a0663d93 => generated 311 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 03:09:14.215475 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de04ba38-193b-4c63-8161-3a5c03003737 None None] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:09:14.320269 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de04ba38-193b-4c63-8161-3a5c03003737 None None] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:09:14.334389 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:09:14.356477 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c979b47c-169d-4829-a74a-9dc8c75c2385 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:09:14.357077 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c979b47c-169d-4829-a74a-9dc8c75c2385 None admin] REQUEST_METHOD: `PUT` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:09:14.357347 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c979b47c-169d-4829-a74a-9dc8c75c2385 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:09:14.357610 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c979b47c-169d-4829-a74a-9dc8c75c2385 None admin] PATH_INFO: `/v3/projects/dc806d4e65dc49168d7393b2a0663d93/users/2534727f8f3f4cd5b92c726c21c9b4e4/roles/425d4b481c7846e0bc07e7e805d5b15f` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:09:14.359527 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c979b47c-169d-4829-a74a-9dc8c75c2385 None admin] RBAC: Authorizing `identity:create_grant(project_id=dc806d4e65dc49168d7393b2a0663d93, user_id=2534727f8f3f4cd5b92c726c21c9b4e4, role_id=425d4b481c7846e0bc07e7e805d5b15f)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:09:14.360944 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c979b47c-169d-4829-a74a-9dc8c75c2385 None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:09:14.373985 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 545/1081] 10.222.0.18 () {58 vars in 1439 bytes} [Tue Sep 13 03:09:14 2022] PUT /identity/v3/projects/dc806d4e65dc49168d7393b2a0663d93/users/2534727f8f3f4cd5b92c726c21c9b4e4/roles/425d4b481c7846e0bc07e7e805d5b15f => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 03:09:14.383871 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9a6c218-c0cd-4555-a206-0befb910b1e7 None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:09:14.487240 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:09:14.495058 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9a6c218-c0cd-4555-a206-0befb910b1e7 None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:09:14.532106 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b85e1ff5-a91d-4501-977b-a6ae2c0b4da3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:09:14.532731 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b85e1ff5-a91d-4501-977b-a6ae2c0b4da3 None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:09:14.533006 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b85e1ff5-a91d-4501-977b-a6ae2c0b4da3 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:09:14.533272 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b85e1ff5-a91d-4501-977b-a6ae2c0b4da3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:09:14.604945 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b85e1ff5-a91d-4501-977b-a6ae2c0b4da3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:09:14.606388 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b85e1ff5-a91d-4501-977b-a6ae2c0b4da3 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:09:14.697490 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:09:14.706330 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 537/1082] 10.222.0.18 () {58 vars in 1282 bytes} [Tue Sep 13 03:09:14 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 323 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 13 03:09:14.712111 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c979b47c-169d-4829-a74a-9dc8c75c2385 None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:09:14.827717 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c979b47c-169d-4829-a74a-9dc8c75c2385 None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:09:14.863186 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72a34ef2-a9e5-4cbc-ab6c-8b547d8980de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:09:14.863821 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a34ef2-a9e5-4cbc-ab6c-8b547d8980de None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:09:14.864097 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a34ef2-a9e5-4cbc-ab6c-8b547d8980de None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:09:14.864368 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a34ef2-a9e5-4cbc-ab6c-8b547d8980de None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:09:14.875072 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72a34ef2-a9e5-4cbc-ab6c-8b547d8980de None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:09:14.876503 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72a34ef2-a9e5-4cbc-ab6c-8b547d8980de None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:09:14.891029 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application [None req-72a34ef2-a9e5-4cbc-ab6c-8b547d8980de None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 03:09:14.891029 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 03:09:14.891029 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 03:09:14.891029 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 03:09:14.891029 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 03:09:14.891029 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 03:09:14.891029 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 03:09:14.891029 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 03:09:14.891029 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 03:09:14.891029 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 03:09:14.891029 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 03:09:14.891029 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 03:09:14.891029 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 13 03:09:14.891029 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 13 03:09:14.891029 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 13 03:09:14.891029 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 13 03:09:14.892019 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 03:09:14.892019 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 03:09:14.892019 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 03:09:14.892019 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 03:09:14.892019 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 03:09:14.892019 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 03:09:14.892019 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 03:09:14.892019 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application with Lock( Sep 13 03:09:14.892019 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 03:09:14.892019 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._enter() Sep 13 03:09:14.892019 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 03:09:14.892019 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 03:09:14.892019 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 03:09:14.892019 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.creator() Sep 13 03:09:14.892019 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 03:09:14.892019 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application created_value = creator( Sep 13 03:09:14.892019 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 13 03:09:14.892924 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 13 03:09:14.892924 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 03:09:14.892924 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Sep 13 03:09:14.894932 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 546/1083] 10.222.0.18 () {56 vars in 1108 bytes} [Tue Sep 13 03:09:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 03:09:14.898227 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b85e1ff5-a91d-4501-977b-a6ae2c0b4da3 None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:09:14.996274 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b85e1ff5-a91d-4501-977b-a6ae2c0b4da3 None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:09:15.030033 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f29c7dd-c25e-45f2-9d5c-69c6f0898668 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:09:15.030688 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f29c7dd-c25e-45f2-9d5c-69c6f0898668 None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:09:15.031077 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f29c7dd-c25e-45f2-9d5c-69c6f0898668 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:09:15.031372 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f29c7dd-c25e-45f2-9d5c-69c6f0898668 None admin] PATH_INFO: `/v3/domains` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:09:15.032134 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f29c7dd-c25e-45f2-9d5c-69c6f0898668 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 03:09:15.032441 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f29c7dd-c25e-45f2-9d5c-69c6f0898668 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:09:15.033843 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f29c7dd-c25e-45f2-9d5c-69c6f0898668 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:09:15.050833 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 538/1084] 10.222.0.18 () {56 vars in 1109 bytes} [Tue Sep 13 03:09:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 03:09:15.269973 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3334faa1-aba7-4deb-8108-70bd77d3989f None None] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:09:15.270497 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3334faa1-aba7-4deb-8108-70bd77d3989f None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:09:15.270497 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3334faa1-aba7-4deb-8108-70bd77d3989f None None] PATH_INFO: `/` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:09:15.271864 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 547/1085] 10.222.0.18 () {54 vars in 927 bytes} [Tue Sep 13 03:09:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 03:09:15.277843 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daec0dbc-576e-4a4c-aae6-701ca4686c47 None None] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:09:15.278155 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daec0dbc-576e-4a4c-aae6-701ca4686c47 None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:09:15.278946 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daec0dbc-576e-4a4c-aae6-701ca4686c47 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:09:15.297943 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.auth.core [None req-daec0dbc-576e-4a4c-aae6-701ca4686c47 None None] MFA Rules not processed for user `49f280ab086b4610855c4ffa378517d9`. Rule list: `[]` (Enabled: `True`). {{(pid=81399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 03:09:15.299155 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-daec0dbc-576e-4a4c-aae6-701ca4686c47 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:09:15.354485 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:09:15.374517 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 539/1086] 10.222.0.18 () {58 vars in 1024 bytes} [Tue Sep 13 03:09:15 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 13 03:09:15.643708 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3334faa1-aba7-4deb-8108-70bd77d3989f None None] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:09:15.669178 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:09:15.744744 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3334faa1-aba7-4deb-8108-70bd77d3989f None None] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:09:15.777042 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4104e199-105b-40d8-b26a-f0af5d0fc592 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:09:15.777656 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4104e199-105b-40d8-b26a-f0af5d0fc592 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:09:15.777933 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4104e199-105b-40d8-b26a-f0af5d0fc592 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:09:15.778202 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4104e199-105b-40d8-b26a-f0af5d0fc592 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:09:15.788987 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4104e199-105b-40d8-b26a-f0af5d0fc592 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 03:09:15.793791 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application [None req-4104e199-105b-40d8-b26a-f0af5d0fc592 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 13 03:09:15.793791 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 03:09:15.793791 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 03:09:15.793791 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 03:09:15.793791 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 03:09:15.793791 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 03:09:15.793791 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 03:09:15.793791 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 03:09:15.793791 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 03:09:15.793791 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 03:09:15.793791 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 03:09:15.793791 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 03:09:15.793791 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 03:09:15.793791 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 03:09:15.793791 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 03:09:15.793791 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 03:09:15.798764 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 03:09:15.798764 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 03:09:15.798764 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 03:09:15.798764 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 03:09:15.798764 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 03:09:15.798764 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 03:09:15.798764 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 03:09:15.798764 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 03:09:15.798764 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 03:09:15.798764 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 03:09:15.798764 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 03:09:15.798764 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application with Lock( Sep 13 03:09:15.798764 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 03:09:15.798764 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._enter() Sep 13 03:09:15.798764 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 03:09:15.798764 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 03:09:15.799895 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 03:09:15.799895 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.creator() Sep 13 03:09:15.799895 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 03:09:15.799895 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application created_value = creator( Sep 13 03:09:15.799895 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 03:09:15.799895 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 03:09:15.799895 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 03:09:15.799895 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 03:09:15.799895 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 03:09:15.799895 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 03:09:15.799895 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 13 03:09:15.799895 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Sep 13 03:09:15.799895 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 548/1087] 10.222.0.18 () {56 vars in 1120 bytes} [Tue Sep 13 03:09:15 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 03:09:15.800985 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daec0dbc-576e-4a4c-aae6-701ca4686c47 None None] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:09:15.894841 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daec0dbc-576e-4a4c-aae6-701ca4686c47 None None] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:09:15.927020 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2a60bcd-aac2-402c-8573-bbeec4f0d315 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:09:15.927618 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a60bcd-aac2-402c-8573-bbeec4f0d315 None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:09:15.927894 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a60bcd-aac2-402c-8573-bbeec4f0d315 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:09:15.928162 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a60bcd-aac2-402c-8573-bbeec4f0d315 None admin] PATH_INFO: `/v3/roles` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:09:15.929082 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2a60bcd-aac2-402c-8573-bbeec4f0d315 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 03:09:15.929380 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2a60bcd-aac2-402c-8573-bbeec4f0d315 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:09:15.930868 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2a60bcd-aac2-402c-8573-bbeec4f0d315 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:09:15.939225 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 540/1088] 10.222.0.18 () {56 vars in 1115 bytes} [Tue Sep 13 03:09:15 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 03:09:15.944858 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4104e199-105b-40d8-b26a-f0af5d0fc592 None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:09:16.026869 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:09:16.042601 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4104e199-105b-40d8-b26a-f0af5d0fc592 None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:09:16.076610 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6de61b0e-dc9f-4053-b84a-90dbccd799dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:09:16.077235 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de61b0e-dc9f-4053-b84a-90dbccd799dc None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:09:16.077509 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de61b0e-dc9f-4053-b84a-90dbccd799dc None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:09:16.077778 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de61b0e-dc9f-4053-b84a-90dbccd799dc None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:09:16.088116 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6de61b0e-dc9f-4053-b84a-90dbccd799dc None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:09:16.089547 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6de61b0e-dc9f-4053-b84a-90dbccd799dc None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:09:16.103316 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application [None req-6de61b0e-dc9f-4053-b84a-90dbccd799dc None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 03:09:16.103316 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 03:09:16.103316 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 03:09:16.103316 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 03:09:16.103316 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 03:09:16.103316 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 03:09:16.103316 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 03:09:16.103316 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 03:09:16.103316 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 03:09:16.103316 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 03:09:16.103316 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 03:09:16.103316 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 03:09:16.103316 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 13 03:09:16.103316 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 13 03:09:16.103316 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 13 03:09:16.103316 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 13 03:09:16.104472 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 03:09:16.104472 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 03:09:16.104472 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 03:09:16.104472 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 03:09:16.104472 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 03:09:16.104472 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 03:09:16.104472 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 03:09:16.104472 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application with Lock( Sep 13 03:09:16.104472 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 03:09:16.104472 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._enter() Sep 13 03:09:16.104472 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 03:09:16.104472 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 03:09:16.104472 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 03:09:16.104472 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.creator() Sep 13 03:09:16.104472 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 03:09:16.104472 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application created_value = creator( Sep 13 03:09:16.104472 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 13 03:09:16.105391 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 13 03:09:16.105391 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 03:09:16.105391 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Sep 13 03:09:16.105391 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 549/1089] 10.222.0.18 () {56 vars in 1108 bytes} [Tue Sep 13 03:09:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 03:09:16.110145 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2a60bcd-aac2-402c-8573-bbeec4f0d315 None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:09:16.157913 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:09:16.208545 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2a60bcd-aac2-402c-8573-bbeec4f0d315 None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:09:16.240113 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c5324f2-43fc-421a-a732-54ffb49ff154 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:09:16.240710 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c5324f2-43fc-421a-a732-54ffb49ff154 None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:09:16.240985 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c5324f2-43fc-421a-a732-54ffb49ff154 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:09:16.241254 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c5324f2-43fc-421a-a732-54ffb49ff154 None admin] PATH_INFO: `/v3/domains` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:09:16.241881 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c5324f2-43fc-421a-a732-54ffb49ff154 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 03:09:16.242179 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c5324f2-43fc-421a-a732-54ffb49ff154 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:09:16.243560 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c5324f2-43fc-421a-a732-54ffb49ff154 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:09:16.255305 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 541/1090] 10.222.0.18 () {56 vars in 1109 bytes} [Tue Sep 13 03:09:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 03:09:16.260927 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6de61b0e-dc9f-4053-b84a-90dbccd799dc None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:09:16.358541 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6de61b0e-dc9f-4053-b84a-90dbccd799dc None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:09:16.391079 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3721c5ad-592f-4e0f-acb3-fd877e44c01a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:09:16.391696 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3721c5ad-592f-4e0f-acb3-fd877e44c01a None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:09:16.391965 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3721c5ad-592f-4e0f-acb3-fd877e44c01a None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:09:16.392228 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3721c5ad-592f-4e0f-acb3-fd877e44c01a None admin] PATH_INFO: `/v3/users/nova` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:09:16.394857 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:09:16.400659 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3721c5ad-592f-4e0f-acb3-fd877e44c01a None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:09:16.402091 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3721c5ad-592f-4e0f-acb3-fd877e44c01a None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:09:16.413094 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application [None req-3721c5ad-592f-4e0f-acb3-fd877e44c01a None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 13 03:09:16.413094 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 03:09:16.413094 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 03:09:16.413094 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 03:09:16.413094 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 03:09:16.413094 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 03:09:16.413094 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 03:09:16.413094 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 03:09:16.413094 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 03:09:16.413094 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 03:09:16.413094 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 03:09:16.413094 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 03:09:16.413094 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 13 03:09:16.413094 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 13 03:09:16.413094 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 13 03:09:16.413094 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 13 03:09:16.414279 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 03:09:16.414279 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 03:09:16.414279 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 13 03:09:16.414279 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 13 03:09:16.414279 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 13 03:09:16.414279 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 13 03:09:16.414279 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 03:09:16.414279 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 03:09:16.414279 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 03:09:16.414279 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 03:09:16.414279 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 03:09:16.414279 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application with Lock( Sep 13 03:09:16.414279 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 03:09:16.414279 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._enter() Sep 13 03:09:16.414279 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 03:09:16.414279 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 03:09:16.414279 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 03:09:16.414279 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.creator() Sep 13 03:09:16.415339 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 03:09:16.415339 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application created_value = creator( Sep 13 03:09:16.415339 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 13 03:09:16.415339 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 13 03:09:16.415339 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 13 03:09:16.415339 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 13 03:09:16.415339 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 13 03:09:16.415339 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 13 03:09:16.415339 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 13 03:09:16.415339 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Sep 13 03:09:16.415339 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 550/1091] 10.222.0.18 () {56 vars in 1093 bytes} [Tue Sep 13 03:09:16 2022] GET /identity/v3/users/nova => generated 82 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 03:09:16.420317 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c5324f2-43fc-421a-a732-54ffb49ff154 None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:09:16.519849 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c5324f2-43fc-421a-a732-54ffb49ff154 None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:09:16.528814 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:09:16.552140 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77931b32-d692-4b1a-b857-616f8eef2e60 None None] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:09:16.552415 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77931b32-d692-4b1a-b857-616f8eef2e60 None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:09:16.552681 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77931b32-d692-4b1a-b857-616f8eef2e60 None None] PATH_INFO: `/` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:09:16.553557 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1211701-6cc4-439c-9d7b-81e7b174d173 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:09:16.554290 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 551/1092] 10.222.0.18 () {54 vars in 927 bytes} [Tue Sep 13 03:09:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 13 03:09:16.554470 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1211701-6cc4-439c-9d7b-81e7b174d173 None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:09:16.554779 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1211701-6cc4-439c-9d7b-81e7b174d173 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:09:16.555052 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1211701-6cc4-439c-9d7b-81e7b174d173 None admin] PATH_INFO: `/v3/users` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:09:16.555839 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1211701-6cc4-439c-9d7b-81e7b174d173 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 03:09:16.556148 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1211701-6cc4-439c-9d7b-81e7b174d173 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:09:16.557562 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1211701-6cc4-439c-9d7b-81e7b174d173 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:09:16.559195 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed7d66a-8207-4700-b43e-88ee78b59698 None None] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:09:16.559704 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed7d66a-8207-4700-b43e-88ee78b59698 None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:09:16.560516 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed7d66a-8207-4700-b43e-88ee78b59698 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:09:16.578635 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.auth.core [None req-bed7d66a-8207-4700-b43e-88ee78b59698 None None] MFA Rules not processed for user `49f280ab086b4610855c4ffa378517d9`. Rule list: `[]` (Enabled: `True`). {{(pid=81400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 03:09:16.579806 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-bed7d66a-8207-4700-b43e-88ee78b59698 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:09:16.581550 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 542/1093] 10.222.0.18 () {56 vars in 1133 bytes} [Tue Sep 13 03:09:16 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 03:09:16.587293 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1211701-6cc4-439c-9d7b-81e7b174d173 None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:09:16.649988 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 552/1094] 10.222.0.18 () {58 vars in 1024 bytes} [Tue Sep 13 03:09:16 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 13 03:09:16.682446 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1211701-6cc4-439c-9d7b-81e7b174d173 None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:09:16.718941 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-155896fa-a51c-43d4-879f-664c7598c83d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:09:16.719814 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-155896fa-a51c-43d4-879f-664c7598c83d None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:09:16.720458 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-155896fa-a51c-43d4-879f-664c7598c83d None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:09:16.720905 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-155896fa-a51c-43d4-879f-664c7598c83d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:09:16.755293 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:09:16.791961 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-155896fa-a51c-43d4-879f-664c7598c83d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:09:16.793427 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-155896fa-a51c-43d4-879f-664c7598c83d None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:09:16.891618 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 543/1095] 10.222.0.18 () {58 vars in 1282 bytes} [Tue Sep 13 03:09:16 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 305 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Sep 13 03:09:16.897682 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bed7d66a-8207-4700-b43e-88ee78b59698 None None] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:09:16.918388 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-155896fa-a51c-43d4-879f-664c7598c83d None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:09:16.987835 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:09:17.006537 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bed7d66a-8207-4700-b43e-88ee78b59698 None None] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:09:17.022540 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-155896fa-a51c-43d4-879f-664c7598c83d None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:09:17.032418 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:09:17.041503 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-206ce39e-13b2-4ac8-882e-42666afcc1f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:09:17.042118 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-206ce39e-13b2-4ac8-882e-42666afcc1f7 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:09:17.042390 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-206ce39e-13b2-4ac8-882e-42666afcc1f7 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:09:17.042689 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-206ce39e-13b2-4ac8-882e-42666afcc1f7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:09:17.052452 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-206ce39e-13b2-4ac8-882e-42666afcc1f7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:09:17.054145 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-206ce39e-13b2-4ac8-882e-42666afcc1f7 None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:09:17.056675 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcde0bc3-7d9c-49a5-af29-8ba006843cd6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:09:17.057295 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcde0bc3-7d9c-49a5-af29-8ba006843cd6 None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:09:17.058025 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcde0bc3-7d9c-49a5-af29-8ba006843cd6 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:09:17.058302 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcde0bc3-7d9c-49a5-af29-8ba006843cd6 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:09:17.069399 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcde0bc3-7d9c-49a5-af29-8ba006843cd6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 03:09:17.070468 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application [None req-206ce39e-13b2-4ac8-882e-42666afcc1f7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 03:09:17.070468 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 03:09:17.070468 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 03:09:17.070468 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 03:09:17.070468 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 03:09:17.070468 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 03:09:17.070468 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 03:09:17.070468 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 03:09:17.070468 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 03:09:17.070468 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 03:09:17.070468 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 03:09:17.070468 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 03:09:17.070468 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 13 03:09:17.070468 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 13 03:09:17.070468 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 13 03:09:17.070468 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 13 03:09:17.071622 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 03:09:17.071622 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 03:09:17.071622 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 03:09:17.071622 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 03:09:17.071622 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 03:09:17.071622 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 03:09:17.071622 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 03:09:17.071622 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application with Lock( Sep 13 03:09:17.071622 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 03:09:17.071622 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._enter() Sep 13 03:09:17.071622 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 03:09:17.071622 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 03:09:17.071622 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 03:09:17.071622 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.creator() Sep 13 03:09:17.071622 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 03:09:17.071622 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application created_value = creator( Sep 13 03:09:17.071622 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 13 03:09:17.072550 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 13 03:09:17.072550 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 03:09:17.072550 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Sep 13 03:09:17.072550 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 553/1096] 10.222.0.18 () {56 vars in 1108 bytes} [Tue Sep 13 03:09:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 03:09:17.074810 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application [None req-bcde0bc3-7d9c-49a5-af29-8ba006843cd6 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 13 03:09:17.074810 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 03:09:17.074810 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 03:09:17.074810 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 03:09:17.074810 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 03:09:17.074810 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 03:09:17.074810 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 03:09:17.074810 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 03:09:17.074810 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 03:09:17.074810 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 03:09:17.074810 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 03:09:17.074810 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 03:09:17.074810 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 03:09:17.074810 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 03:09:17.074810 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 03:09:17.074810 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 03:09:17.078670 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 03:09:17.078670 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 03:09:17.078670 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 03:09:17.078670 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 03:09:17.078670 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 03:09:17.078670 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 03:09:17.078670 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 03:09:17.078670 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 03:09:17.078670 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 03:09:17.078670 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 03:09:17.078670 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 03:09:17.078670 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application with Lock( Sep 13 03:09:17.078670 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 03:09:17.078670 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._enter() Sep 13 03:09:17.078670 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 03:09:17.078670 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 03:09:17.079648 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 03:09:17.079648 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.creator() Sep 13 03:09:17.079648 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 03:09:17.079648 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application created_value = creator( Sep 13 03:09:17.079648 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 03:09:17.079648 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 03:09:17.079648 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 03:09:17.079648 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 03:09:17.079648 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 03:09:17.079648 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 03:09:17.079648 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 13 03:09:17.079648 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Sep 13 03:09:17.079648 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 544/1097] 10.222.0.18 () {56 vars in 1096 bytes} [Tue Sep 13 03:09:16 2022] GET /identity/v3/roles/admin => generated 83 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 13 03:09:17.080396 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-206ce39e-13b2-4ac8-882e-42666afcc1f7 None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:09:17.082038 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcde0bc3-7d9c-49a5-af29-8ba006843cd6 None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:09:17.176355 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-206ce39e-13b2-4ac8-882e-42666afcc1f7 None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:09:17.177876 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcde0bc3-7d9c-49a5-af29-8ba006843cd6 None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:09:17.202927 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:09:17.209490 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6314ffc-798f-443c-917f-e21725ad7226 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:09:17.210083 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6314ffc-798f-443c-917f-e21725ad7226 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:09:17.210355 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6314ffc-798f-443c-917f-e21725ad7226 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:09:17.210639 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6314ffc-798f-443c-917f-e21725ad7226 None admin] PATH_INFO: `/v3/domains` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:09:17.211267 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6314ffc-798f-443c-917f-e21725ad7226 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 03:09:17.211561 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6314ffc-798f-443c-917f-e21725ad7226 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:09:17.212596 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f662210-7f81-4612-9c1d-0d3534471e65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:09:17.212906 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6314ffc-798f-443c-917f-e21725ad7226 None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:09:17.213192 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f662210-7f81-4612-9c1d-0d3534471e65 None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:09:17.213472 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f662210-7f81-4612-9c1d-0d3534471e65 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:09:17.213743 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f662210-7f81-4612-9c1d-0d3534471e65 None admin] PATH_INFO: `/v3/roles` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:09:17.214391 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f662210-7f81-4612-9c1d-0d3534471e65 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 03:09:17.214719 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f662210-7f81-4612-9c1d-0d3534471e65 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:09:17.216651 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f662210-7f81-4612-9c1d-0d3534471e65 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:09:17.224608 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 545/1098] 10.222.0.18 () {56 vars in 1099 bytes} [Tue Sep 13 03:09:17 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 03:09:17.225703 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 554/1099] 10.222.0.18 () {56 vars in 1109 bytes} [Tue Sep 13 03:09:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 03:09:17.230414 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f662210-7f81-4612-9c1d-0d3534471e65 None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:09:17.230562 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6314ffc-798f-443c-917f-e21725ad7226 None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:09:17.268849 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:09:17.328314 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f662210-7f81-4612-9c1d-0d3534471e65 None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:09:17.329018 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6314ffc-798f-443c-917f-e21725ad7226 None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:09:17.361976 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f144fcc-384f-454f-879a-f02dba937b4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:09:17.362597 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f144fcc-384f-454f-879a-f02dba937b4d None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:09:17.362897 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f144fcc-384f-454f-879a-f02dba937b4d None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:09:17.363165 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f144fcc-384f-454f-879a-f02dba937b4d None admin] PATH_INFO: `/v3/projects/service` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:09:17.364566 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2040ff1-dd52-4b69-8f5e-7b9ad339031e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:09:17.365355 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2040ff1-dd52-4b69-8f5e-7b9ad339031e None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:09:17.365914 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2040ff1-dd52-4b69-8f5e-7b9ad339031e None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:09:17.366191 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2040ff1-dd52-4b69-8f5e-7b9ad339031e None admin] PATH_INFO: `/v3/users/2534727f8f3f4cd5b92c726c21c9b4e4` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:09:17.367325 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2040ff1-dd52-4b69-8f5e-7b9ad339031e None admin] RBAC: Authorizing `identity:get_user(user_id=2534727f8f3f4cd5b92c726c21c9b4e4)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:09:17.368772 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2040ff1-dd52-4b69-8f5e-7b9ad339031e None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:09:17.369702 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f144fcc-384f-454f-879a-f02dba937b4d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:09:17.370338 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 555/1100] 10.222.0.18 () {56 vars in 1177 bytes} [Tue Sep 13 03:09:17 2022] GET /identity/v3/users/2534727f8f3f4cd5b92c726c21c9b4e4 => generated 285 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 03:09:17.371172 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f144fcc-384f-454f-879a-f02dba937b4d None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:09:17.376020 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2040ff1-dd52-4b69-8f5e-7b9ad339031e None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:09:17.382471 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application [None req-4f144fcc-384f-454f-879a-f02dba937b4d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 13 03:09:17.382471 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 03:09:17.382471 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 03:09:17.382471 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 03:09:17.382471 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 03:09:17.382471 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 03:09:17.382471 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 03:09:17.382471 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 03:09:17.382471 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 03:09:17.382471 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 03:09:17.382471 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 03:09:17.382471 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 03:09:17.382471 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 13 03:09:17.382471 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 13 03:09:17.382471 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 13 03:09:17.383585 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 13 03:09:17.383585 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 03:09:17.383585 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 03:09:17.383585 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 03:09:17.383585 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 03:09:17.383585 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 03:09:17.383585 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 03:09:17.383585 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 03:09:17.383585 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application with Lock( Sep 13 03:09:17.383585 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 03:09:17.383585 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._enter() Sep 13 03:09:17.383585 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 03:09:17.383585 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 03:09:17.383585 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 03:09:17.383585 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.creator() Sep 13 03:09:17.383585 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 03:09:17.383585 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application created_value = creator( Sep 13 03:09:17.384600 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 13 03:09:17.384600 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 13 03:09:17.384600 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 13 03:09:17.384600 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 13 03:09:17.384600 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 13 03:09:17.384600 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 13 03:09:17.384600 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 13 03:09:17.384600 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Sep 13 03:09:17.384600 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 546/1101] 10.222.0.18 () {56 vars in 1111 bytes} [Tue Sep 13 03:09:17 2022] GET /identity/v3/projects/service => generated 88 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 03:09:17.389654 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f144fcc-384f-454f-879a-f02dba937b4d None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:09:17.438056 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:09:17.479229 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2040ff1-dd52-4b69-8f5e-7b9ad339031e None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:09:17.486224 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:09:17.487751 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f144fcc-384f-454f-879a-f02dba937b4d None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:09:17.513834 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6d497bb-de74-4cb0-8c89-760605c5fdd7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:09:17.514435 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6d497bb-de74-4cb0-8c89-760605c5fdd7 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:09:17.514740 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6d497bb-de74-4cb0-8c89-760605c5fdd7 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:09:17.515013 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6d497bb-de74-4cb0-8c89-760605c5fdd7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:09:17.521415 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfb5da4b-ec38-468a-a796-274528bbd85a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:09:17.521952 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb5da4b-ec38-468a-a796-274528bbd85a None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:09:17.522226 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb5da4b-ec38-468a-a796-274528bbd85a None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:09:17.522496 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb5da4b-ec38-468a-a796-274528bbd85a None admin] PATH_INFO: `/v3/projects` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:09:17.523200 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfb5da4b-ec38-468a-a796-274528bbd85a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 03:09:17.523501 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfb5da4b-ec38-468a-a796-274528bbd85a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:09:17.524911 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfb5da4b-ec38-468a-a796-274528bbd85a None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:09:17.537013 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 547/1102] 10.222.0.18 () {56 vars in 1148 bytes} [Tue Sep 13 03:09:17 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 03:09:17.543013 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfb5da4b-ec38-468a-a796-274528bbd85a None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:09:17.584811 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6d497bb-de74-4cb0-8c89-760605c5fdd7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:09:17.586252 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6d497bb-de74-4cb0-8c89-760605c5fdd7 None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:09:17.638797 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfb5da4b-ec38-468a-a796-274528bbd85a None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:09:17.647641 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:09:17.672184 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39ea13de-283c-4686-95be-a90633734412 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:09:17.672777 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ea13de-283c-4686-95be-a90633734412 None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:09:17.673048 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ea13de-283c-4686-95be-a90633734412 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:09:17.673321 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ea13de-283c-4686-95be-a90633734412 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:09:17.674030 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39ea13de-283c-4686-95be-a90633734412 None admin] RBAC: Adding query filter params (role.id=57ec3ac460b749a5910aea16ad1d643e, scope.project.id=b9d94f6a68f7470d9494f112175620dc, user.id=c7377f5bee4a49959182e1bd34bc97a4) {{(pid=81399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 03:09:17.674333 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39ea13de-283c-4686-95be-a90633734412 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:09:17.675772 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39ea13de-283c-4686-95be-a90633734412 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:09:17.681582 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 548/1103] 10.222.0.18 () {56 vars in 1374 bytes} [Tue Sep 13 03:09:17 2022] GET /identity/v3/role_assignments?user.id=c7377f5bee4a49959182e1bd34bc97a4&scope.project.id=b9d94f6a68f7470d9494f112175620dc&role.id=57ec3ac460b749a5910aea16ad1d643e => generated 608 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 03:09:17.683330 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 556/1104] 10.222.0.18 () {58 vars in 1282 bytes} [Tue Sep 13 03:09:17 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Sep 13 03:09:17.689621 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39ea13de-283c-4686-95be-a90633734412 None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:09:17.797345 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39ea13de-283c-4686-95be-a90633734412 None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:09:17.832634 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46b22427-7ade-45f3-b220-c517181fa135 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:09:17.833282 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b22427-7ade-45f3-b220-c517181fa135 None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:09:17.833561 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b22427-7ade-45f3-b220-c517181fa135 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:09:17.833833 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b22427-7ade-45f3-b220-c517181fa135 None admin] PATH_INFO: `/v3/projects/dc806d4e65dc49168d7393b2a0663d93` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:09:17.835033 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46b22427-7ade-45f3-b220-c517181fa135 None admin] RBAC: Authorizing `identity:get_project(project_id=dc806d4e65dc49168d7393b2a0663d93)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:09:17.836482 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46b22427-7ade-45f3-b220-c517181fa135 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:09:17.838191 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 549/1105] 10.222.0.18 () {56 vars in 1186 bytes} [Tue Sep 13 03:09:17 2022] GET /identity/v3/projects/dc806d4e65dc49168d7393b2a0663d93 => generated 311 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 03:09:17.844523 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6d497bb-de74-4cb0-8c89-760605c5fdd7 None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:09:17.852793 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:09:17.946555 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6d497bb-de74-4cb0-8c89-760605c5fdd7 None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:09:17.981025 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80426c55-c079-4b65-960f-46d1e7fed8a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:09:17.982213 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80426c55-c079-4b65-960f-46d1e7fed8a4 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:09:17.982490 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80426c55-c079-4b65-960f-46d1e7fed8a4 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:09:17.982790 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80426c55-c079-4b65-960f-46d1e7fed8a4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:09:17.983514 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80426c55-c079-4b65-960f-46d1e7fed8a4 None admin] RBAC: Adding query filter params (role.id=425d4b481c7846e0bc07e7e805d5b15f, scope.project.id=dc806d4e65dc49168d7393b2a0663d93, user.id=2534727f8f3f4cd5b92c726c21c9b4e4) {{(pid=81400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 03:09:17.983820 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80426c55-c079-4b65-960f-46d1e7fed8a4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:09:17.985253 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80426c55-c079-4b65-960f-46d1e7fed8a4 None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:09:17.991501 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 557/1106] 10.222.0.18 () {56 vars in 1374 bytes} [Tue Sep 13 03:09:17 2022] GET /identity/v3/role_assignments?user.id=2534727f8f3f4cd5b92c726c21c9b4e4&scope.project.id=dc806d4e65dc49168d7393b2a0663d93&role.id=425d4b481c7846e0bc07e7e805d5b15f => generated 608 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 03:09:20.151881 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93b02f21-64b7-4b0e-b93b-e130dcc42173 None None] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:09:20.152401 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93b02f21-64b7-4b0e-b93b-e130dcc42173 None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:09:20.152401 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93b02f21-64b7-4b0e-b93b-e130dcc42173 None None] PATH_INFO: `/` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:09:20.154374 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 550/1107] 10.222.0.18 () {54 vars in 927 bytes} [Tue Sep 13 03:09:20 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 03:09:20.159619 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0090b396-a326-441a-a63e-370725a0a7b4 None None] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:09:20.160022 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0090b396-a326-441a-a63e-370725a0a7b4 None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:09:20.160167 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0090b396-a326-441a-a63e-370725a0a7b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:09:20.177834 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.auth.core [None req-0090b396-a326-441a-a63e-370725a0a7b4 None None] MFA Rules not processed for user `49f280ab086b4610855c4ffa378517d9`. Rule list: `[]` (Enabled: `True`). {{(pid=81400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 03:09:20.179289 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-0090b396-a326-441a-a63e-370725a0a7b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:09:20.239062 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:09:20.243927 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 558/1108] 10.222.0.18 () {58 vars in 1024 bytes} [Tue Sep 13 03:09:20 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 13 03:09:20.511424 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93b02f21-64b7-4b0e-b93b-e130dcc42173 None None] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:09:20.552339 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:09:20.609244 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93b02f21-64b7-4b0e-b93b-e130dcc42173 None None] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:09:20.641329 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dafec774-00c6-436f-b2c8-5259cc922541 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:09:20.641940 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dafec774-00c6-436f-b2c8-5259cc922541 None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:09:20.642212 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dafec774-00c6-436f-b2c8-5259cc922541 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:09:20.642477 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dafec774-00c6-436f-b2c8-5259cc922541 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:09:20.643584 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dafec774-00c6-436f-b2c8-5259cc922541 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:09:20.644990 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dafec774-00c6-436f-b2c8-5259cc922541 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:09:20.646526 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 551/1109] 10.222.0.18 () {56 vars in 1108 bytes} [Tue Sep 13 03:09:20 2022] GET /identity/v3/domains/default => generated 199 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 03:09:20.652142 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0090b396-a326-441a-a63e-370725a0a7b4 None None] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:09:20.749264 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0090b396-a326-441a-a63e-370725a0a7b4 None None] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:09:20.782250 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14dd5075-84cb-4c8e-9360-e93b051e06b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:09:20.782815 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14dd5075-84cb-4c8e-9360-e93b051e06b2 None admin] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:09:20.783097 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14dd5075-84cb-4c8e-9360-e93b051e06b2 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:09:20.783370 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14dd5075-84cb-4c8e-9360-e93b051e06b2 None admin] PATH_INFO: `/v3/users` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:09:20.784010 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14dd5075-84cb-4c8e-9360-e93b051e06b2 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:09:20.785437 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14dd5075-84cb-4c8e-9360-e93b051e06b2 None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:09:20.806309 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 559/1110] 10.222.0.18 () {60 vars in 1132 bytes} [Tue Sep 13 03:09:20 2022] POST /identity/v3/users => generated 286 bytes in 154 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 03:09:22.970687 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d81c5311-812a-4bd4-9400-61db9f4c6f4a None None] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:09:22.971218 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d81c5311-812a-4bd4-9400-61db9f4c6f4a None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:09:22.971218 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d81c5311-812a-4bd4-9400-61db9f4c6f4a None None] PATH_INFO: `/` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:09:22.972564 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 552/1111] 10.222.0.18 () {54 vars in 927 bytes} [Tue Sep 13 03:09:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 03:09:22.977716 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f4d6c2d-45e1-46b9-a67b-fb277abf81c1 None None] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:09:22.978090 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f4d6c2d-45e1-46b9-a67b-fb277abf81c1 None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:09:22.978268 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f4d6c2d-45e1-46b9-a67b-fb277abf81c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:09:22.997255 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.auth.core [None req-3f4d6c2d-45e1-46b9-a67b-fb277abf81c1 None None] MFA Rules not processed for user `49f280ab086b4610855c4ffa378517d9`. Rule list: `[]` (Enabled: `True`). {{(pid=81400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 03:09:22.998442 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-3f4d6c2d-45e1-46b9-a67b-fb277abf81c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:09:23.047430 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:09:23.064648 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 560/1112] 10.222.0.18 () {58 vars in 1024 bytes} [Tue Sep 13 03:09:22 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 13 03:09:23.331373 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d81c5311-812a-4bd4-9400-61db9f4c6f4a None None] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:09:23.429599 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d81c5311-812a-4bd4-9400-61db9f4c6f4a None None] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:09:23.438522 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:09:23.463546 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3051d68-cab4-4a30-b281-4c5b70f8d419 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:09:23.464157 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3051d68-cab4-4a30-b281-4c5b70f8d419 None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:09:23.464433 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3051d68-cab4-4a30-b281-4c5b70f8d419 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:09:23.464704 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3051d68-cab4-4a30-b281-4c5b70f8d419 None admin] PATH_INFO: `/v3/roles/a9aa35ebda4d4aff80e62f750e9104ab` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:09:23.466093 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3051d68-cab4-4a30-b281-4c5b70f8d419 None admin] RBAC: Authorizing `identity:get_role(role_id=a9aa35ebda4d4aff80e62f750e9104ab)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:09:23.467475 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3051d68-cab4-4a30-b281-4c5b70f8d419 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:09:23.468800 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 553/1113] 10.222.0.18 () {56 vars in 1177 bytes} [Tue Sep 13 03:09:23 2022] GET /identity/v3/roles/a9aa35ebda4d4aff80e62f750e9104ab => generated 223 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 03:09:23.474540 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f4d6c2d-45e1-46b9-a67b-fb277abf81c1 None None] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:09:23.569680 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f4d6c2d-45e1-46b9-a67b-fb277abf81c1 None None] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:09:23.602394 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ec3cd69-1758-4e4b-90ee-ac3aac2fa7a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:09:23.602976 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec3cd69-1758-4e4b-90ee-ac3aac2fa7a2 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:09:23.603255 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec3cd69-1758-4e4b-90ee-ac3aac2fa7a2 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:09:23.603524 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec3cd69-1758-4e4b-90ee-ac3aac2fa7a2 None admin] PATH_INFO: `/v3/users/7a81e9df9d374406ad38d54c868598ee` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:09:23.615949 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ec3cd69-1758-4e4b-90ee-ac3aac2fa7a2 None admin] RBAC: Authorizing `identity:get_user(user_id=7a81e9df9d374406ad38d54c868598ee)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:09:23.617376 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ec3cd69-1758-4e4b-90ee-ac3aac2fa7a2 None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:09:23.618971 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 561/1114] 10.222.0.18 () {56 vars in 1177 bytes} [Tue Sep 13 03:09:23 2022] GET /identity/v3/users/7a81e9df9d374406ad38d54c868598ee => generated 286 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 03:09:23.624985 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3051d68-cab4-4a30-b281-4c5b70f8d419 None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:09:23.724174 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3051d68-cab4-4a30-b281-4c5b70f8d419 None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:09:23.758234 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe13278a-6995-4b2f-9a5d-13661884c461 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:09:23.758858 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe13278a-6995-4b2f-9a5d-13661884c461 None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:09:23.759139 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe13278a-6995-4b2f-9a5d-13661884c461 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:09:23.759408 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe13278a-6995-4b2f-9a5d-13661884c461 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:09:23.802277 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:09:23.824824 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe13278a-6995-4b2f-9a5d-13661884c461 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:09:23.826324 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe13278a-6995-4b2f-9a5d-13661884c461 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:09:23.919601 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 554/1115] 10.222.0.18 () {58 vars in 1282 bytes} [Tue Sep 13 03:09:23 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 295 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 13 03:09:23.925479 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ec3cd69-1758-4e4b-90ee-ac3aac2fa7a2 None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:09:23.970259 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:09:24.024105 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ec3cd69-1758-4e4b-90ee-ac3aac2fa7a2 None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:09:24.057774 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f08068cc-06e6-4aab-8e65-2762b713579b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:09:24.058401 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f08068cc-06e6-4aab-8e65-2762b713579b None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:09:24.058716 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f08068cc-06e6-4aab-8e65-2762b713579b None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:09:24.058992 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f08068cc-06e6-4aab-8e65-2762b713579b None admin] PATH_INFO: `/v3/projects/dc806d4e65dc49168d7393b2a0663d93` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:09:24.060652 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f08068cc-06e6-4aab-8e65-2762b713579b None admin] RBAC: Authorizing `identity:get_project(project_id=dc806d4e65dc49168d7393b2a0663d93)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:09:24.062088 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f08068cc-06e6-4aab-8e65-2762b713579b None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:09:24.063775 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 562/1116] 10.222.0.18 () {56 vars in 1186 bytes} [Tue Sep 13 03:09:23 2022] GET /identity/v3/projects/dc806d4e65dc49168d7393b2a0663d93 => generated 311 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 03:09:24.070257 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe13278a-6995-4b2f-9a5d-13661884c461 None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:09:24.142901 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:09:24.165530 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe13278a-6995-4b2f-9a5d-13661884c461 None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:09:24.198105 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-823076bc-dc31-420d-b101-6a6f2cc5100f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:09:24.198717 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-823076bc-dc31-420d-b101-6a6f2cc5100f None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:09:24.198993 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-823076bc-dc31-420d-b101-6a6f2cc5100f None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:09:24.199259 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-823076bc-dc31-420d-b101-6a6f2cc5100f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:09:24.199963 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-823076bc-dc31-420d-b101-6a6f2cc5100f None admin] RBAC: Adding query filter params (role.id=a9aa35ebda4d4aff80e62f750e9104ab, scope.project.id=dc806d4e65dc49168d7393b2a0663d93, user.id=7a81e9df9d374406ad38d54c868598ee) {{(pid=81399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 03:09:24.200265 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-823076bc-dc31-420d-b101-6a6f2cc5100f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:09:24.201664 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-823076bc-dc31-420d-b101-6a6f2cc5100f None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:09:24.206805 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 555/1117] 10.222.0.18 () {56 vars in 1374 bytes} [Tue Sep 13 03:09:24 2022] GET /identity/v3/role_assignments?user.id=7a81e9df9d374406ad38d54c868598ee&scope.project.id=dc806d4e65dc49168d7393b2a0663d93&role.id=a9aa35ebda4d4aff80e62f750e9104ab => generated 259 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 03:09:26.301196 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef5138aa-9bc6-4319-b027-541d92d91d60 None None] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:09:26.301772 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef5138aa-9bc6-4319-b027-541d92d91d60 None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:09:26.301772 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef5138aa-9bc6-4319-b027-541d92d91d60 None None] PATH_INFO: `/` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:09:26.303095 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 563/1118] 10.222.0.18 () {54 vars in 927 bytes} [Tue Sep 13 03:09:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 03:09:26.308507 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e8579a-8319-4884-8d18-d32e0d16d462 None None] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:09:26.308936 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e8579a-8319-4884-8d18-d32e0d16d462 None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:09:26.309120 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e8579a-8319-4884-8d18-d32e0d16d462 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:09:26.327337 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.auth.core [None req-e5e8579a-8319-4884-8d18-d32e0d16d462 None None] MFA Rules not processed for user `49f280ab086b4610855c4ffa378517d9`. Rule list: `[]` (Enabled: `True`). {{(pid=81399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 03:09:26.328455 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-e5e8579a-8319-4884-8d18-d32e0d16d462 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:09:26.391437 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 556/1119] 10.222.0.18 () {58 vars in 1024 bytes} [Tue Sep 13 03:09:26 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 13 03:09:26.659614 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef5138aa-9bc6-4319-b027-541d92d91d60 None None] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:09:26.757080 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef5138aa-9bc6-4319-b027-541d92d91d60 None None] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:09:26.770116 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:09:26.790781 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bde94cf3-1c13-46ba-b30f-4984b7cf01a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:09:26.791386 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde94cf3-1c13-46ba-b30f-4984b7cf01a8 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:09:26.791662 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde94cf3-1c13-46ba-b30f-4984b7cf01a8 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:09:26.791933 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde94cf3-1c13-46ba-b30f-4984b7cf01a8 None admin] PATH_INFO: `/v3/roles/a9aa35ebda4d4aff80e62f750e9104ab` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:09:26.793295 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bde94cf3-1c13-46ba-b30f-4984b7cf01a8 None admin] RBAC: Authorizing `identity:get_role(role_id=a9aa35ebda4d4aff80e62f750e9104ab)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:09:26.794698 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bde94cf3-1c13-46ba-b30f-4984b7cf01a8 None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:09:26.795986 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 564/1120] 10.222.0.18 () {56 vars in 1177 bytes} [Tue Sep 13 03:09:26 2022] GET /identity/v3/roles/a9aa35ebda4d4aff80e62f750e9104ab => generated 223 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 03:09:26.801572 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5e8579a-8319-4884-8d18-d32e0d16d462 None None] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:09:26.868105 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:09:26.897481 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5e8579a-8319-4884-8d18-d32e0d16d462 None None] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:09:26.929818 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de0acf8b-9cc1-41d1-9410-24f5c56fe09f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:09:26.930424 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de0acf8b-9cc1-41d1-9410-24f5c56fe09f None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:09:26.930727 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de0acf8b-9cc1-41d1-9410-24f5c56fe09f None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:09:26.930992 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de0acf8b-9cc1-41d1-9410-24f5c56fe09f None admin] PATH_INFO: `/v3/users/7a81e9df9d374406ad38d54c868598ee` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:09:26.932075 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de0acf8b-9cc1-41d1-9410-24f5c56fe09f None admin] RBAC: Authorizing `identity:get_user(user_id=7a81e9df9d374406ad38d54c868598ee)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:09:26.933474 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de0acf8b-9cc1-41d1-9410-24f5c56fe09f None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:09:26.935090 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 557/1121] 10.222.0.18 () {56 vars in 1177 bytes} [Tue Sep 13 03:09:26 2022] GET /identity/v3/users/7a81e9df9d374406ad38d54c868598ee => generated 286 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 03:09:26.940786 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bde94cf3-1c13-46ba-b30f-4984b7cf01a8 None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:09:27.036401 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bde94cf3-1c13-46ba-b30f-4984b7cf01a8 None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:09:27.068966 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6744a83a-cb98-4df8-9f74-be13bebc8e7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:09:27.069586 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6744a83a-cb98-4df8-9f74-be13bebc8e7c None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:09:27.070209 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6744a83a-cb98-4df8-9f74-be13bebc8e7c None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:09:27.070209 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6744a83a-cb98-4df8-9f74-be13bebc8e7c None admin] PATH_INFO: `/v3/projects/dc806d4e65dc49168d7393b2a0663d93` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:09:27.071209 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6744a83a-cb98-4df8-9f74-be13bebc8e7c None admin] RBAC: Authorizing `identity:get_project(project_id=dc806d4e65dc49168d7393b2a0663d93)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:09:27.072635 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6744a83a-cb98-4df8-9f74-be13bebc8e7c None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:09:27.074244 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 565/1122] 10.222.0.18 () {56 vars in 1186 bytes} [Tue Sep 13 03:09:26 2022] GET /identity/v3/projects/dc806d4e65dc49168d7393b2a0663d93 => generated 311 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 03:09:27.080566 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de0acf8b-9cc1-41d1-9410-24f5c56fe09f None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:09:27.176395 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de0acf8b-9cc1-41d1-9410-24f5c56fe09f None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:09:27.209448 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f16ed5f-474b-4d99-8ed7-eabf8dacb8c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:09:27.210031 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f16ed5f-474b-4d99-8ed7-eabf8dacb8c9 None admin] REQUEST_METHOD: `PUT` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:09:27.210306 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f16ed5f-474b-4d99-8ed7-eabf8dacb8c9 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:09:27.210577 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f16ed5f-474b-4d99-8ed7-eabf8dacb8c9 None admin] PATH_INFO: `/v3/projects/dc806d4e65dc49168d7393b2a0663d93/users/7a81e9df9d374406ad38d54c868598ee/roles/a9aa35ebda4d4aff80e62f750e9104ab` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:09:27.212695 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f16ed5f-474b-4d99-8ed7-eabf8dacb8c9 None admin] RBAC: Authorizing `identity:create_grant(project_id=dc806d4e65dc49168d7393b2a0663d93, user_id=7a81e9df9d374406ad38d54c868598ee, role_id=a9aa35ebda4d4aff80e62f750e9104ab)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:09:27.214108 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f16ed5f-474b-4d99-8ed7-eabf8dacb8c9 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:09:27.221752 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:09:27.227527 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 558/1123] 10.222.0.18 () {58 vars in 1439 bytes} [Tue Sep 13 03:09:27 2022] PUT /identity/v3/projects/dc806d4e65dc49168d7393b2a0663d93/users/7a81e9df9d374406ad38d54c868598ee/roles/a9aa35ebda4d4aff80e62f750e9104ab => generated 0 bytes in 147 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 03:09:29.328397 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-249664ba-53d6-4453-921d-ca217a861521 None None] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:09:29.328912 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-249664ba-53d6-4453-921d-ca217a861521 None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:09:29.328912 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-249664ba-53d6-4453-921d-ca217a861521 None None] PATH_INFO: `/` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:09:29.330181 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 566/1124] 10.222.0.18 () {54 vars in 927 bytes} [Tue Sep 13 03:09:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 03:09:29.335444 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05680847-c7cd-400c-b463-91a79c5df614 None None] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:09:29.335831 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05680847-c7cd-400c-b463-91a79c5df614 None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:09:29.335993 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05680847-c7cd-400c-b463-91a79c5df614 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:09:29.354104 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.auth.core [None req-05680847-c7cd-400c-b463-91a79c5df614 None None] MFA Rules not processed for user `49f280ab086b4610855c4ffa378517d9`. Rule list: `[]` (Enabled: `True`). {{(pid=81399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 03:09:29.355305 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-05680847-c7cd-400c-b463-91a79c5df614 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:09:29.419097 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 559/1125] 10.222.0.18 () {58 vars in 1024 bytes} [Tue Sep 13 03:09:29 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 13 03:09:29.686317 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-249664ba-53d6-4453-921d-ca217a861521 None None] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:09:29.734305 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:09:29.783804 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-249664ba-53d6-4453-921d-ca217a861521 None None] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:09:29.817354 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d28c7485-b74d-42cc-8824-9190a8c3ae51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:09:29.817955 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d28c7485-b74d-42cc-8824-9190a8c3ae51 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:09:29.818225 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d28c7485-b74d-42cc-8824-9190a8c3ae51 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:09:29.818490 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d28c7485-b74d-42cc-8824-9190a8c3ae51 None admin] PATH_INFO: `/v3/roles/a9aa35ebda4d4aff80e62f750e9104ab` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:09:29.819859 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d28c7485-b74d-42cc-8824-9190a8c3ae51 None admin] RBAC: Authorizing `identity:get_role(role_id=a9aa35ebda4d4aff80e62f750e9104ab)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:09:29.821208 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d28c7485-b74d-42cc-8824-9190a8c3ae51 None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:09:29.822492 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 567/1126] 10.222.0.18 () {56 vars in 1177 bytes} [Tue Sep 13 03:09:29 2022] GET /identity/v3/roles/a9aa35ebda4d4aff80e62f750e9104ab => generated 223 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 03:09:29.828167 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05680847-c7cd-400c-b463-91a79c5df614 None None] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:09:29.922157 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05680847-c7cd-400c-b463-91a79c5df614 None None] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:09:29.947871 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:09:29.955724 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e98864f-70e9-4554-8fe0-4c33930d825a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:09:29.956340 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e98864f-70e9-4554-8fe0-4c33930d825a None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:09:29.956618 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e98864f-70e9-4554-8fe0-4c33930d825a None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:09:29.956915 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e98864f-70e9-4554-8fe0-4c33930d825a None admin] PATH_INFO: `/v3/users/7a81e9df9d374406ad38d54c868598ee` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:09:29.958012 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e98864f-70e9-4554-8fe0-4c33930d825a None admin] RBAC: Authorizing `identity:get_user(user_id=7a81e9df9d374406ad38d54c868598ee)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:09:29.959459 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e98864f-70e9-4554-8fe0-4c33930d825a None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:09:29.961031 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 560/1127] 10.222.0.18 () {56 vars in 1177 bytes} [Tue Sep 13 03:09:29 2022] GET /identity/v3/users/7a81e9df9d374406ad38d54c868598ee => generated 286 bytes in 133 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 03:09:29.966748 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d28c7485-b74d-42cc-8824-9190a8c3ae51 None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:09:30.066049 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d28c7485-b74d-42cc-8824-9190a8c3ae51 None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:09:30.082373 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:09:30.099870 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83cf22b5-cf30-42ec-b21c-fbc8c54eb06e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:09:30.100468 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83cf22b5-cf30-42ec-b21c-fbc8c54eb06e None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:09:30.100744 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83cf22b5-cf30-42ec-b21c-fbc8c54eb06e None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:09:30.101015 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83cf22b5-cf30-42ec-b21c-fbc8c54eb06e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:09:30.166712 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83cf22b5-cf30-42ec-b21c-fbc8c54eb06e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:09:30.168147 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83cf22b5-cf30-42ec-b21c-fbc8c54eb06e None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:09:30.262787 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 568/1128] 10.222.0.18 () {58 vars in 1282 bytes} [Tue Sep 13 03:09:29 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 296 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 13 03:09:30.268624 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e98864f-70e9-4554-8fe0-4c33930d825a None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:09:30.363665 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e98864f-70e9-4554-8fe0-4c33930d825a None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:09:30.395336 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57b68091-3d5b-4c53-8bd9-e4217e699573 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:09:30.395937 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57b68091-3d5b-4c53-8bd9-e4217e699573 None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:09:30.396211 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57b68091-3d5b-4c53-8bd9-e4217e699573 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:09:30.396477 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57b68091-3d5b-4c53-8bd9-e4217e699573 None admin] PATH_INFO: `/v3/projects/dc806d4e65dc49168d7393b2a0663d93` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:09:30.397552 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57b68091-3d5b-4c53-8bd9-e4217e699573 None admin] RBAC: Authorizing `identity:get_project(project_id=dc806d4e65dc49168d7393b2a0663d93)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:09:30.398973 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57b68091-3d5b-4c53-8bd9-e4217e699573 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:09:30.400558 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 561/1129] 10.222.0.18 () {56 vars in 1186 bytes} [Tue Sep 13 03:09:30 2022] GET /identity/v3/projects/dc806d4e65dc49168d7393b2a0663d93 => generated 311 bytes in 132 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 03:09:30.407027 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83cf22b5-cf30-42ec-b21c-fbc8c54eb06e None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:09:30.424228 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:09:30.503147 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83cf22b5-cf30-42ec-b21c-fbc8c54eb06e None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:09:30.536044 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6013436-cc2e-48e8-98fb-2b04fb5d439d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:09:30.536574 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6013436-cc2e-48e8-98fb-2b04fb5d439d None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:09:30.536849 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6013436-cc2e-48e8-98fb-2b04fb5d439d None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:09:30.537121 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6013436-cc2e-48e8-98fb-2b04fb5d439d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:09:30.537831 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6013436-cc2e-48e8-98fb-2b04fb5d439d None admin] RBAC: Adding query filter params (role.id=a9aa35ebda4d4aff80e62f750e9104ab, scope.project.id=dc806d4e65dc49168d7393b2a0663d93, user.id=7a81e9df9d374406ad38d54c868598ee) {{(pid=81400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 03:09:30.538134 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6013436-cc2e-48e8-98fb-2b04fb5d439d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:09:30.539581 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6013436-cc2e-48e8-98fb-2b04fb5d439d None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:09:30.545240 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 569/1130] 10.222.0.18 () {56 vars in 1374 bytes} [Tue Sep 13 03:09:30 2022] GET /identity/v3/role_assignments?user.id=7a81e9df9d374406ad38d54c868598ee&scope.project.id=dc806d4e65dc49168d7393b2a0663d93&role.id=a9aa35ebda4d4aff80e62f750e9104ab => generated 608 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 03:09:32.678361 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ac1dfe-b6eb-4553-b418-cd99e62e0715 None None] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:09:32.679179 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ac1dfe-b6eb-4553-b418-cd99e62e0715 None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:09:32.679179 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ac1dfe-b6eb-4553-b418-cd99e62e0715 None None] PATH_INFO: `/` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:09:32.680537 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 562/1131] 10.222.0.18 () {54 vars in 927 bytes} [Tue Sep 13 03:09:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 03:09:32.685945 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cbc0bf8-2885-410a-b7c4-f2411eeaddcd None None] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:09:32.686325 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cbc0bf8-2885-410a-b7c4-f2411eeaddcd None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:09:32.686499 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cbc0bf8-2885-410a-b7c4-f2411eeaddcd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:09:32.705200 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.auth.core [None req-6cbc0bf8-2885-410a-b7c4-f2411eeaddcd None None] MFA Rules not processed for user `49f280ab086b4610855c4ffa378517d9`. Rule list: `[]` (Enabled: `True`). {{(pid=81400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 03:09:32.706382 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-6cbc0bf8-2885-410a-b7c4-f2411eeaddcd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:09:32.771336 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 570/1132] 10.222.0.18 () {58 vars in 1024 bytes} [Tue Sep 13 03:09:32 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 13 03:09:33.042685 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90ac1dfe-b6eb-4553-b418-cd99e62e0715 None None] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:09:33.103801 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:09:33.140218 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90ac1dfe-b6eb-4553-b418-cd99e62e0715 None None] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:09:33.172614 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83902b51-995c-4ab6-8255-9aeb955255df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:09:33.173230 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83902b51-995c-4ab6-8255-9aeb955255df None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:09:33.173506 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83902b51-995c-4ab6-8255-9aeb955255df None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:09:33.173774 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83902b51-995c-4ab6-8255-9aeb955255df None admin] PATH_INFO: `/v3/domains/default` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:09:33.174892 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83902b51-995c-4ab6-8255-9aeb955255df None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:09:33.176305 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83902b51-995c-4ab6-8255-9aeb955255df None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:09:33.177871 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 563/1133] 10.222.0.18 () {56 vars in 1108 bytes} [Tue Sep 13 03:09:33 2022] GET /identity/v3/domains/default => generated 199 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 03:09:33.183835 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cbc0bf8-2885-410a-b7c4-f2411eeaddcd None None] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:09:33.195544 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:09:33.280165 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cbc0bf8-2885-410a-b7c4-f2411eeaddcd None None] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:09:33.312921 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a813888-403a-4d2e-a1e6-6481358a7b94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:09:33.313452 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a813888-403a-4d2e-a1e6-6481358a7b94 None admin] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:09:33.313728 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a813888-403a-4d2e-a1e6-6481358a7b94 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:09:33.314000 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a813888-403a-4d2e-a1e6-6481358a7b94 None admin] PATH_INFO: `/v3/projects` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:09:33.314666 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a813888-403a-4d2e-a1e6-6481358a7b94 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:09:33.316081 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a813888-403a-4d2e-a1e6-6481358a7b94 None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:09:33.341920 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 571/1134] 10.222.0.18 () {60 vars in 1141 bytes} [Tue Sep 13 03:09:33 2022] POST /identity/v3/projects => generated 311 bytes in 159 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 03:09:35.491677 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1555163d-b94e-4aa7-9b1f-a5a78bcea71e None None] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:09:35.492193 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1555163d-b94e-4aa7-9b1f-a5a78bcea71e None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:09:35.492193 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1555163d-b94e-4aa7-9b1f-a5a78bcea71e None None] PATH_INFO: `/` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:09:35.493541 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 564/1135] 10.222.0.18 () {54 vars in 927 bytes} [Tue Sep 13 03:09:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 03:09:35.498904 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be5cdc0-80a8-4076-8412-f1b0b1848887 None None] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:09:35.499323 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be5cdc0-80a8-4076-8412-f1b0b1848887 None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:09:35.499446 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be5cdc0-80a8-4076-8412-f1b0b1848887 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:09:35.517744 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.auth.core [None req-0be5cdc0-80a8-4076-8412-f1b0b1848887 None None] MFA Rules not processed for user `49f280ab086b4610855c4ffa378517d9`. Rule list: `[]` (Enabled: `True`). {{(pid=81400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 03:09:35.518931 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-0be5cdc0-80a8-4076-8412-f1b0b1848887 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:09:35.573437 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:09:35.584088 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 572/1136] 10.222.0.18 () {58 vars in 1024 bytes} [Tue Sep 13 03:09:35 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 13 03:09:35.849462 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1555163d-b94e-4aa7-9b1f-a5a78bcea71e None None] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:09:35.945446 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1555163d-b94e-4aa7-9b1f-a5a78bcea71e None None] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:09:35.974031 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:09:35.979111 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7d4dcc0-3707-466b-b887-bfb6a0465d87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:09:35.979731 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7d4dcc0-3707-466b-b887-bfb6a0465d87 None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:09:35.980010 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7d4dcc0-3707-466b-b887-bfb6a0465d87 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:09:35.980282 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7d4dcc0-3707-466b-b887-bfb6a0465d87 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:09:35.981382 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7d4dcc0-3707-466b-b887-bfb6a0465d87 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:09:35.982824 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7d4dcc0-3707-466b-b887-bfb6a0465d87 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:09:35.984383 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 565/1137] 10.222.0.18 () {56 vars in 1108 bytes} [Tue Sep 13 03:09:35 2022] GET /identity/v3/domains/default => generated 199 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 03:09:35.990036 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0be5cdc0-80a8-4076-8412-f1b0b1848887 None None] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:09:36.086200 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0be5cdc0-80a8-4076-8412-f1b0b1848887 None None] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:09:36.119223 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6993df06-691e-4550-8f63-93b1ff0adec5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:09:36.119756 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6993df06-691e-4550-8f63-93b1ff0adec5 None admin] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:09:36.120031 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6993df06-691e-4550-8f63-93b1ff0adec5 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:09:36.120300 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6993df06-691e-4550-8f63-93b1ff0adec5 None admin] PATH_INFO: `/v3/users` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:09:36.120946 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6993df06-691e-4550-8f63-93b1ff0adec5 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:09:36.122367 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6993df06-691e-4550-8f63-93b1ff0adec5 None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:09:36.143088 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 573/1138] 10.222.0.18 () {60 vars in 1132 bytes} [Tue Sep 13 03:09:35 2022] POST /identity/v3/users => generated 286 bytes in 153 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 03:09:38.298960 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d10b16-4d97-4994-8cd3-02c34ec29f6a None None] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:09:38.299489 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d10b16-4d97-4994-8cd3-02c34ec29f6a None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:09:38.299489 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d10b16-4d97-4994-8cd3-02c34ec29f6a None None] PATH_INFO: `/` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:09:38.300847 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 566/1139] 10.222.0.18 () {54 vars in 927 bytes} [Tue Sep 13 03:09:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 03:09:38.306391 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3061f480-e571-43f5-8a71-d5fda6d1522a None None] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:09:38.306821 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3061f480-e571-43f5-8a71-d5fda6d1522a None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:09:38.306963 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3061f480-e571-43f5-8a71-d5fda6d1522a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:09:38.325343 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.auth.core [None req-3061f480-e571-43f5-8a71-d5fda6d1522a None None] MFA Rules not processed for user `49f280ab086b4610855c4ffa378517d9`. Rule list: `[]` (Enabled: `True`). {{(pid=81400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 03:09:38.326446 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-3061f480-e571-43f5-8a71-d5fda6d1522a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:09:38.367722 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:09:38.392178 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 574/1140] 10.222.0.18 () {58 vars in 1024 bytes} [Tue Sep 13 03:09:38 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 13 03:09:38.662250 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89d10b16-4d97-4994-8cd3-02c34ec29f6a None None] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:09:38.761438 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89d10b16-4d97-4994-8cd3-02c34ec29f6a None None] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:09:38.794583 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a44ddb3b-7540-414d-aa5e-233b84a46f10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:09:38.795224 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a44ddb3b-7540-414d-aa5e-233b84a46f10 None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:09:38.795502 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a44ddb3b-7540-414d-aa5e-233b84a46f10 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:09:38.795772 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a44ddb3b-7540-414d-aa5e-233b84a46f10 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:09:38.805748 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a44ddb3b-7540-414d-aa5e-233b84a46f10 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 03:09:38.810659 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application [None req-a44ddb3b-7540-414d-aa5e-233b84a46f10 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 13 03:09:38.810659 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 03:09:38.810659 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 03:09:38.810659 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 03:09:38.810659 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 03:09:38.810659 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 03:09:38.810659 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 03:09:38.810659 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 03:09:38.810659 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 03:09:38.810659 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 03:09:38.810659 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 03:09:38.810659 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 03:09:38.810659 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 03:09:38.810659 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 03:09:38.810659 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 03:09:38.810659 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 03:09:38.811807 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 03:09:38.811807 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 03:09:38.811807 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 03:09:38.811807 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 03:09:38.811807 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 03:09:38.811807 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 03:09:38.811807 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 03:09:38.811807 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 03:09:38.811807 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 03:09:38.811807 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 03:09:38.811807 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 03:09:38.811807 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application with Lock( Sep 13 03:09:38.811807 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 03:09:38.811807 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._enter() Sep 13 03:09:38.811807 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 03:09:38.811807 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 03:09:38.812897 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 03:09:38.812897 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.creator() Sep 13 03:09:38.812897 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 03:09:38.812897 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application created_value = creator( Sep 13 03:09:38.812897 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 03:09:38.812897 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 03:09:38.812897 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 03:09:38.812897 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 03:09:38.812897 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 03:09:38.812897 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 03:09:38.812897 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 13 03:09:38.812897 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Sep 13 03:09:38.812897 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 567/1141] 10.222.0.18 () {56 vars in 1096 bytes} [Tue Sep 13 03:09:38 2022] GET /identity/v3/roles/admin => generated 83 bytes in 150 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 03:09:38.817683 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3061f480-e571-43f5-8a71-d5fda6d1522a None None] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:09:38.913730 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3061f480-e571-43f5-8a71-d5fda6d1522a None None] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:09:38.946195 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad17bdf6-9a02-4c86-8888-f6a95e510207 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:09:38.946808 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad17bdf6-9a02-4c86-8888-f6a95e510207 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:09:38.947088 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad17bdf6-9a02-4c86-8888-f6a95e510207 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:09:38.947360 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad17bdf6-9a02-4c86-8888-f6a95e510207 None admin] PATH_INFO: `/v3/roles` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:09:38.948013 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad17bdf6-9a02-4c86-8888-f6a95e510207 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 03:09:38.948313 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad17bdf6-9a02-4c86-8888-f6a95e510207 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:09:38.950204 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad17bdf6-9a02-4c86-8888-f6a95e510207 None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:09:38.957926 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 575/1142] 10.222.0.18 () {56 vars in 1099 bytes} [Tue Sep 13 03:09:38 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 03:09:38.963535 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a44ddb3b-7540-414d-aa5e-233b84a46f10 None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:09:39.021062 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:09:39.062194 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a44ddb3b-7540-414d-aa5e-233b84a46f10 None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:09:39.096629 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a197379-4360-4b06-a0d9-ce37834fb237 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:09:39.097251 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a197379-4360-4b06-a0d9-ce37834fb237 None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:09:39.097530 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a197379-4360-4b06-a0d9-ce37834fb237 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:09:39.097801 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a197379-4360-4b06-a0d9-ce37834fb237 None admin] PATH_INFO: `/v3/users/116b5ee612f842e9a976236a24b25d1d` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:09:39.110409 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a197379-4360-4b06-a0d9-ce37834fb237 None admin] RBAC: Authorizing `identity:get_user(user_id=116b5ee612f842e9a976236a24b25d1d)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:09:39.111869 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a197379-4360-4b06-a0d9-ce37834fb237 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:09:39.113459 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 568/1143] 10.222.0.18 () {56 vars in 1177 bytes} [Tue Sep 13 03:09:38 2022] GET /identity/v3/users/116b5ee612f842e9a976236a24b25d1d => generated 286 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 03:09:39.119201 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad17bdf6-9a02-4c86-8888-f6a95e510207 None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:09:39.160355 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:09:39.217331 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad17bdf6-9a02-4c86-8888-f6a95e510207 None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:09:39.249428 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3846dce-e014-4d3f-a9c2-c5bb33e04534 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:09:39.250016 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3846dce-e014-4d3f-a9c2-c5bb33e04534 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:09:39.250292 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3846dce-e014-4d3f-a9c2-c5bb33e04534 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:09:39.250562 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3846dce-e014-4d3f-a9c2-c5bb33e04534 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:09:39.314795 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3846dce-e014-4d3f-a9c2-c5bb33e04534 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:09:39.316231 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3846dce-e014-4d3f-a9c2-c5bb33e04534 None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:09:39.354219 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:09:39.411515 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 576/1144] 10.222.0.18 () {58 vars in 1282 bytes} [Tue Sep 13 03:09:39 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 293 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 13 03:09:39.417427 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a197379-4360-4b06-a0d9-ce37834fb237 None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:09:39.513898 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a197379-4360-4b06-a0d9-ce37834fb237 None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:09:39.530197 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:09:39.547139 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f9e28bf-ef51-4d47-b875-a47ff5185ca1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:09:39.547755 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f9e28bf-ef51-4d47-b875-a47ff5185ca1 None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:09:39.548036 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f9e28bf-ef51-4d47-b875-a47ff5185ca1 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:09:39.548309 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f9e28bf-ef51-4d47-b875-a47ff5185ca1 None admin] PATH_INFO: `/v3/projects/0c2176a13b7845238c273729504ccbd0` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:09:39.549409 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f9e28bf-ef51-4d47-b875-a47ff5185ca1 None admin] RBAC: Authorizing `identity:get_project(project_id=0c2176a13b7845238c273729504ccbd0)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:09:39.550853 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f9e28bf-ef51-4d47-b875-a47ff5185ca1 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:09:39.552456 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 569/1145] 10.222.0.18 () {56 vars in 1186 bytes} [Tue Sep 13 03:09:39 2022] GET /identity/v3/projects/0c2176a13b7845238c273729504ccbd0 => generated 311 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 03:09:39.558594 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3846dce-e014-4d3f-a9c2-c5bb33e04534 None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:09:39.652327 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3846dce-e014-4d3f-a9c2-c5bb33e04534 None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:09:39.685785 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6e68a97-3728-4f65-adc4-7674fbd457a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:09:39.686370 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e68a97-3728-4f65-adc4-7674fbd457a3 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:09:39.686678 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e68a97-3728-4f65-adc4-7674fbd457a3 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:09:39.687048 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e68a97-3728-4f65-adc4-7674fbd457a3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:09:39.687769 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6e68a97-3728-4f65-adc4-7674fbd457a3 None admin] RBAC: Adding query filter params (role.id=425d4b481c7846e0bc07e7e805d5b15f, scope.project.id=0c2176a13b7845238c273729504ccbd0, user.id=116b5ee612f842e9a976236a24b25d1d) {{(pid=81400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 03:09:39.688073 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6e68a97-3728-4f65-adc4-7674fbd457a3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:09:39.689480 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6e68a97-3728-4f65-adc4-7674fbd457a3 None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:09:39.694579 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 577/1146] 10.222.0.18 () {56 vars in 1374 bytes} [Tue Sep 13 03:09:39 2022] GET /identity/v3/role_assignments?user.id=116b5ee612f842e9a976236a24b25d1d&scope.project.id=0c2176a13b7845238c273729504ccbd0&role.id=425d4b481c7846e0bc07e7e805d5b15f => generated 259 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 03:09:41.774844 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1818b41-e99d-4040-bd8b-4f3d1ac7aff5 None None] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:09:41.775386 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1818b41-e99d-4040-bd8b-4f3d1ac7aff5 None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:09:41.775386 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1818b41-e99d-4040-bd8b-4f3d1ac7aff5 None None] PATH_INFO: `/` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:09:41.776716 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 570/1147] 10.222.0.18 () {54 vars in 927 bytes} [Tue Sep 13 03:09:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 03:09:41.782421 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a110e9c-632d-4897-8c08-2a40123efa20 None None] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:09:41.782863 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a110e9c-632d-4897-8c08-2a40123efa20 None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:09:41.782996 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a110e9c-632d-4897-8c08-2a40123efa20 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:09:41.790142 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:09:41.801092 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.auth.core [None req-5a110e9c-632d-4897-8c08-2a40123efa20 None None] MFA Rules not processed for user `49f280ab086b4610855c4ffa378517d9`. Rule list: `[]` (Enabled: `True`). {{(pid=81400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 03:09:41.802286 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-5a110e9c-632d-4897-8c08-2a40123efa20 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:09:41.866753 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 578/1148] 10.222.0.18 () {58 vars in 1024 bytes} [Tue Sep 13 03:09:41 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 13 03:09:42.136659 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1818b41-e99d-4040-bd8b-4f3d1ac7aff5 None None] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:09:42.233422 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1818b41-e99d-4040-bd8b-4f3d1ac7aff5 None None] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:09:42.265924 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c4fff76-6f1f-49b4-b842-225a6407de3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:09:42.266528 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c4fff76-6f1f-49b4-b842-225a6407de3a None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:09:42.266829 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c4fff76-6f1f-49b4-b842-225a6407de3a None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:09:42.267100 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c4fff76-6f1f-49b4-b842-225a6407de3a None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:09:42.276701 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c4fff76-6f1f-49b4-b842-225a6407de3a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 03:09:42.281517 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application [None req-8c4fff76-6f1f-49b4-b842-225a6407de3a None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 13 03:09:42.281517 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 03:09:42.281517 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 03:09:42.281517 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 03:09:42.281517 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 03:09:42.281517 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 03:09:42.281517 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 03:09:42.281517 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 03:09:42.281517 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 03:09:42.281517 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 03:09:42.281517 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 03:09:42.281517 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 03:09:42.281517 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 03:09:42.281517 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 03:09:42.281517 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 03:09:42.281517 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 03:09:42.282630 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 03:09:42.282630 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 03:09:42.282630 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 03:09:42.282630 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 03:09:42.282630 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 03:09:42.282630 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 03:09:42.282630 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 03:09:42.282630 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 03:09:42.282630 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 03:09:42.282630 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 03:09:42.282630 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 03:09:42.282630 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application with Lock( Sep 13 03:09:42.282630 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 03:09:42.282630 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._enter() Sep 13 03:09:42.282630 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 03:09:42.282630 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 03:09:42.283517 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 03:09:42.283517 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.creator() Sep 13 03:09:42.283517 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 03:09:42.283517 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application created_value = creator( Sep 13 03:09:42.283517 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 03:09:42.283517 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 03:09:42.283517 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 03:09:42.283517 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 03:09:42.283517 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 03:09:42.283517 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 03:09:42.283517 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 13 03:09:42.283517 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Sep 13 03:09:42.283517 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 571/1149] 10.222.0.18 () {56 vars in 1096 bytes} [Tue Sep 13 03:09:42 2022] GET /identity/v3/roles/admin => generated 83 bytes in 146 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 03:09:42.288937 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a110e9c-632d-4897-8c08-2a40123efa20 None None] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:09:42.383913 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a110e9c-632d-4897-8c08-2a40123efa20 None None] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:09:42.415955 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0b7d9c7-5818-411b-b955-7f6787f38fbd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:09:42.416534 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0b7d9c7-5818-411b-b955-7f6787f38fbd None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:09:42.416805 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0b7d9c7-5818-411b-b955-7f6787f38fbd None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:09:42.417069 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0b7d9c7-5818-411b-b955-7f6787f38fbd None admin] PATH_INFO: `/v3/roles` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:09:42.417710 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0b7d9c7-5818-411b-b955-7f6787f38fbd None admin] RBAC: Adding query filter params (name=admin) {{(pid=81400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 03:09:42.418004 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0b7d9c7-5818-411b-b955-7f6787f38fbd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:09:42.419379 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0b7d9c7-5818-411b-b955-7f6787f38fbd None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:09:42.420650 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:09:42.427824 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 579/1150] 10.222.0.18 () {56 vars in 1099 bytes} [Tue Sep 13 03:09:42 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 03:09:42.433628 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c4fff76-6f1f-49b4-b842-225a6407de3a None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:09:42.479017 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:09:42.529636 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c4fff76-6f1f-49b4-b842-225a6407de3a None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:09:42.562271 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c9ffe57-35ef-42ac-bdc8-c8516a17e0b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:09:42.562911 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c9ffe57-35ef-42ac-bdc8-c8516a17e0b0 None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:09:42.563184 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c9ffe57-35ef-42ac-bdc8-c8516a17e0b0 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:09:42.563449 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c9ffe57-35ef-42ac-bdc8-c8516a17e0b0 None admin] PATH_INFO: `/v3/users/116b5ee612f842e9a976236a24b25d1d` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:09:42.564536 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c9ffe57-35ef-42ac-bdc8-c8516a17e0b0 None admin] RBAC: Authorizing `identity:get_user(user_id=116b5ee612f842e9a976236a24b25d1d)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:09:42.565936 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c9ffe57-35ef-42ac-bdc8-c8516a17e0b0 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:09:42.567503 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 572/1151] 10.222.0.18 () {56 vars in 1177 bytes} [Tue Sep 13 03:09:42 2022] GET /identity/v3/users/116b5ee612f842e9a976236a24b25d1d => generated 286 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 03:09:42.573061 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0b7d9c7-5818-411b-b955-7f6787f38fbd None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:09:42.668552 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0b7d9c7-5818-411b-b955-7f6787f38fbd None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:09:42.701511 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db291466-98af-4c34-bf85-d0d2bf4b7b56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:09:42.702116 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db291466-98af-4c34-bf85-d0d2bf4b7b56 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:09:42.702391 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db291466-98af-4c34-bf85-d0d2bf4b7b56 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:09:42.702685 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db291466-98af-4c34-bf85-d0d2bf4b7b56 None admin] PATH_INFO: `/v3/projects/0c2176a13b7845238c273729504ccbd0` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:09:42.703779 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db291466-98af-4c34-bf85-d0d2bf4b7b56 None admin] RBAC: Authorizing `identity:get_project(project_id=0c2176a13b7845238c273729504ccbd0)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:09:42.705185 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db291466-98af-4c34-bf85-d0d2bf4b7b56 None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:09:42.706795 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 580/1152] 10.222.0.18 () {56 vars in 1186 bytes} [Tue Sep 13 03:09:42 2022] GET /identity/v3/projects/0c2176a13b7845238c273729504ccbd0 => generated 311 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 03:09:42.712405 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c9ffe57-35ef-42ac-bdc8-c8516a17e0b0 None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:09:42.806786 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c9ffe57-35ef-42ac-bdc8-c8516a17e0b0 None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:09:42.820449 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:09:42.839586 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c32a166-fc29-4348-92ec-3953a95cc5ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:09:42.840225 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c32a166-fc29-4348-92ec-3953a95cc5ca None admin] REQUEST_METHOD: `PUT` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:09:42.840500 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c32a166-fc29-4348-92ec-3953a95cc5ca None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:09:42.840788 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c32a166-fc29-4348-92ec-3953a95cc5ca None admin] PATH_INFO: `/v3/projects/0c2176a13b7845238c273729504ccbd0/users/116b5ee612f842e9a976236a24b25d1d/roles/425d4b481c7846e0bc07e7e805d5b15f` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:09:42.842656 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c32a166-fc29-4348-92ec-3953a95cc5ca None admin] RBAC: Authorizing `identity:create_grant(project_id=0c2176a13b7845238c273729504ccbd0, user_id=116b5ee612f842e9a976236a24b25d1d, role_id=425d4b481c7846e0bc07e7e805d5b15f)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:09:42.844058 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c32a166-fc29-4348-92ec-3953a95cc5ca None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:09:42.856005 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 573/1153] 10.222.0.18 () {58 vars in 1439 bytes} [Tue Sep 13 03:09:42 2022] PUT /identity/v3/projects/0c2176a13b7845238c273729504ccbd0/users/116b5ee612f842e9a976236a24b25d1d/roles/425d4b481c7846e0bc07e7e805d5b15f => generated 0 bytes in 144 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 03:09:44.938856 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d5ef01c-1372-4cd6-bc03-9adb35b7f90f None None] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:09:44.939412 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d5ef01c-1372-4cd6-bc03-9adb35b7f90f None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:09:44.939412 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d5ef01c-1372-4cd6-bc03-9adb35b7f90f None None] PATH_INFO: `/` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:09:44.940731 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 581/1154] 10.222.0.18 () {54 vars in 927 bytes} [Tue Sep 13 03:09:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 03:09:44.945796 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f53ca2b-b609-4e8a-82a9-012684f61a0b None None] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:09:44.946178 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f53ca2b-b609-4e8a-82a9-012684f61a0b None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:09:44.946336 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f53ca2b-b609-4e8a-82a9-012684f61a0b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:09:44.964441 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.auth.core [None req-0f53ca2b-b609-4e8a-82a9-012684f61a0b None None] MFA Rules not processed for user `49f280ab086b4610855c4ffa378517d9`. Rule list: `[]` (Enabled: `True`). {{(pid=81399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 03:09:44.965594 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-0f53ca2b-b609-4e8a-82a9-012684f61a0b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:09:45.029593 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 574/1155] 10.222.0.18 () {58 vars in 1024 bytes} [Tue Sep 13 03:09:44 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 13 03:09:45.298773 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d5ef01c-1372-4cd6-bc03-9adb35b7f90f None None] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:09:45.359787 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:09:45.397639 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d5ef01c-1372-4cd6-bc03-9adb35b7f90f None None] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:09:45.429545 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58397a77-3ab9-474e-8aae-94e4f35a8921 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:09:45.430155 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58397a77-3ab9-474e-8aae-94e4f35a8921 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:09:45.430431 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58397a77-3ab9-474e-8aae-94e4f35a8921 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:09:45.430726 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58397a77-3ab9-474e-8aae-94e4f35a8921 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:09:45.441391 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58397a77-3ab9-474e-8aae-94e4f35a8921 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 03:09:45.446192 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application [None req-58397a77-3ab9-474e-8aae-94e4f35a8921 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 13 03:09:45.446192 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 03:09:45.446192 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 03:09:45.446192 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 03:09:45.446192 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 03:09:45.446192 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 03:09:45.446192 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 03:09:45.446192 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 03:09:45.446192 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 03:09:45.446192 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 03:09:45.446192 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 03:09:45.446192 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 03:09:45.446192 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 03:09:45.446192 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 03:09:45.446192 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 03:09:45.446192 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 03:09:45.447329 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 03:09:45.447329 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 03:09:45.447329 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 03:09:45.447329 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 03:09:45.447329 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 03:09:45.447329 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 03:09:45.447329 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 03:09:45.447329 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 03:09:45.447329 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 03:09:45.447329 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 03:09:45.447329 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 03:09:45.447329 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application with Lock( Sep 13 03:09:45.447329 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 03:09:45.447329 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._enter() Sep 13 03:09:45.447329 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 03:09:45.447329 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 03:09:45.448396 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 03:09:45.448396 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.creator() Sep 13 03:09:45.448396 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 03:09:45.448396 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application created_value = creator( Sep 13 03:09:45.448396 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 03:09:45.448396 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 03:09:45.448396 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 03:09:45.448396 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 03:09:45.448396 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 03:09:45.448396 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 03:09:45.448396 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 13 03:09:45.448396 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Sep 13 03:09:45.448396 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 582/1156] 10.222.0.18 () {56 vars in 1096 bytes} [Tue Sep 13 03:09:45 2022] GET /identity/v3/roles/admin => generated 83 bytes in 149 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 03:09:45.453870 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f53ca2b-b609-4e8a-82a9-012684f61a0b None None] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:09:45.547476 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f53ca2b-b609-4e8a-82a9-012684f61a0b None None] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:09:45.549749 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:09:45.580974 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3f3e6d3-6c1e-4f40-981d-0839a671ad14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:09:45.581552 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3f3e6d3-6c1e-4f40-981d-0839a671ad14 None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:09:45.581828 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3f3e6d3-6c1e-4f40-981d-0839a671ad14 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:09:45.582097 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3f3e6d3-6c1e-4f40-981d-0839a671ad14 None admin] PATH_INFO: `/v3/roles` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:09:45.582774 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3f3e6d3-6c1e-4f40-981d-0839a671ad14 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 03:09:45.583075 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3f3e6d3-6c1e-4f40-981d-0839a671ad14 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:09:45.584426 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3f3e6d3-6c1e-4f40-981d-0839a671ad14 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:09:45.592117 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 575/1157] 10.222.0.18 () {56 vars in 1099 bytes} [Tue Sep 13 03:09:45 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 03:09:45.597784 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58397a77-3ab9-474e-8aae-94e4f35a8921 None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:09:45.694906 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58397a77-3ab9-474e-8aae-94e4f35a8921 None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:09:45.716620 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:09:45.727453 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef51eae3-76e3-491c-aad3-7185872000f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:09:45.728063 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef51eae3-76e3-491c-aad3-7185872000f1 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:09:45.728340 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef51eae3-76e3-491c-aad3-7185872000f1 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:09:45.728609 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef51eae3-76e3-491c-aad3-7185872000f1 None admin] PATH_INFO: `/v3/users/116b5ee612f842e9a976236a24b25d1d` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:09:45.729717 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef51eae3-76e3-491c-aad3-7185872000f1 None admin] RBAC: Authorizing `identity:get_user(user_id=116b5ee612f842e9a976236a24b25d1d)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:09:45.731174 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef51eae3-76e3-491c-aad3-7185872000f1 None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:09:45.733244 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 583/1158] 10.222.0.18 () {56 vars in 1177 bytes} [Tue Sep 13 03:09:45 2022] GET /identity/v3/users/116b5ee612f842e9a976236a24b25d1d => generated 286 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 03:09:45.738827 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3f3e6d3-6c1e-4f40-981d-0839a671ad14 None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:09:45.834359 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3f3e6d3-6c1e-4f40-981d-0839a671ad14 None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:09:45.866328 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27f37d23-2566-41b2-b2a7-c18d276a647d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:09:45.866938 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27f37d23-2566-41b2-b2a7-c18d276a647d None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:09:45.867218 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27f37d23-2566-41b2-b2a7-c18d276a647d None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:09:45.867485 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27f37d23-2566-41b2-b2a7-c18d276a647d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:09:45.899157 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:09:45.933017 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27f37d23-2566-41b2-b2a7-c18d276a647d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:09:45.934465 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27f37d23-2566-41b2-b2a7-c18d276a647d None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:09:46.026319 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 576/1159] 10.222.0.18 () {58 vars in 1282 bytes} [Tue Sep 13 03:09:45 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 288 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 13 03:09:46.032740 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef51eae3-76e3-491c-aad3-7185872000f1 None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:09:46.129472 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef51eae3-76e3-491c-aad3-7185872000f1 None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:09:46.162367 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c548505-a26e-4f8e-8a7b-d5392510a305 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:09:46.163001 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c548505-a26e-4f8e-8a7b-d5392510a305 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:09:46.163274 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c548505-a26e-4f8e-8a7b-d5392510a305 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:09:46.163540 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c548505-a26e-4f8e-8a7b-d5392510a305 None admin] PATH_INFO: `/v3/projects/0c2176a13b7845238c273729504ccbd0` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:09:46.164862 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c548505-a26e-4f8e-8a7b-d5392510a305 None admin] RBAC: Authorizing `identity:get_project(project_id=0c2176a13b7845238c273729504ccbd0)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:09:46.166280 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c548505-a26e-4f8e-8a7b-d5392510a305 None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:09:46.167922 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 584/1160] 10.222.0.18 () {56 vars in 1186 bytes} [Tue Sep 13 03:09:46 2022] GET /identity/v3/projects/0c2176a13b7845238c273729504ccbd0 => generated 311 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 03:09:46.174845 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27f37d23-2566-41b2-b2a7-c18d276a647d None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:09:46.238339 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:09:46.272486 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27f37d23-2566-41b2-b2a7-c18d276a647d None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:09:46.305983 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-832b0ebd-d3db-4a0a-860e-af87bba44407 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:09:46.306578 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-832b0ebd-d3db-4a0a-860e-af87bba44407 None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:09:46.306890 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-832b0ebd-d3db-4a0a-860e-af87bba44407 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:09:46.307164 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-832b0ebd-d3db-4a0a-860e-af87bba44407 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:09:46.307882 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-832b0ebd-d3db-4a0a-860e-af87bba44407 None admin] RBAC: Adding query filter params (role.id=425d4b481c7846e0bc07e7e805d5b15f, scope.project.id=0c2176a13b7845238c273729504ccbd0, user.id=116b5ee612f842e9a976236a24b25d1d) {{(pid=81399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 03:09:46.308191 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-832b0ebd-d3db-4a0a-860e-af87bba44407 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:09:46.309602 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-832b0ebd-d3db-4a0a-860e-af87bba44407 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:09:46.315273 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 577/1161] 10.222.0.18 () {56 vars in 1374 bytes} [Tue Sep 13 03:09:46 2022] GET /identity/v3/role_assignments?user.id=116b5ee612f842e9a976236a24b25d1d&scope.project.id=0c2176a13b7845238c273729504ccbd0&role.id=425d4b481c7846e0bc07e7e805d5b15f => generated 608 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 03:09:48.450884 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f26e8a5-5d6f-4905-9c1e-6fe75752fb2a None None] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:09:48.451423 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f26e8a5-5d6f-4905-9c1e-6fe75752fb2a None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:09:48.451423 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f26e8a5-5d6f-4905-9c1e-6fe75752fb2a None None] PATH_INFO: `/` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:09:48.452742 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 585/1162] 10.222.0.18 () {54 vars in 927 bytes} [Tue Sep 13 03:09:48 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 03:09:48.458017 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea967531-f6db-4049-b855-73569ba8dbf1 None None] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:09:48.458435 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea967531-f6db-4049-b855-73569ba8dbf1 None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:09:48.458572 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea967531-f6db-4049-b855-73569ba8dbf1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:09:48.476312 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.auth.core [None req-ea967531-f6db-4049-b855-73569ba8dbf1 None None] MFA Rules not processed for user `49f280ab086b4610855c4ffa378517d9`. Rule list: `[]` (Enabled: `True`). {{(pid=81399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 03:09:48.477489 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-ea967531-f6db-4049-b855-73569ba8dbf1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:09:48.541309 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 578/1163] 10.222.0.18 () {58 vars in 1024 bytes} [Tue Sep 13 03:09:48 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 13 03:09:48.805067 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f26e8a5-5d6f-4905-9c1e-6fe75752fb2a None None] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:09:48.862852 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:09:48.902825 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f26e8a5-5d6f-4905-9c1e-6fe75752fb2a None None] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:09:48.935458 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-387ee6eb-9642-4050-8684-135508fc8f4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:09:48.936043 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-387ee6eb-9642-4050-8684-135508fc8f4f None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:09:48.936313 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-387ee6eb-9642-4050-8684-135508fc8f4f None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:09:48.936577 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-387ee6eb-9642-4050-8684-135508fc8f4f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:09:49.000391 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-387ee6eb-9642-4050-8684-135508fc8f4f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:09:49.001862 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-387ee6eb-9642-4050-8684-135508fc8f4f None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:09:49.056379 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:09:49.096297 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 586/1164] 10.222.0.18 () {58 vars in 1282 bytes} [Tue Sep 13 03:09:48 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 292 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 13 03:09:49.102222 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea967531-f6db-4049-b855-73569ba8dbf1 None None] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:09:49.158243 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:09:49.197267 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea967531-f6db-4049-b855-73569ba8dbf1 None None] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:09:49.229326 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ae4808f-2ff9-4be5-94ff-93bbb340aff5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:09:49.229942 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ae4808f-2ff9-4be5-94ff-93bbb340aff5 None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:09:49.230218 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ae4808f-2ff9-4be5-94ff-93bbb340aff5 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:09:49.230487 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ae4808f-2ff9-4be5-94ff-93bbb340aff5 None admin] PATH_INFO: `/v3/domains/swift_test` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:09:49.237467 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ae4808f-2ff9-4be5-94ff-93bbb340aff5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=swift_test)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:09:49.238906 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ae4808f-2ff9-4be5-94ff-93bbb340aff5 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:09:49.250010 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application [None req-1ae4808f-2ff9-4be5-94ff-93bbb340aff5 None admin] Could not find domain: swift_test.: keystone.exception.DomainNotFound: Could not find domain: swift_test. Sep 13 03:09:49.250010 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 03:09:49.250010 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 712, in get_domain Sep 13 03:09:49.250010 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application project = self.driver.get_project(domain_id) Sep 13 03:09:49.250010 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 13 03:09:49.250010 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 13 03:09:49.250010 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 13 03:09:49.250010 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 13 03:09:49.250010 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: swift_test. Sep 13 03:09:49.250010 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Sep 13 03:09:49.250010 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 13 03:09:49.250010 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Sep 13 03:09:49.250010 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 03:09:49.250010 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 03:09:49.250010 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 03:09:49.250010 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 03:09:49.250010 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 03:09:49.251296 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 03:09:49.251296 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 03:09:49.251296 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 03:09:49.251296 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 03:09:49.251296 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 03:09:49.251296 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 03:09:49.251296 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 13 03:09:49.251296 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 13 03:09:49.251296 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 13 03:09:49.251296 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 13 03:09:49.251296 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 03:09:49.251296 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 03:09:49.251296 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 03:09:49.251296 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 03:09:49.251296 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 03:09:49.251296 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 03:09:49.252521 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 03:09:49.252521 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application with Lock( Sep 13 03:09:49.252521 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 03:09:49.252521 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._enter() Sep 13 03:09:49.252521 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 03:09:49.252521 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 03:09:49.252521 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 03:09:49.252521 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.creator() Sep 13 03:09:49.252521 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 03:09:49.252521 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application created_value = creator( Sep 13 03:09:49.252521 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 718, in get_domain Sep 13 03:09:49.252521 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 13 03:09:49.252521 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: swift_test. Sep 13 03:09:49.252521 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Sep 13 03:09:49.252521 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 579/1165] 10.222.0.18 () {56 vars in 1117 bytes} [Tue Sep 13 03:09:49 2022] GET /identity/v3/domains/swift_test => generated 90 bytes in 149 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 13 03:09:49.257010 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-387ee6eb-9642-4050-8684-135508fc8f4f None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:09:49.352198 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-387ee6eb-9642-4050-8684-135508fc8f4f None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:09:49.384210 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dfc9c9d-32b8-4a2a-92a1-53fc7eecce23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:09:49.384790 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dfc9c9d-32b8-4a2a-92a1-53fc7eecce23 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:09:49.385062 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dfc9c9d-32b8-4a2a-92a1-53fc7eecce23 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:09:49.385327 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dfc9c9d-32b8-4a2a-92a1-53fc7eecce23 None admin] PATH_INFO: `/v3/domains` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:09:49.385944 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dfc9c9d-32b8-4a2a-92a1-53fc7eecce23 None admin] RBAC: Adding query filter params (name=swift_test) {{(pid=81400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 03:09:49.386238 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dfc9c9d-32b8-4a2a-92a1-53fc7eecce23 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:09:49.388106 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dfc9c9d-32b8-4a2a-92a1-53fc7eecce23 None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:09:49.394951 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 587/1166] 10.222.0.18 () {56 vars in 1115 bytes} [Tue Sep 13 03:09:49 2022] GET /identity/v3/domains?name=swift_test => generated 125 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 03:09:51.502533 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e39a4b3c-1538-458e-9957-13863e015eb1 None None] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:09:51.503066 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e39a4b3c-1538-458e-9957-13863e015eb1 None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:09:51.503189 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e39a4b3c-1538-458e-9957-13863e015eb1 None None] PATH_INFO: `/` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:09:51.504442 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 580/1167] 10.222.0.18 () {54 vars in 927 bytes} [Tue Sep 13 03:09:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 03:09:51.509761 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac218c8a-beb1-4bef-b6a6-1b3c37b9bfc2 None None] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:09:51.510198 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac218c8a-beb1-4bef-b6a6-1b3c37b9bfc2 None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:09:51.510303 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac218c8a-beb1-4bef-b6a6-1b3c37b9bfc2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:09:51.528475 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.auth.core [None req-ac218c8a-beb1-4bef-b6a6-1b3c37b9bfc2 None None] MFA Rules not processed for user `49f280ab086b4610855c4ffa378517d9`. Rule list: `[]` (Enabled: `True`). {{(pid=81400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 03:09:51.529629 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-ac218c8a-beb1-4bef-b6a6-1b3c37b9bfc2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:09:51.546534 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:09:51.594047 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 588/1168] 10.222.0.18 () {58 vars in 1024 bytes} [Tue Sep 13 03:09:51 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 13 03:09:51.860476 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e39a4b3c-1538-458e-9957-13863e015eb1 None None] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:09:51.957719 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e39a4b3c-1538-458e-9957-13863e015eb1 None None] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:09:51.977117 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:09:51.991414 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca9bec72-d62c-4474-a696-14aaee6ee9e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:09:51.992004 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca9bec72-d62c-4474-a696-14aaee6ee9e2 None admin] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:09:51.992278 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca9bec72-d62c-4474-a696-14aaee6ee9e2 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:09:51.992548 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca9bec72-d62c-4474-a696-14aaee6ee9e2 None admin] PATH_INFO: `/v3/domains` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:09:51.993207 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca9bec72-d62c-4474-a696-14aaee6ee9e2 None admin] RBAC: Authorizing `identity:create_domain()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:09:51.994544 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca9bec72-d62c-4474-a696-14aaee6ee9e2 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:09:52.009068 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 581/1169] 10.222.0.18 () {60 vars in 1138 bytes} [Tue Sep 13 03:09:51 2022] POST /identity/v3/domains => generated 267 bytes in 149 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 13 03:09:54.143141 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1efe97fb-f988-4028-87cb-9d7c37d5dec7 None None] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:09:54.143660 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1efe97fb-f988-4028-87cb-9d7c37d5dec7 None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:09:54.143760 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1efe97fb-f988-4028-87cb-9d7c37d5dec7 None None] PATH_INFO: `/` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:09:54.145013 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 589/1170] 10.222.0.18 () {54 vars in 927 bytes} [Tue Sep 13 03:09:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 03:09:54.150132 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-268dce40-b884-4974-a75b-c7d5194219b8 None None] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:09:54.150500 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-268dce40-b884-4974-a75b-c7d5194219b8 None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:09:54.150714 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-268dce40-b884-4974-a75b-c7d5194219b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:09:54.168833 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.auth.core [None req-268dce40-b884-4974-a75b-c7d5194219b8 None None] MFA Rules not processed for user `49f280ab086b4610855c4ffa378517d9`. Rule list: `[]` (Enabled: `True`). {{(pid=81399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 03:09:54.170696 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-268dce40-b884-4974-a75b-c7d5194219b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:09:54.235141 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 582/1171] 10.222.0.18 () {58 vars in 1024 bytes} [Tue Sep 13 03:09:54 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 13 03:09:54.500446 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1efe97fb-f988-4028-87cb-9d7c37d5dec7 None None] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:09:54.596584 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1efe97fb-f988-4028-87cb-9d7c37d5dec7 None None] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:09:54.629118 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3db4765e-7908-4700-a95e-c240a522cc99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:09:54.629718 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3db4765e-7908-4700-a95e-c240a522cc99 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:09:54.629989 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3db4765e-7908-4700-a95e-c240a522cc99 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:09:54.630254 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3db4765e-7908-4700-a95e-c240a522cc99 None admin] PATH_INFO: `/v3/domains/64f0c48751a44400b29e8f4ae266554f` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:09:54.640098 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3db4765e-7908-4700-a95e-c240a522cc99 None admin] RBAC: Authorizing `identity:get_domain(domain_id=64f0c48751a44400b29e8f4ae266554f)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:09:54.641517 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3db4765e-7908-4700-a95e-c240a522cc99 None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:09:54.643075 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 590/1172] 10.222.0.18 () {56 vars in 1183 bytes} [Tue Sep 13 03:09:54 2022] GET /identity/v3/domains/64f0c48751a44400b29e8f4ae266554f => generated 267 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 03:09:54.648719 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-268dce40-b884-4974-a75b-c7d5194219b8 None None] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:09:54.744828 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-268dce40-b884-4974-a75b-c7d5194219b8 None None] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:09:54.755611 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:09:54.778187 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-952c7e2e-0dd6-4fdb-b8d5-c7b8918b2f84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:09:54.778826 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-952c7e2e-0dd6-4fdb-b8d5-c7b8918b2f84 None admin] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:09:54.779109 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-952c7e2e-0dd6-4fdb-b8d5-c7b8918b2f84 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:09:54.779410 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-952c7e2e-0dd6-4fdb-b8d5-c7b8918b2f84 None admin] PATH_INFO: `/v3/projects` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:09:54.780043 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-952c7e2e-0dd6-4fdb-b8d5-c7b8918b2f84 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:09:54.781449 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-952c7e2e-0dd6-4fdb-b8d5-c7b8918b2f84 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:09:54.806481 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 583/1173] 10.222.0.18 () {60 vars in 1141 bytes} [Tue Sep 13 03:09:54 2022] POST /identity/v3/projects => generated 361 bytes in 158 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 03:09:56.973794 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e47c95-bab1-4703-985e-56a34ce5ca5d None None] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:09:56.974351 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e47c95-bab1-4703-985e-56a34ce5ca5d None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:09:56.974351 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e47c95-bab1-4703-985e-56a34ce5ca5d None None] PATH_INFO: `/` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:09:56.975705 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 591/1174] 10.222.0.18 () {54 vars in 927 bytes} [Tue Sep 13 03:09:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 03:09:56.981076 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f466512-3a4c-4682-869f-bf7d1d5de0ae None None] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:09:56.981504 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f466512-3a4c-4682-869f-bf7d1d5de0ae None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:09:56.981608 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f466512-3a4c-4682-869f-bf7d1d5de0ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:09:56.999411 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.auth.core [None req-6f466512-3a4c-4682-869f-bf7d1d5de0ae None None] MFA Rules not processed for user `49f280ab086b4610855c4ffa378517d9`. Rule list: `[]` (Enabled: `True`). {{(pid=81399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 03:09:57.000583 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-6f466512-3a4c-4682-869f-bf7d1d5de0ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:09:57.064130 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 584/1175] 10.222.0.18 () {58 vars in 1024 bytes} [Tue Sep 13 03:09:56 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 13 03:09:57.335553 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9e47c95-bab1-4703-985e-56a34ce5ca5d None None] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:09:57.351404 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:09:57.434201 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9e47c95-bab1-4703-985e-56a34ce5ca5d None None] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:09:57.466712 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea9364c0-5257-47ac-b74d-5d4af885b413 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:09:57.467319 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea9364c0-5257-47ac-b74d-5d4af885b413 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:09:57.467590 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea9364c0-5257-47ac-b74d-5d4af885b413 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:09:57.467855 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea9364c0-5257-47ac-b74d-5d4af885b413 None admin] PATH_INFO: `/v3/domains/64f0c48751a44400b29e8f4ae266554f` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:09:57.468938 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea9364c0-5257-47ac-b74d-5d4af885b413 None admin] RBAC: Authorizing `identity:get_domain(domain_id=64f0c48751a44400b29e8f4ae266554f)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:09:57.470348 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea9364c0-5257-47ac-b74d-5d4af885b413 None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:09:57.471900 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 592/1176] 10.222.0.18 () {56 vars in 1183 bytes} [Tue Sep 13 03:09:57 2022] GET /identity/v3/domains/64f0c48751a44400b29e8f4ae266554f => generated 267 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 03:09:57.477500 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f466512-3a4c-4682-869f-bf7d1d5de0ae None None] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:09:57.564586 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:09:57.573607 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f466512-3a4c-4682-869f-bf7d1d5de0ae None None] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:09:57.606433 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be6cf705-3ea7-45d8-a876-33f2596caed4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:09:57.607482 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be6cf705-3ea7-45d8-a876-33f2596caed4 None admin] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:09:57.607755 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be6cf705-3ea7-45d8-a876-33f2596caed4 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:09:57.608017 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be6cf705-3ea7-45d8-a876-33f2596caed4 None admin] PATH_INFO: `/v3/users` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:09:57.608642 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be6cf705-3ea7-45d8-a876-33f2596caed4 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:09:57.610030 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be6cf705-3ea7-45d8-a876-33f2596caed4 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:09:57.631059 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 585/1177] 10.222.0.18 () {60 vars in 1132 bytes} [Tue Sep 13 03:09:57 2022] POST /identity/v3/users => generated 311 bytes in 154 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 03:09:59.775411 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f17f7a7e-d7a1-4d25-87d4-44221d5ae8a0 None None] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:09:59.775940 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f17f7a7e-d7a1-4d25-87d4-44221d5ae8a0 None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:09:59.775940 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f17f7a7e-d7a1-4d25-87d4-44221d5ae8a0 None None] PATH_INFO: `/` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:09:59.777282 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 593/1178] 10.222.0.18 () {54 vars in 927 bytes} [Tue Sep 13 03:09:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 03:09:59.782549 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a35c6ab-3a84-41f7-98ee-4f8554e2bf01 None None] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:09:59.782966 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a35c6ab-3a84-41f7-98ee-4f8554e2bf01 None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:09:59.783118 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a35c6ab-3a84-41f7-98ee-4f8554e2bf01 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:09:59.801275 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.auth.core [None req-2a35c6ab-3a84-41f7-98ee-4f8554e2bf01 None None] MFA Rules not processed for user `49f280ab086b4610855c4ffa378517d9`. Rule list: `[]` (Enabled: `True`). {{(pid=81399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 03:09:59.802444 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-2a35c6ab-3a84-41f7-98ee-4f8554e2bf01 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:09:59.866196 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 586/1179] 10.222.0.18 () {58 vars in 1024 bytes} [Tue Sep 13 03:09:59 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 13 03:10:00.132131 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f17f7a7e-d7a1-4d25-87d4-44221d5ae8a0 None None] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:10:00.212930 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:10:00.231628 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f17f7a7e-d7a1-4d25-87d4-44221d5ae8a0 None None] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:10:00.263773 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38ccad64-1012-46a4-a779-b73ea951ae05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:10:00.264381 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ccad64-1012-46a4-a779-b73ea951ae05 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:10:00.264656 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ccad64-1012-46a4-a779-b73ea951ae05 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:10:00.264924 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ccad64-1012-46a4-a779-b73ea951ae05 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:10:00.275000 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38ccad64-1012-46a4-a779-b73ea951ae05 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 03:10:00.280322 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application [None req-38ccad64-1012-46a4-a779-b73ea951ae05 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 13 03:10:00.280322 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 03:10:00.280322 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 03:10:00.280322 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 03:10:00.280322 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 03:10:00.280322 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 03:10:00.280322 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 03:10:00.280322 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 03:10:00.280322 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 03:10:00.280322 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 03:10:00.280322 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 03:10:00.280322 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 03:10:00.280322 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 03:10:00.280322 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 03:10:00.280322 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 03:10:00.280322 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 03:10:00.281413 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 03:10:00.281413 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 03:10:00.281413 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 03:10:00.281413 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 03:10:00.281413 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 03:10:00.281413 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 03:10:00.281413 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 03:10:00.281413 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 03:10:00.281413 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 03:10:00.281413 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 03:10:00.281413 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 03:10:00.281413 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application with Lock( Sep 13 03:10:00.281413 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 03:10:00.281413 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._enter() Sep 13 03:10:00.281413 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 03:10:00.281413 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 03:10:00.282295 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 03:10:00.282295 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.creator() Sep 13 03:10:00.282295 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 03:10:00.282295 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application created_value = creator( Sep 13 03:10:00.282295 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 03:10:00.282295 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 03:10:00.282295 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 03:10:00.282295 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 03:10:00.282295 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 03:10:00.282295 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 03:10:00.282295 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 13 03:10:00.282295 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Sep 13 03:10:00.282295 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 594/1180] 10.222.0.18 () {56 vars in 1096 bytes} [Tue Sep 13 03:10:00 2022] GET /identity/v3/roles/admin => generated 83 bytes in 150 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 03:10:00.287430 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a35c6ab-3a84-41f7-98ee-4f8554e2bf01 None None] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:10:00.360858 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:10:00.382170 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a35c6ab-3a84-41f7-98ee-4f8554e2bf01 None None] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:10:00.414509 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd26581c-623f-4236-956d-c5e8304912ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:10:00.415130 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd26581c-623f-4236-956d-c5e8304912ba None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:10:00.415406 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd26581c-623f-4236-956d-c5e8304912ba None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:10:00.415677 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd26581c-623f-4236-956d-c5e8304912ba None admin] PATH_INFO: `/v3/roles` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:10:00.416330 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd26581c-623f-4236-956d-c5e8304912ba None admin] RBAC: Adding query filter params (name=admin) {{(pid=81399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 03:10:00.416632 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd26581c-623f-4236-956d-c5e8304912ba None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:10:00.417993 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd26581c-623f-4236-956d-c5e8304912ba None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:10:00.426470 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 587/1181] 10.222.0.18 () {56 vars in 1099 bytes} [Tue Sep 13 03:10:00 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 03:10:00.432345 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38ccad64-1012-46a4-a779-b73ea951ae05 None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:10:00.527700 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38ccad64-1012-46a4-a779-b73ea951ae05 None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:10:00.560502 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba5f02f0-896b-4dfe-890a-b410458e0db5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:10:00.561110 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba5f02f0-896b-4dfe-890a-b410458e0db5 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:10:00.561385 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba5f02f0-896b-4dfe-890a-b410458e0db5 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:10:00.561654 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba5f02f0-896b-4dfe-890a-b410458e0db5 None admin] PATH_INFO: `/v3/users/00976b80b34c4c1a9da3207b33ff1358` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:10:00.572237 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:10:00.574748 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba5f02f0-896b-4dfe-890a-b410458e0db5 None admin] RBAC: Authorizing `identity:get_user(user_id=00976b80b34c4c1a9da3207b33ff1358)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:10:00.577193 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba5f02f0-896b-4dfe-890a-b410458e0db5 None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:10:00.579066 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 595/1182] 10.222.0.18 () {56 vars in 1177 bytes} [Tue Sep 13 03:10:00 2022] GET /identity/v3/users/00976b80b34c4c1a9da3207b33ff1358 => generated 311 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 03:10:00.584518 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd26581c-623f-4236-956d-c5e8304912ba None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:10:00.679927 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd26581c-623f-4236-956d-c5e8304912ba None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:10:00.711685 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b44da6c-538f-432d-bd67-8bde39a009a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:10:00.712268 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b44da6c-538f-432d-bd67-8bde39a009a3 None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:10:00.712542 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b44da6c-538f-432d-bd67-8bde39a009a3 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:10:00.712807 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b44da6c-538f-432d-bd67-8bde39a009a3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:10:00.723157 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:10:00.776958 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b44da6c-538f-432d-bd67-8bde39a009a3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:10:00.778383 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b44da6c-538f-432d-bd67-8bde39a009a3 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:10:00.871392 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 588/1183] 10.222.0.18 () {58 vars in 1282 bytes} [Tue Sep 13 03:10:00 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 287 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 13 03:10:00.877822 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba5f02f0-896b-4dfe-890a-b410458e0db5 None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:10:00.973120 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba5f02f0-896b-4dfe-890a-b410458e0db5 None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:10:01.006242 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63ed3310-12c8-4a65-8ac8-937edd2cf184 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:10:01.006863 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ed3310-12c8-4a65-8ac8-937edd2cf184 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:10:01.007135 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ed3310-12c8-4a65-8ac8-937edd2cf184 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:10:01.007399 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ed3310-12c8-4a65-8ac8-937edd2cf184 None admin] PATH_INFO: `/v3/projects/d707409af2f24c2d9e05aa82e93f57cb` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:10:01.008475 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63ed3310-12c8-4a65-8ac8-937edd2cf184 None admin] RBAC: Authorizing `identity:get_project(project_id=d707409af2f24c2d9e05aa82e93f57cb)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:10:01.009883 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63ed3310-12c8-4a65-8ac8-937edd2cf184 None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:10:01.011490 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 596/1184] 10.222.0.18 () {56 vars in 1186 bytes} [Tue Sep 13 03:10:00 2022] GET /identity/v3/projects/d707409af2f24c2d9e05aa82e93f57cb => generated 361 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 03:10:01.018067 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b44da6c-538f-432d-bd67-8bde39a009a3 None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:10:01.057223 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:10:01.114294 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b44da6c-538f-432d-bd67-8bde39a009a3 None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:10:01.147244 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1015d149-80fb-4bc6-bab4-ec25c3936691 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:10:01.147841 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1015d149-80fb-4bc6-bab4-ec25c3936691 None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:10:01.148120 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1015d149-80fb-4bc6-bab4-ec25c3936691 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:10:01.148392 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1015d149-80fb-4bc6-bab4-ec25c3936691 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:10:01.149109 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1015d149-80fb-4bc6-bab4-ec25c3936691 None admin] RBAC: Adding query filter params (role.id=425d4b481c7846e0bc07e7e805d5b15f, scope.project.id=d707409af2f24c2d9e05aa82e93f57cb, user.id=00976b80b34c4c1a9da3207b33ff1358) {{(pid=81399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 03:10:01.149415 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1015d149-80fb-4bc6-bab4-ec25c3936691 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:10:01.150847 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1015d149-80fb-4bc6-bab4-ec25c3936691 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:10:01.156043 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 589/1185] 10.222.0.18 () {56 vars in 1374 bytes} [Tue Sep 13 03:10:01 2022] GET /identity/v3/role_assignments?user.id=00976b80b34c4c1a9da3207b33ff1358&scope.project.id=d707409af2f24c2d9e05aa82e93f57cb&role.id=425d4b481c7846e0bc07e7e805d5b15f => generated 259 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 03:10:03.260091 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72476b43-b858-4346-880b-4d19164d30a0 None None] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:10:03.260603 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72476b43-b858-4346-880b-4d19164d30a0 None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:10:03.260603 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72476b43-b858-4346-880b-4d19164d30a0 None None] PATH_INFO: `/` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:10:03.261933 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 597/1186] 10.222.0.18 () {54 vars in 927 bytes} [Tue Sep 13 03:10:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 03:10:03.267295 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceac90c2-9571-44cb-a3d1-2060fcdb8cf6 None None] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:10:03.267692 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceac90c2-9571-44cb-a3d1-2060fcdb8cf6 None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:10:03.267836 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceac90c2-9571-44cb-a3d1-2060fcdb8cf6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:10:03.286710 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.auth.core [None req-ceac90c2-9571-44cb-a3d1-2060fcdb8cf6 None None] MFA Rules not processed for user `49f280ab086b4610855c4ffa378517d9`. Rule list: `[]` (Enabled: `True`). {{(pid=81399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 03:10:03.287878 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-ceac90c2-9571-44cb-a3d1-2060fcdb8cf6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:10:03.351165 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 590/1187] 10.222.0.18 () {58 vars in 1024 bytes} [Tue Sep 13 03:10:03 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 13 03:10:03.620547 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72476b43-b858-4346-880b-4d19164d30a0 None None] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:10:03.685336 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:10:03.718475 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72476b43-b858-4346-880b-4d19164d30a0 None None] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:10:03.750211 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e66b43a6-106f-4351-9a80-c8f3b40ff9bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:10:03.750834 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e66b43a6-106f-4351-9a80-c8f3b40ff9bb None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:10:03.751106 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e66b43a6-106f-4351-9a80-c8f3b40ff9bb None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:10:03.751395 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e66b43a6-106f-4351-9a80-c8f3b40ff9bb None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:10:03.760942 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e66b43a6-106f-4351-9a80-c8f3b40ff9bb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 03:10:03.766229 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application [None req-e66b43a6-106f-4351-9a80-c8f3b40ff9bb None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 13 03:10:03.766229 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 03:10:03.766229 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 03:10:03.766229 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 03:10:03.766229 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 03:10:03.766229 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 03:10:03.766229 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 03:10:03.766229 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 03:10:03.766229 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 03:10:03.766229 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 03:10:03.766229 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 03:10:03.766229 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 03:10:03.766229 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 03:10:03.766229 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 03:10:03.766229 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 03:10:03.766229 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 03:10:03.767372 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 03:10:03.767372 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 03:10:03.767372 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 03:10:03.767372 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 03:10:03.767372 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 03:10:03.767372 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 03:10:03.767372 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 03:10:03.767372 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 03:10:03.767372 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 03:10:03.767372 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 03:10:03.767372 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 03:10:03.767372 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application with Lock( Sep 13 03:10:03.767372 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 03:10:03.767372 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._enter() Sep 13 03:10:03.767372 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 03:10:03.767372 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 03:10:03.768416 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 03:10:03.768416 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.creator() Sep 13 03:10:03.768416 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 03:10:03.768416 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application created_value = creator( Sep 13 03:10:03.768416 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 03:10:03.768416 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 03:10:03.768416 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 03:10:03.768416 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 03:10:03.768416 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 03:10:03.768416 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 03:10:03.768416 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 13 03:10:03.768416 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Sep 13 03:10:03.768416 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 598/1188] 10.222.0.18 () {56 vars in 1096 bytes} [Tue Sep 13 03:10:03 2022] GET /identity/v3/roles/admin => generated 83 bytes in 147 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 03:10:03.773190 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceac90c2-9571-44cb-a3d1-2060fcdb8cf6 None None] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:10:03.808001 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:10:03.870908 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceac90c2-9571-44cb-a3d1-2060fcdb8cf6 None None] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:10:03.903507 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41011833-a5cc-4d45-bcc9-7d85f7806d64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:10:03.904094 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41011833-a5cc-4d45-bcc9-7d85f7806d64 None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:10:03.904368 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41011833-a5cc-4d45-bcc9-7d85f7806d64 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:10:03.904630 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41011833-a5cc-4d45-bcc9-7d85f7806d64 None admin] PATH_INFO: `/v3/roles` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:10:03.905271 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41011833-a5cc-4d45-bcc9-7d85f7806d64 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 03:10:03.905564 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41011833-a5cc-4d45-bcc9-7d85f7806d64 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:10:03.906928 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41011833-a5cc-4d45-bcc9-7d85f7806d64 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:10:03.914499 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 591/1189] 10.222.0.18 () {56 vars in 1099 bytes} [Tue Sep 13 03:10:03 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 03:10:03.920315 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e66b43a6-106f-4351-9a80-c8f3b40ff9bb None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:10:04.017551 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e66b43a6-106f-4351-9a80-c8f3b40ff9bb None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:10:04.042543 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:10:04.051089 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6de32d8f-9a64-4b5b-a703-512e5a5f19fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:10:04.051706 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de32d8f-9a64-4b5b-a703-512e5a5f19fa None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:10:04.052018 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de32d8f-9a64-4b5b-a703-512e5a5f19fa None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:10:04.052290 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de32d8f-9a64-4b5b-a703-512e5a5f19fa None admin] PATH_INFO: `/v3/users/00976b80b34c4c1a9da3207b33ff1358` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:10:04.053647 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6de32d8f-9a64-4b5b-a703-512e5a5f19fa None admin] RBAC: Authorizing `identity:get_user(user_id=00976b80b34c4c1a9da3207b33ff1358)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:10:04.055090 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6de32d8f-9a64-4b5b-a703-512e5a5f19fa None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:10:04.056649 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 599/1190] 10.222.0.18 () {56 vars in 1177 bytes} [Tue Sep 13 03:10:03 2022] GET /identity/v3/users/00976b80b34c4c1a9da3207b33ff1358 => generated 311 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 03:10:04.062920 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41011833-a5cc-4d45-bcc9-7d85f7806d64 None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:10:04.152258 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:10:04.158125 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41011833-a5cc-4d45-bcc9-7d85f7806d64 None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:10:04.191738 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-440cbfe5-65fe-4fb2-91e7-fe5b5752aa43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:10:04.192360 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-440cbfe5-65fe-4fb2-91e7-fe5b5752aa43 None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:10:04.192639 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-440cbfe5-65fe-4fb2-91e7-fe5b5752aa43 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:10:04.192912 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-440cbfe5-65fe-4fb2-91e7-fe5b5752aa43 None admin] PATH_INFO: `/v3/projects/d707409af2f24c2d9e05aa82e93f57cb` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:10:04.194016 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-440cbfe5-65fe-4fb2-91e7-fe5b5752aa43 None admin] RBAC: Authorizing `identity:get_project(project_id=d707409af2f24c2d9e05aa82e93f57cb)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:10:04.195500 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-440cbfe5-65fe-4fb2-91e7-fe5b5752aa43 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:10:04.197144 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 592/1191] 10.222.0.18 () {56 vars in 1186 bytes} [Tue Sep 13 03:10:04 2022] GET /identity/v3/projects/d707409af2f24c2d9e05aa82e93f57cb => generated 361 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 03:10:04.203728 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6de32d8f-9a64-4b5b-a703-512e5a5f19fa None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:10:04.299459 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6de32d8f-9a64-4b5b-a703-512e5a5f19fa None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:10:04.333038 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a410b08f-9767-4746-8f99-1e2740cb103d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:10:04.333679 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a410b08f-9767-4746-8f99-1e2740cb103d None admin] REQUEST_METHOD: `PUT` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:10:04.333956 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a410b08f-9767-4746-8f99-1e2740cb103d None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:10:04.334227 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a410b08f-9767-4746-8f99-1e2740cb103d None admin] PATH_INFO: `/v3/projects/d707409af2f24c2d9e05aa82e93f57cb/users/00976b80b34c4c1a9da3207b33ff1358/roles/425d4b481c7846e0bc07e7e805d5b15f` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:10:04.336293 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a410b08f-9767-4746-8f99-1e2740cb103d None admin] RBAC: Authorizing `identity:create_grant(project_id=d707409af2f24c2d9e05aa82e93f57cb, user_id=00976b80b34c4c1a9da3207b33ff1358, role_id=425d4b481c7846e0bc07e7e805d5b15f)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:10:04.337716 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a410b08f-9767-4746-8f99-1e2740cb103d None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:10:04.350147 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 600/1192] 10.222.0.18 () {58 vars in 1439 bytes} [Tue Sep 13 03:10:04 2022] PUT /identity/v3/projects/d707409af2f24c2d9e05aa82e93f57cb/users/00976b80b34c4c1a9da3207b33ff1358/roles/425d4b481c7846e0bc07e7e805d5b15f => generated 0 bytes in 147 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 03:10:06.448423 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-effb4b59-9c2e-4810-8953-0cff0b010eff None None] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:10:06.448946 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-effb4b59-9c2e-4810-8953-0cff0b010eff None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:10:06.449042 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-effb4b59-9c2e-4810-8953-0cff0b010eff None None] PATH_INFO: `/` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:10:06.450386 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 593/1193] 10.222.0.18 () {54 vars in 927 bytes} [Tue Sep 13 03:10:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 03:10:06.455652 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-342fdb07-c1bc-4a60-b1b8-067b53cea1df None None] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:10:06.456073 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-342fdb07-c1bc-4a60-b1b8-067b53cea1df None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:10:06.456196 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-342fdb07-c1bc-4a60-b1b8-067b53cea1df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:10:06.474306 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.auth.core [None req-342fdb07-c1bc-4a60-b1b8-067b53cea1df None None] MFA Rules not processed for user `49f280ab086b4610855c4ffa378517d9`. Rule list: `[]` (Enabled: `True`). {{(pid=81400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 03:10:06.475492 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-342fdb07-c1bc-4a60-b1b8-067b53cea1df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:10:06.523617 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:10:06.540650 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 601/1194] 10.222.0.18 () {58 vars in 1024 bytes} [Tue Sep 13 03:10:06 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 13 03:10:06.806358 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-effb4b59-9c2e-4810-8953-0cff0b010eff None None] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:10:06.903805 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-effb4b59-9c2e-4810-8953-0cff0b010eff None None] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:10:06.935969 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f3ed965-a560-426a-8a20-3c276ef9757a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:10:06.936579 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f3ed965-a560-426a-8a20-3c276ef9757a None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:10:06.936859 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f3ed965-a560-426a-8a20-3c276ef9757a None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:10:06.937129 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f3ed965-a560-426a-8a20-3c276ef9757a None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:10:06.947258 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f3ed965-a560-426a-8a20-3c276ef9757a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 03:10:06.952726 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application [None req-8f3ed965-a560-426a-8a20-3c276ef9757a None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 13 03:10:06.952726 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 03:10:06.952726 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 03:10:06.952726 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 03:10:06.952726 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 03:10:06.952726 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 03:10:06.952726 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 03:10:06.952726 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 03:10:06.952726 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 03:10:06.952726 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 03:10:06.952726 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 03:10:06.952726 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 03:10:06.952726 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 03:10:06.952726 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 03:10:06.952726 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 03:10:06.952726 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 03:10:06.953818 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 03:10:06.953818 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 03:10:06.953818 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 03:10:06.953818 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 03:10:06.953818 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 03:10:06.953818 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 03:10:06.953818 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 03:10:06.953818 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 03:10:06.953818 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 03:10:06.953818 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 03:10:06.953818 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 03:10:06.953818 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application with Lock( Sep 13 03:10:06.953818 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 03:10:06.953818 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._enter() Sep 13 03:10:06.953818 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 03:10:06.953818 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 03:10:06.954725 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 03:10:06.954725 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.creator() Sep 13 03:10:06.954725 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 03:10:06.954725 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application created_value = creator( Sep 13 03:10:06.954725 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 03:10:06.954725 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 03:10:06.954725 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 03:10:06.954725 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 03:10:06.954725 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 03:10:06.954725 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 03:10:06.954725 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 13 03:10:06.954725 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Sep 13 03:10:06.954725 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 594/1195] 10.222.0.18 () {56 vars in 1096 bytes} [Tue Sep 13 03:10:06 2022] GET /identity/v3/roles/admin => generated 83 bytes in 148 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 03:10:06.960490 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-342fdb07-c1bc-4a60-b1b8-067b53cea1df None None] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:10:07.053864 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-342fdb07-c1bc-4a60-b1b8-067b53cea1df None None] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:10:07.087781 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1895880-0262-4aa9-a02f-070c9fd98a8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:10:07.088382 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1895880-0262-4aa9-a02f-070c9fd98a8b None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:10:07.088661 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1895880-0262-4aa9-a02f-070c9fd98a8b None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:10:07.088934 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1895880-0262-4aa9-a02f-070c9fd98a8b None admin] PATH_INFO: `/v3/roles` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:10:07.089589 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1895880-0262-4aa9-a02f-070c9fd98a8b None admin] RBAC: Adding query filter params (name=admin) {{(pid=81400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 03:10:07.089890 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1895880-0262-4aa9-a02f-070c9fd98a8b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:10:07.091282 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1895880-0262-4aa9-a02f-070c9fd98a8b None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:10:07.099083 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 602/1196] 10.222.0.18 () {56 vars in 1099 bytes} [Tue Sep 13 03:10:06 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 03:10:07.104887 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f3ed965-a560-426a-8a20-3c276ef9757a None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:10:07.129889 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:10:07.202014 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f3ed965-a560-426a-8a20-3c276ef9757a None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:10:07.238215 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94c82fd9-5267-459f-8617-b84e1c113724 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:10:07.238854 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94c82fd9-5267-459f-8617-b84e1c113724 None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:10:07.239132 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94c82fd9-5267-459f-8617-b84e1c113724 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:10:07.239407 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94c82fd9-5267-459f-8617-b84e1c113724 None admin] PATH_INFO: `/v3/users/00976b80b34c4c1a9da3207b33ff1358` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:10:07.240491 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94c82fd9-5267-459f-8617-b84e1c113724 None admin] RBAC: Authorizing `identity:get_user(user_id=00976b80b34c4c1a9da3207b33ff1358)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:10:07.241894 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94c82fd9-5267-459f-8617-b84e1c113724 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:10:07.243480 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 595/1197] 10.222.0.18 () {56 vars in 1177 bytes} [Tue Sep 13 03:10:07 2022] GET /identity/v3/users/00976b80b34c4c1a9da3207b33ff1358 => generated 311 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 03:10:07.249245 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1895880-0262-4aa9-a02f-070c9fd98a8b None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:10:07.297567 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:10:07.348009 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1895880-0262-4aa9-a02f-070c9fd98a8b None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:10:07.380815 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51bcdc3b-a2d1-4e73-b9c7-428e99d55dc9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:10:07.381405 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51bcdc3b-a2d1-4e73-b9c7-428e99d55dc9 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:10:07.381680 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51bcdc3b-a2d1-4e73-b9c7-428e99d55dc9 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:10:07.381947 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51bcdc3b-a2d1-4e73-b9c7-428e99d55dc9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:10:07.446633 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51bcdc3b-a2d1-4e73-b9c7-428e99d55dc9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:10:07.448074 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51bcdc3b-a2d1-4e73-b9c7-428e99d55dc9 None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:10:07.492851 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:10:07.542297 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 603/1198] 10.222.0.18 () {58 vars in 1282 bytes} [Tue Sep 13 03:10:07 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 294 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 13 03:10:07.548633 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94c82fd9-5267-459f-8617-b84e1c113724 None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:10:07.635884 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:10:07.644312 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94c82fd9-5267-459f-8617-b84e1c113724 None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:10:07.676410 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81dabdd9-284c-461b-8812-47b73a67dcad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:10:07.676999 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81dabdd9-284c-461b-8812-47b73a67dcad None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:10:07.677273 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81dabdd9-284c-461b-8812-47b73a67dcad None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:10:07.677540 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81dabdd9-284c-461b-8812-47b73a67dcad None admin] PATH_INFO: `/v3/projects/d707409af2f24c2d9e05aa82e93f57cb` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:10:07.678906 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81dabdd9-284c-461b-8812-47b73a67dcad None admin] RBAC: Authorizing `identity:get_project(project_id=d707409af2f24c2d9e05aa82e93f57cb)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:10:07.680301 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81dabdd9-284c-461b-8812-47b73a67dcad None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:10:07.681883 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 596/1199] 10.222.0.18 () {56 vars in 1186 bytes} [Tue Sep 13 03:10:07 2022] GET /identity/v3/projects/d707409af2f24c2d9e05aa82e93f57cb => generated 361 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 03:10:07.687843 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51bcdc3b-a2d1-4e73-b9c7-428e99d55dc9 None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:10:07.783201 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51bcdc3b-a2d1-4e73-b9c7-428e99d55dc9 None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:10:07.816296 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5248af79-5213-4391-a7cb-39237caf5db9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:10:07.816877 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5248af79-5213-4391-a7cb-39237caf5db9 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:10:07.817150 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5248af79-5213-4391-a7cb-39237caf5db9 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:10:07.817417 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5248af79-5213-4391-a7cb-39237caf5db9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:10:07.818120 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5248af79-5213-4391-a7cb-39237caf5db9 None admin] RBAC: Adding query filter params (role.id=425d4b481c7846e0bc07e7e805d5b15f, scope.project.id=d707409af2f24c2d9e05aa82e93f57cb, user.id=00976b80b34c4c1a9da3207b33ff1358) {{(pid=81400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 03:10:07.818422 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5248af79-5213-4391-a7cb-39237caf5db9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:10:07.819856 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5248af79-5213-4391-a7cb-39237caf5db9 None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:10:07.825343 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 604/1200] 10.222.0.18 () {56 vars in 1374 bytes} [Tue Sep 13 03:10:07 2022] GET /identity/v3/role_assignments?user.id=00976b80b34c4c1a9da3207b33ff1358&scope.project.id=d707409af2f24c2d9e05aa82e93f57cb&role.id=425d4b481c7846e0bc07e7e805d5b15f => generated 608 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 03:10:17.672437 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=20 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:10:17.822847 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=94 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:11:03.318879 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-960c540c-91b6-4c89-ac11-d2b448bfe0f8 None None] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:11:03.319456 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-960c540c-91b6-4c89-ac11-d2b448bfe0f8 None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:11:03.319456 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-960c540c-91b6-4c89-ac11-d2b448bfe0f8 None None] PATH_INFO: `/` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:11:03.321321 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 597/1201] 10.222.0.18 () {54 vars in 927 bytes} [Tue Sep 13 03:11:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 03:11:03.330442 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05630e4e-4c55-4068-bb83-d98f1b44b26e None None] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:11:03.330442 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05630e4e-4c55-4068-bb83-d98f1b44b26e None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:11:03.330442 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05630e4e-4c55-4068-bb83-d98f1b44b26e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:11:03.347700 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.auth.core [None req-05630e4e-4c55-4068-bb83-d98f1b44b26e None None] MFA Rules not processed for user `49f280ab086b4610855c4ffa378517d9`. Rule list: `[]` (Enabled: `True`). {{(pid=81400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 03:11:03.349131 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-05630e4e-4c55-4068-bb83-d98f1b44b26e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:11:03.454937 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 605/1202] 10.222.0.18 () {58 vars in 1024 bytes} [Tue Sep 13 03:11:03 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 13 03:11:03.738433 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-960c540c-91b6-4c89-ac11-d2b448bfe0f8 None None] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:11:03.873711 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-960c540c-91b6-4c89-ac11-d2b448bfe0f8 None None] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:11:03.921419 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f841a8f1-ff84-42f6-9373-1b005a5e00d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:11:03.922126 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f841a8f1-ff84-42f6-9373-1b005a5e00d6 None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:11:03.922404 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f841a8f1-ff84-42f6-9373-1b005a5e00d6 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:11:03.922762 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f841a8f1-ff84-42f6-9373-1b005a5e00d6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:11:03.944475 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f841a8f1-ff84-42f6-9373-1b005a5e00d6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:11:03.946046 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f841a8f1-ff84-42f6-9373-1b005a5e00d6 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:11:03.962853 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application [None req-f841a8f1-ff84-42f6-9373-1b005a5e00d6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 03:11:03.962853 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 03:11:03.962853 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 03:11:03.962853 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 03:11:03.962853 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 03:11:03.962853 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 03:11:03.962853 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 03:11:03.962853 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 03:11:03.962853 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 03:11:03.962853 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 03:11:03.962853 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 03:11:03.962853 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 03:11:03.962853 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 13 03:11:03.962853 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 13 03:11:03.962853 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 13 03:11:03.962853 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 13 03:11:03.963967 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 03:11:03.963967 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 03:11:03.963967 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 03:11:03.963967 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 03:11:03.963967 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 03:11:03.963967 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 03:11:03.963967 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 03:11:03.963967 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application with Lock( Sep 13 03:11:03.963967 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 03:11:03.963967 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._enter() Sep 13 03:11:03.963967 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 03:11:03.963967 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 03:11:03.963967 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 03:11:03.963967 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.creator() Sep 13 03:11:03.963967 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 03:11:03.963967 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application created_value = creator( Sep 13 03:11:03.963967 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 13 03:11:03.964980 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 13 03:11:03.964980 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 03:11:03.964980 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Sep 13 03:11:03.964980 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 598/1203] 10.222.0.18 () {56 vars in 1108 bytes} [Tue Sep 13 03:11:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 03:11:03.970812 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05630e4e-4c55-4068-bb83-d98f1b44b26e None None] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:11:04.115028 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05630e4e-4c55-4068-bb83-d98f1b44b26e None None] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:11:04.158766 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:11:04.168007 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-111dd349-ca25-4028-8076-f2358bb37a5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:11:04.169782 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-111dd349-ca25-4028-8076-f2358bb37a5c None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:11:04.170468 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-111dd349-ca25-4028-8076-f2358bb37a5c None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:11:04.171504 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-111dd349-ca25-4028-8076-f2358bb37a5c None admin] PATH_INFO: `/v3/domains` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:11:04.172366 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-111dd349-ca25-4028-8076-f2358bb37a5c None admin] RBAC: Adding query filter params (name=Default) {{(pid=81400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 03:11:04.172793 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-111dd349-ca25-4028-8076-f2358bb37a5c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:11:04.175864 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-111dd349-ca25-4028-8076-f2358bb37a5c None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:11:04.194817 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 606/1204] 10.222.0.18 () {56 vars in 1109 bytes} [Tue Sep 13 03:11:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 03:11:04.200081 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f841a8f1-ff84-42f6-9373-1b005a5e00d6 None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:11:04.269470 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:11:04.328798 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f841a8f1-ff84-42f6-9373-1b005a5e00d6 None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:11:04.373646 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45272e9e-212b-4fbe-b93f-1fb911267a13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:11:04.374308 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45272e9e-212b-4fbe-b93f-1fb911267a13 None admin] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:11:04.374578 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45272e9e-212b-4fbe-b93f-1fb911267a13 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:11:04.374874 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45272e9e-212b-4fbe-b93f-1fb911267a13 None admin] PATH_INFO: `/v3/users` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:11:04.375539 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45272e9e-212b-4fbe-b93f-1fb911267a13 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:11:04.377263 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45272e9e-212b-4fbe-b93f-1fb911267a13 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:11:04.405479 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 599/1205] 10.222.0.18 () {60 vars in 1132 bytes} [Tue Sep 13 03:11:04 2022] POST /identity/v3/users => generated 251 bytes in 206 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 03:11:06.815035 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17e3e8ba-7148-4ae7-81c5-96589c585444 None None] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:11:06.815568 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17e3e8ba-7148-4ae7-81c5-96589c585444 None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:11:06.815568 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17e3e8ba-7148-4ae7-81c5-96589c585444 None None] PATH_INFO: `/` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:11:06.817367 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 607/1206] 10.222.0.18 () {54 vars in 927 bytes} [Tue Sep 13 03:11:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 03:11:06.823591 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab4b317-7218-479e-99da-e4eaca6b7700 None None] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:11:06.824280 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab4b317-7218-479e-99da-e4eaca6b7700 None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:11:06.824280 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab4b317-7218-479e-99da-e4eaca6b7700 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:11:06.845480 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.auth.core [None req-0ab4b317-7218-479e-99da-e4eaca6b7700 None None] MFA Rules not processed for user `49f280ab086b4610855c4ffa378517d9`. Rule list: `[]` (Enabled: `True`). {{(pid=81399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 03:11:06.846770 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-0ab4b317-7218-479e-99da-e4eaca6b7700 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:11:06.921679 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 600/1207] 10.222.0.18 () {58 vars in 1024 bytes} [Tue Sep 13 03:11:06 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 13 03:11:07.238351 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17e3e8ba-7148-4ae7-81c5-96589c585444 None None] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:11:07.361520 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17e3e8ba-7148-4ae7-81c5-96589c585444 None None] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:11:07.401875 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35a42b6c-609c-4886-b3e3-7709fe04b96a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:11:07.402502 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35a42b6c-609c-4886-b3e3-7709fe04b96a None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:11:07.403348 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35a42b6c-609c-4886-b3e3-7709fe04b96a None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:11:07.403667 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35a42b6c-609c-4886-b3e3-7709fe04b96a None admin] PATH_INFO: `/v3/roles/service` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:11:07.415411 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35a42b6c-609c-4886-b3e3-7709fe04b96a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 03:11:07.420927 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application [None req-35a42b6c-609c-4886-b3e3-7709fe04b96a None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 13 03:11:07.420927 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 03:11:07.420927 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 03:11:07.420927 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 03:11:07.420927 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 03:11:07.420927 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 03:11:07.420927 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 03:11:07.420927 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 03:11:07.420927 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 03:11:07.420927 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 03:11:07.420927 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 03:11:07.420927 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 03:11:07.420927 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 03:11:07.420927 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 03:11:07.420927 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 03:11:07.420927 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 03:11:07.422251 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 03:11:07.422251 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 03:11:07.422251 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 03:11:07.422251 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 03:11:07.422251 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 03:11:07.422251 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 03:11:07.422251 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 03:11:07.422251 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 03:11:07.422251 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 03:11:07.422251 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 03:11:07.422251 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 03:11:07.422251 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application with Lock( Sep 13 03:11:07.422251 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 03:11:07.422251 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._enter() Sep 13 03:11:07.422251 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 03:11:07.422251 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 03:11:07.423256 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 03:11:07.423256 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.creator() Sep 13 03:11:07.423256 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 03:11:07.423256 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application created_value = creator( Sep 13 03:11:07.423256 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 03:11:07.423256 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 03:11:07.423256 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 03:11:07.423256 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 03:11:07.423256 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 03:11:07.423256 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 03:11:07.423256 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 13 03:11:07.423256 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Sep 13 03:11:07.423256 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 608/1208] 10.222.0.18 () {56 vars in 1102 bytes} [Tue Sep 13 03:11:07 2022] GET /identity/v3/roles/service => generated 85 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 13 03:11:07.428435 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ab4b317-7218-479e-99da-e4eaca6b7700 None None] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:11:07.482026 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:11:07.555488 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ab4b317-7218-479e-99da-e4eaca6b7700 None None] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:11:07.595211 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2045cafa-4a67-446c-9217-4bef1635f8fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:11:07.595831 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2045cafa-4a67-446c-9217-4bef1635f8fb None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:11:07.596139 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2045cafa-4a67-446c-9217-4bef1635f8fb None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:11:07.596411 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2045cafa-4a67-446c-9217-4bef1635f8fb None admin] PATH_INFO: `/v3/roles` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:11:07.597085 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2045cafa-4a67-446c-9217-4bef1635f8fb None admin] RBAC: Adding query filter params (name=service) {{(pid=81399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 03:11:07.597383 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2045cafa-4a67-446c-9217-4bef1635f8fb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:11:07.599192 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2045cafa-4a67-446c-9217-4bef1635f8fb None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:11:07.608636 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 601/1209] 10.222.0.18 () {56 vars in 1103 bytes} [Tue Sep 13 03:11:07 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 03:11:07.615234 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35a42b6c-609c-4886-b3e3-7709fe04b96a None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:11:07.671281 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:11:07.727257 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35a42b6c-609c-4886-b3e3-7709fe04b96a None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:11:07.770938 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03d36e00-7c3c-400d-be45-b1aa93d83349 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:11:07.771724 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03d36e00-7c3c-400d-be45-b1aa93d83349 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:11:07.772028 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03d36e00-7c3c-400d-be45-b1aa93d83349 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:11:07.772350 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03d36e00-7c3c-400d-be45-b1aa93d83349 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:11:07.784621 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03d36e00-7c3c-400d-be45-b1aa93d83349 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:11:07.786414 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03d36e00-7c3c-400d-be45-b1aa93d83349 None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:11:07.807525 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application [None req-03d36e00-7c3c-400d-be45-b1aa93d83349 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 03:11:07.807525 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 03:11:07.807525 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 03:11:07.807525 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 03:11:07.807525 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 03:11:07.807525 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 03:11:07.807525 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 03:11:07.807525 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 03:11:07.807525 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 03:11:07.807525 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 03:11:07.807525 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 03:11:07.807525 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 03:11:07.807525 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 13 03:11:07.807525 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 13 03:11:07.807525 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 13 03:11:07.807525 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 13 03:11:07.808749 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 03:11:07.808749 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 03:11:07.808749 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 03:11:07.808749 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 03:11:07.808749 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 03:11:07.808749 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 03:11:07.808749 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 03:11:07.808749 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application with Lock( Sep 13 03:11:07.808749 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 03:11:07.808749 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._enter() Sep 13 03:11:07.808749 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 03:11:07.808749 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 03:11:07.808749 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 03:11:07.808749 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.creator() Sep 13 03:11:07.808749 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 03:11:07.808749 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application created_value = creator( Sep 13 03:11:07.808749 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 13 03:11:07.810386 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 13 03:11:07.810386 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 03:11:07.810386 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Sep 13 03:11:07.811016 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 609/1210] 10.222.0.18 () {56 vars in 1108 bytes} [Tue Sep 13 03:11:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 03:11:07.817294 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2045cafa-4a67-446c-9217-4bef1635f8fb None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:11:07.926646 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:11:07.934663 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2045cafa-4a67-446c-9217-4bef1635f8fb None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:11:07.971742 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c9956d6-c962-4bcf-8e9c-ecf61a08dd5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:11:07.972460 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c9956d6-c962-4bcf-8e9c-ecf61a08dd5d None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:11:07.972809 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c9956d6-c962-4bcf-8e9c-ecf61a08dd5d None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:11:07.973150 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c9956d6-c962-4bcf-8e9c-ecf61a08dd5d None admin] PATH_INFO: `/v3/domains` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:11:07.973874 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c9956d6-c962-4bcf-8e9c-ecf61a08dd5d None admin] RBAC: Adding query filter params (name=Default) {{(pid=81399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 03:11:07.974248 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c9956d6-c962-4bcf-8e9c-ecf61a08dd5d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:11:07.975735 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c9956d6-c962-4bcf-8e9c-ecf61a08dd5d None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:11:07.991659 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 602/1211] 10.222.0.18 () {56 vars in 1109 bytes} [Tue Sep 13 03:11:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 03:11:07.999047 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03d36e00-7c3c-400d-be45-b1aa93d83349 None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:11:08.120069 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03d36e00-7c3c-400d-be45-b1aa93d83349 None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:11:08.122255 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:11:08.158124 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2098a78a-38f6-42d6-a1c8-12b75ed45763 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:11:08.158794 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2098a78a-38f6-42d6-a1c8-12b75ed45763 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:11:08.159073 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2098a78a-38f6-42d6-a1c8-12b75ed45763 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:11:08.159346 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2098a78a-38f6-42d6-a1c8-12b75ed45763 None admin] PATH_INFO: `/v3/users/placement` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:11:08.167387 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2098a78a-38f6-42d6-a1c8-12b75ed45763 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:11:08.169834 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2098a78a-38f6-42d6-a1c8-12b75ed45763 None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:11:08.191320 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application [None req-2098a78a-38f6-42d6-a1c8-12b75ed45763 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Sep 13 03:11:08.191320 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 03:11:08.191320 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 03:11:08.191320 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 03:11:08.191320 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 03:11:08.191320 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 03:11:08.191320 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 03:11:08.191320 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 03:11:08.191320 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 03:11:08.191320 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 03:11:08.191320 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 03:11:08.191320 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 03:11:08.191320 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 13 03:11:08.191320 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 13 03:11:08.191320 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 13 03:11:08.191320 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 13 03:11:08.192674 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 03:11:08.192674 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 03:11:08.192674 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 13 03:11:08.192674 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 13 03:11:08.192674 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 13 03:11:08.192674 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 13 03:11:08.192674 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 03:11:08.192674 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 03:11:08.192674 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 03:11:08.192674 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 03:11:08.192674 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 03:11:08.192674 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application with Lock( Sep 13 03:11:08.192674 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 03:11:08.192674 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._enter() Sep 13 03:11:08.192674 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 03:11:08.192674 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 03:11:08.192674 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 03:11:08.192674 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.creator() Sep 13 03:11:08.193962 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 03:11:08.193962 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application created_value = creator( Sep 13 03:11:08.193962 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 13 03:11:08.193962 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 13 03:11:08.193962 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 13 03:11:08.193962 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 13 03:11:08.193962 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 13 03:11:08.193962 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 13 03:11:08.193962 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Sep 13 03:11:08.193962 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Sep 13 03:11:08.193962 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 610/1212] 10.222.0.18 () {56 vars in 1108 bytes} [Tue Sep 13 03:11:07 2022] GET /identity/v3/users/placement => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 03:11:08.194740 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c9956d6-c962-4bcf-8e9c-ecf61a08dd5d None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:11:08.306192 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c9956d6-c962-4bcf-8e9c-ecf61a08dd5d None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:11:08.342989 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1276ec3-1892-4f05-9197-5a84bc1987cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:11:08.344245 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1276ec3-1892-4f05-9197-5a84bc1987cf None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:11:08.344245 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1276ec3-1892-4f05-9197-5a84bc1987cf None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:11:08.344245 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1276ec3-1892-4f05-9197-5a84bc1987cf None admin] PATH_INFO: `/v3/users` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:11:08.344902 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1276ec3-1892-4f05-9197-5a84bc1987cf None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 03:11:08.345187 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1276ec3-1892-4f05-9197-5a84bc1987cf None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:11:08.346674 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1276ec3-1892-4f05-9197-5a84bc1987cf None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:11:08.364630 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 603/1213] 10.222.0.18 () {56 vars in 1143 bytes} [Tue Sep 13 03:11:08 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 03:11:08.370299 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2098a78a-38f6-42d6-a1c8-12b75ed45763 None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:11:08.479211 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2098a78a-38f6-42d6-a1c8-12b75ed45763 None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:11:08.514334 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11a2883d-9fc9-4ad0-961b-99c0a3122672 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:11:08.515498 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a2883d-9fc9-4ad0-961b-99c0a3122672 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:11:08.515498 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a2883d-9fc9-4ad0-961b-99c0a3122672 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:11:08.515658 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a2883d-9fc9-4ad0-961b-99c0a3122672 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:11:08.554584 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:11:08.594133 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11a2883d-9fc9-4ad0-961b-99c0a3122672 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:11:08.595753 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11a2883d-9fc9-4ad0-961b-99c0a3122672 None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:11:08.709807 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 611/1214] 10.222.0.18 () {58 vars in 1282 bytes} [Tue Sep 13 03:11:08 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 340 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 13 03:11:08.716477 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1276ec3-1892-4f05-9197-5a84bc1987cf None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:11:08.742580 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:11:08.842231 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1276ec3-1892-4f05-9197-5a84bc1987cf None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:11:08.877251 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1ee9935-28fd-4766-934b-c7a7502dfbf0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:11:08.877978 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ee9935-28fd-4766-934b-c7a7502dfbf0 None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:11:08.878321 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ee9935-28fd-4766-934b-c7a7502dfbf0 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:11:08.878713 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ee9935-28fd-4766-934b-c7a7502dfbf0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:11:08.891985 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1ee9935-28fd-4766-934b-c7a7502dfbf0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:11:08.893576 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1ee9935-28fd-4766-934b-c7a7502dfbf0 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:11:08.911370 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application [None req-b1ee9935-28fd-4766-934b-c7a7502dfbf0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 03:11:08.911370 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 03:11:08.911370 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 03:11:08.911370 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 03:11:08.911370 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 03:11:08.911370 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 03:11:08.911370 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 03:11:08.911370 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 03:11:08.911370 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 03:11:08.911370 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 03:11:08.911370 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 03:11:08.911370 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 03:11:08.911370 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 13 03:11:08.911370 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 13 03:11:08.911370 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 13 03:11:08.911370 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 13 03:11:08.912412 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 03:11:08.912412 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 03:11:08.912412 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 03:11:08.912412 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 03:11:08.912412 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 03:11:08.912412 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 03:11:08.912412 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 03:11:08.912412 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application with Lock( Sep 13 03:11:08.912412 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 03:11:08.912412 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._enter() Sep 13 03:11:08.912412 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 03:11:08.912412 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 03:11:08.912412 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 03:11:08.912412 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.creator() Sep 13 03:11:08.912412 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 03:11:08.912412 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application created_value = creator( Sep 13 03:11:08.912412 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 13 03:11:08.914040 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 13 03:11:08.914040 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 03:11:08.914040 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Sep 13 03:11:08.914040 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 604/1215] 10.222.0.18 () {56 vars in 1108 bytes} [Tue Sep 13 03:11:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 03:11:08.918477 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11a2883d-9fc9-4ad0-961b-99c0a3122672 None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:11:08.985293 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:11:09.031351 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11a2883d-9fc9-4ad0-961b-99c0a3122672 None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:11:09.089094 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c169f2af-9356-427e-9b7a-29cc1a355217 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:11:09.089783 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c169f2af-9356-427e-9b7a-29cc1a355217 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:11:09.090187 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c169f2af-9356-427e-9b7a-29cc1a355217 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:11:09.091595 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c169f2af-9356-427e-9b7a-29cc1a355217 None admin] PATH_INFO: `/v3/domains` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:11:09.091769 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c169f2af-9356-427e-9b7a-29cc1a355217 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 03:11:09.092082 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c169f2af-9356-427e-9b7a-29cc1a355217 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:11:09.093548 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c169f2af-9356-427e-9b7a-29cc1a355217 None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:11:09.105930 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 612/1216] 10.222.0.18 () {56 vars in 1109 bytes} [Tue Sep 13 03:11:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 03:11:09.111615 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1ee9935-28fd-4766-934b-c7a7502dfbf0 None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:11:09.197801 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:11:09.239377 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1ee9935-28fd-4766-934b-c7a7502dfbf0 None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:11:09.283884 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-803cc92a-c29e-4593-b5eb-9804f3023706 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:11:09.284346 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-803cc92a-c29e-4593-b5eb-9804f3023706 None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:11:09.284651 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-803cc92a-c29e-4593-b5eb-9804f3023706 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:11:09.284965 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-803cc92a-c29e-4593-b5eb-9804f3023706 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:11:09.292565 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-803cc92a-c29e-4593-b5eb-9804f3023706 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:11:09.295001 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-803cc92a-c29e-4593-b5eb-9804f3023706 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:11:09.309488 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application [None req-803cc92a-c29e-4593-b5eb-9804f3023706 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 13 03:11:09.309488 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 03:11:09.309488 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 03:11:09.309488 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 03:11:09.309488 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 03:11:09.309488 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 03:11:09.309488 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 03:11:09.309488 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 03:11:09.309488 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 03:11:09.309488 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 03:11:09.309488 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 03:11:09.309488 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 03:11:09.309488 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 13 03:11:09.309488 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 13 03:11:09.309488 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 13 03:11:09.310669 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 13 03:11:09.310669 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 03:11:09.310669 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 03:11:09.310669 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 03:11:09.310669 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 03:11:09.310669 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 03:11:09.310669 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 03:11:09.310669 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 03:11:09.310669 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application with Lock( Sep 13 03:11:09.310669 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 03:11:09.310669 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._enter() Sep 13 03:11:09.310669 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 03:11:09.310669 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 03:11:09.310669 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 03:11:09.310669 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.creator() Sep 13 03:11:09.310669 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 03:11:09.310669 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application created_value = creator( Sep 13 03:11:09.311621 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 13 03:11:09.311621 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 13 03:11:09.311621 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 13 03:11:09.311621 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 13 03:11:09.311621 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 13 03:11:09.311621 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 13 03:11:09.311621 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 13 03:11:09.311621 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Sep 13 03:11:09.311621 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 605/1217] 10.222.0.18 () {56 vars in 1111 bytes} [Tue Sep 13 03:11:09 2022] GET /identity/v3/projects/service => generated 88 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 13 03:11:09.316802 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c169f2af-9356-427e-9b7a-29cc1a355217 None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:11:09.435510 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c169f2af-9356-427e-9b7a-29cc1a355217 None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:11:09.460525 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:11:09.473936 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83d8d821-7118-4572-8455-c3e8bafeb497 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:11:09.474599 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d8d821-7118-4572-8455-c3e8bafeb497 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:11:09.474904 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d8d821-7118-4572-8455-c3e8bafeb497 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:11:09.475178 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d8d821-7118-4572-8455-c3e8bafeb497 None admin] PATH_INFO: `/v3/projects` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:11:09.475957 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83d8d821-7118-4572-8455-c3e8bafeb497 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 03:11:09.476277 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83d8d821-7118-4572-8455-c3e8bafeb497 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:11:09.477743 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83d8d821-7118-4572-8455-c3e8bafeb497 None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:11:09.490899 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 613/1218] 10.222.0.18 () {56 vars in 1148 bytes} [Tue Sep 13 03:11:09 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 03:11:09.497090 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-803cc92a-c29e-4593-b5eb-9804f3023706 None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:11:09.606159 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-803cc92a-c29e-4593-b5eb-9804f3023706 None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:11:09.641419 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:11:09.647520 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9abeb62f-248d-4f2c-b9c8-d1136f2479e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:11:09.648126 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9abeb62f-248d-4f2c-b9c8-d1136f2479e3 None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:11:09.648404 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9abeb62f-248d-4f2c-b9c8-d1136f2479e3 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:11:09.648671 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9abeb62f-248d-4f2c-b9c8-d1136f2479e3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:11:09.649395 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9abeb62f-248d-4f2c-b9c8-d1136f2479e3 None admin] RBAC: Adding query filter params (role.id=817d6837cd044890b41cf16b303a092b, scope.project.id=b9d94f6a68f7470d9494f112175620dc, user.id=4b99f1b2374d4d2f8db78306316180d9) {{(pid=81399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 03:11:09.649689 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9abeb62f-248d-4f2c-b9c8-d1136f2479e3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:11:09.651181 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9abeb62f-248d-4f2c-b9c8-d1136f2479e3 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:11:09.657951 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 606/1219] 10.222.0.18 () {56 vars in 1374 bytes} [Tue Sep 13 03:11:09 2022] GET /identity/v3/role_assignments?user.id=4b99f1b2374d4d2f8db78306316180d9&scope.project.id=b9d94f6a68f7470d9494f112175620dc&role.id=817d6837cd044890b41cf16b303a092b => generated 259 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 03:11:11.944581 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03dd4cec-446b-410c-b617-d57c77f277eb None None] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:11:11.945161 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03dd4cec-446b-410c-b617-d57c77f277eb None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:11:11.945309 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03dd4cec-446b-410c-b617-d57c77f277eb None None] PATH_INFO: `/` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:11:11.947012 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 614/1220] 10.222.0.18 () {54 vars in 927 bytes} [Tue Sep 13 03:11:11 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 03:11:11.954024 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b89c920c-0ce6-48b7-a9fd-19421a9f6257 None None] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:11:11.954493 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b89c920c-0ce6-48b7-a9fd-19421a9f6257 None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:11:11.954716 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b89c920c-0ce6-48b7-a9fd-19421a9f6257 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:11:11.976603 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.auth.core [None req-b89c920c-0ce6-48b7-a9fd-19421a9f6257 None None] MFA Rules not processed for user `49f280ab086b4610855c4ffa378517d9`. Rule list: `[]` (Enabled: `True`). {{(pid=81399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 03:11:11.977819 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-b89c920c-0ce6-48b7-a9fd-19421a9f6257 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:11:12.066826 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 607/1221] 10.222.0.18 () {58 vars in 1024 bytes} [Tue Sep 13 03:11:11 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 13 03:11:12.362341 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03dd4cec-446b-410c-b617-d57c77f277eb None None] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:11:12.482032 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03dd4cec-446b-410c-b617-d57c77f277eb None None] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:11:12.516996 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc7bbdd3-0fdf-4cea-9604-3b3598cbd52d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:11:12.517722 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc7bbdd3-0fdf-4cea-9604-3b3598cbd52d None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:11:12.518050 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc7bbdd3-0fdf-4cea-9604-3b3598cbd52d None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:11:12.518333 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc7bbdd3-0fdf-4cea-9604-3b3598cbd52d None admin] PATH_INFO: `/v3/roles/service` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:11:12.531245 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc7bbdd3-0fdf-4cea-9604-3b3598cbd52d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 03:11:12.536921 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application [None req-cc7bbdd3-0fdf-4cea-9604-3b3598cbd52d None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 13 03:11:12.536921 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 03:11:12.536921 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 03:11:12.536921 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 03:11:12.536921 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 03:11:12.536921 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 03:11:12.536921 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 03:11:12.536921 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 03:11:12.536921 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 03:11:12.536921 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 03:11:12.536921 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 03:11:12.536921 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 03:11:12.536921 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 03:11:12.536921 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 03:11:12.536921 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 03:11:12.536921 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 03:11:12.538059 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 03:11:12.538059 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 03:11:12.538059 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 03:11:12.538059 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 03:11:12.538059 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 03:11:12.538059 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 03:11:12.538059 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 03:11:12.538059 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 03:11:12.538059 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 03:11:12.538059 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 03:11:12.538059 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 03:11:12.538059 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application with Lock( Sep 13 03:11:12.538059 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 03:11:12.538059 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._enter() Sep 13 03:11:12.538059 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 03:11:12.538059 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 03:11:12.539060 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 03:11:12.539060 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.creator() Sep 13 03:11:12.539060 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 03:11:12.539060 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application created_value = creator( Sep 13 03:11:12.539060 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 03:11:12.539060 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 03:11:12.539060 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 03:11:12.539060 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 03:11:12.539060 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 03:11:12.539060 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 03:11:12.539060 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 13 03:11:12.539060 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Sep 13 03:11:12.539060 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 615/1222] 10.222.0.18 () {56 vars in 1102 bytes} [Tue Sep 13 03:11:12 2022] GET /identity/v3/roles/service => generated 85 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 03:11:12.546003 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b89c920c-0ce6-48b7-a9fd-19421a9f6257 None None] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:11:12.654736 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b89c920c-0ce6-48b7-a9fd-19421a9f6257 None None] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:11:12.680453 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:11:12.693019 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-230d06ff-facd-45c4-a655-acb8056ede82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:11:12.693792 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-230d06ff-facd-45c4-a655-acb8056ede82 None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:11:12.694181 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-230d06ff-facd-45c4-a655-acb8056ede82 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:11:12.694462 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-230d06ff-facd-45c4-a655-acb8056ede82 None admin] PATH_INFO: `/v3/roles` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:11:12.695176 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-230d06ff-facd-45c4-a655-acb8056ede82 None admin] RBAC: Adding query filter params (name=service) {{(pid=81399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 03:11:12.695478 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-230d06ff-facd-45c4-a655-acb8056ede82 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:11:12.697063 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-230d06ff-facd-45c4-a655-acb8056ede82 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:11:12.706323 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 608/1223] 10.222.0.18 () {56 vars in 1103 bytes} [Tue Sep 13 03:11:12 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 03:11:12.712283 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc7bbdd3-0fdf-4cea-9604-3b3598cbd52d None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:11:12.771824 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:11:12.846162 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc7bbdd3-0fdf-4cea-9604-3b3598cbd52d None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:11:12.885705 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43216a5b-0c0f-4ac4-873f-efb266d5544b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:11:12.886497 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43216a5b-0c0f-4ac4-873f-efb266d5544b None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:11:12.886906 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43216a5b-0c0f-4ac4-873f-efb266d5544b None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:11:12.887221 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43216a5b-0c0f-4ac4-873f-efb266d5544b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:11:12.898209 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43216a5b-0c0f-4ac4-873f-efb266d5544b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:11:12.899746 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43216a5b-0c0f-4ac4-873f-efb266d5544b None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:11:12.915571 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application [None req-43216a5b-0c0f-4ac4-873f-efb266d5544b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 03:11:12.915571 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 03:11:12.915571 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 03:11:12.915571 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 03:11:12.915571 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 03:11:12.915571 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 03:11:12.915571 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 03:11:12.915571 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 03:11:12.915571 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 03:11:12.915571 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 03:11:12.915571 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 03:11:12.915571 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 03:11:12.915571 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 13 03:11:12.915571 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 13 03:11:12.915571 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 13 03:11:12.915571 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 13 03:11:12.916712 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 03:11:12.916712 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 03:11:12.916712 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 03:11:12.916712 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 03:11:12.916712 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 03:11:12.916712 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 03:11:12.916712 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 03:11:12.916712 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application with Lock( Sep 13 03:11:12.916712 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 03:11:12.916712 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._enter() Sep 13 03:11:12.916712 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 03:11:12.916712 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 03:11:12.916712 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 03:11:12.916712 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.creator() Sep 13 03:11:12.916712 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 03:11:12.916712 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application created_value = creator( Sep 13 03:11:12.916712 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 13 03:11:12.917636 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 13 03:11:12.917636 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 03:11:12.917636 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Sep 13 03:11:12.917636 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 616/1224] 10.222.0.18 () {56 vars in 1108 bytes} [Tue Sep 13 03:11:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 03:11:12.923271 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-230d06ff-facd-45c4-a655-acb8056ede82 None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:11:13.032894 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-230d06ff-facd-45c4-a655-acb8056ede82 None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:11:13.071789 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfed8cf9-ab36-4aae-8319-6446e291102e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:11:13.072462 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfed8cf9-ab36-4aae-8319-6446e291102e None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:11:13.072712 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfed8cf9-ab36-4aae-8319-6446e291102e None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:11:13.073029 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfed8cf9-ab36-4aae-8319-6446e291102e None admin] PATH_INFO: `/v3/domains` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:11:13.073663 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfed8cf9-ab36-4aae-8319-6446e291102e None admin] RBAC: Adding query filter params (name=Default) {{(pid=81399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 03:11:13.073963 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfed8cf9-ab36-4aae-8319-6446e291102e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:11:13.075382 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfed8cf9-ab36-4aae-8319-6446e291102e None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:11:13.087921 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 609/1225] 10.222.0.18 () {56 vars in 1109 bytes} [Tue Sep 13 03:11:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 03:11:13.094236 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43216a5b-0c0f-4ac4-873f-efb266d5544b None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:11:13.195178 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:11:13.203953 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43216a5b-0c0f-4ac4-873f-efb266d5544b None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:11:13.243194 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d0b90e8-5992-4470-a460-ee0d2be70557 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:11:13.243820 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d0b90e8-5992-4470-a460-ee0d2be70557 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:11:13.244084 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d0b90e8-5992-4470-a460-ee0d2be70557 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:11:13.244386 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d0b90e8-5992-4470-a460-ee0d2be70557 None admin] PATH_INFO: `/v3/users/placement` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:11:13.252641 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d0b90e8-5992-4470-a460-ee0d2be70557 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:11:13.254207 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d0b90e8-5992-4470-a460-ee0d2be70557 None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:11:13.267070 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application [None req-4d0b90e8-5992-4470-a460-ee0d2be70557 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Sep 13 03:11:13.267070 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 03:11:13.267070 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 03:11:13.267070 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 03:11:13.267070 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 03:11:13.267070 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 03:11:13.267070 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 03:11:13.267070 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 03:11:13.267070 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 03:11:13.267070 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 03:11:13.267070 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 03:11:13.267070 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 03:11:13.267070 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 13 03:11:13.267070 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 13 03:11:13.267070 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 13 03:11:13.267070 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 13 03:11:13.274044 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 03:11:13.274044 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 03:11:13.274044 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 13 03:11:13.274044 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 13 03:11:13.274044 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 13 03:11:13.274044 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 13 03:11:13.274044 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 03:11:13.274044 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 03:11:13.274044 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 03:11:13.274044 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 03:11:13.274044 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 03:11:13.274044 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application with Lock( Sep 13 03:11:13.274044 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 03:11:13.274044 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._enter() Sep 13 03:11:13.274044 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 03:11:13.274044 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 03:11:13.274044 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 03:11:13.274044 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.creator() Sep 13 03:11:13.275848 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 03:11:13.275848 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application created_value = creator( Sep 13 03:11:13.275848 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 13 03:11:13.275848 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 13 03:11:13.275848 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 13 03:11:13.275848 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 13 03:11:13.275848 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 13 03:11:13.275848 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 13 03:11:13.275848 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Sep 13 03:11:13.275848 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Sep 13 03:11:13.275848 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 617/1226] 10.222.0.18 () {56 vars in 1108 bytes} [Tue Sep 13 03:11:13 2022] GET /identity/v3/users/placement => generated 87 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 03:11:13.276872 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfed8cf9-ab36-4aae-8319-6446e291102e None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:11:13.328966 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:11:13.388746 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfed8cf9-ab36-4aae-8319-6446e291102e None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:11:13.427033 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70b4d1c3-99ee-473d-a3bb-07008de375c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:11:13.427899 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b4d1c3-99ee-473d-a3bb-07008de375c9 None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:11:13.428180 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b4d1c3-99ee-473d-a3bb-07008de375c9 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:11:13.428453 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b4d1c3-99ee-473d-a3bb-07008de375c9 None admin] PATH_INFO: `/v3/users` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:11:13.429168 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70b4d1c3-99ee-473d-a3bb-07008de375c9 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 03:11:13.429472 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70b4d1c3-99ee-473d-a3bb-07008de375c9 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:11:13.430916 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70b4d1c3-99ee-473d-a3bb-07008de375c9 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:11:13.449075 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 610/1227] 10.222.0.18 () {56 vars in 1143 bytes} [Tue Sep 13 03:11:13 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 03:11:13.455680 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d0b90e8-5992-4470-a460-ee0d2be70557 None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:11:13.563587 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d0b90e8-5992-4470-a460-ee0d2be70557 None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:11:13.602339 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9410e4af-70a9-4f1d-8ab9-cfe9fb07f290 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:11:13.602957 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9410e4af-70a9-4f1d-8ab9-cfe9fb07f290 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:11:13.603286 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9410e4af-70a9-4f1d-8ab9-cfe9fb07f290 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:11:13.603542 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9410e4af-70a9-4f1d-8ab9-cfe9fb07f290 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:11:13.614876 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9410e4af-70a9-4f1d-8ab9-cfe9fb07f290 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:11:13.616474 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9410e4af-70a9-4f1d-8ab9-cfe9fb07f290 None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:11:13.631792 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application [None req-9410e4af-70a9-4f1d-8ab9-cfe9fb07f290 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 03:11:13.631792 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 03:11:13.631792 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 03:11:13.631792 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 03:11:13.631792 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 03:11:13.631792 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 03:11:13.631792 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 03:11:13.631792 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 03:11:13.631792 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 03:11:13.631792 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 03:11:13.631792 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 03:11:13.631792 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 03:11:13.631792 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 13 03:11:13.631792 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 13 03:11:13.631792 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 13 03:11:13.631792 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 13 03:11:13.632977 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 03:11:13.632977 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 03:11:13.632977 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 03:11:13.632977 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 03:11:13.632977 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 03:11:13.632977 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 03:11:13.632977 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 03:11:13.632977 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application with Lock( Sep 13 03:11:13.632977 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 03:11:13.632977 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._enter() Sep 13 03:11:13.632977 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 03:11:13.632977 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 03:11:13.632977 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 03:11:13.632977 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.creator() Sep 13 03:11:13.632977 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 03:11:13.632977 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application created_value = creator( Sep 13 03:11:13.632977 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 13 03:11:13.634080 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 13 03:11:13.634080 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 03:11:13.634080 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Sep 13 03:11:13.634080 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 618/1228] 10.222.0.18 () {56 vars in 1108 bytes} [Tue Sep 13 03:11:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 13 03:11:13.639366 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70b4d1c3-99ee-473d-a3bb-07008de375c9 None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:11:13.745345 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70b4d1c3-99ee-473d-a3bb-07008de375c9 None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:11:13.748672 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:11:13.790378 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91cdfcb2-89a6-498c-8420-1be33ff21cf8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:11:13.791019 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91cdfcb2-89a6-498c-8420-1be33ff21cf8 None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:11:13.791302 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91cdfcb2-89a6-498c-8420-1be33ff21cf8 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:11:13.791581 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91cdfcb2-89a6-498c-8420-1be33ff21cf8 None admin] PATH_INFO: `/v3/domains` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:11:13.792241 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91cdfcb2-89a6-498c-8420-1be33ff21cf8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 03:11:13.792532 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91cdfcb2-89a6-498c-8420-1be33ff21cf8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:11:13.793918 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91cdfcb2-89a6-498c-8420-1be33ff21cf8 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:11:13.805763 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 611/1229] 10.222.0.18 () {56 vars in 1109 bytes} [Tue Sep 13 03:11:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 03:11:13.811895 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9410e4af-70a9-4f1d-8ab9-cfe9fb07f290 None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:11:13.821760 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:11:13.921802 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9410e4af-70a9-4f1d-8ab9-cfe9fb07f290 None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:11:13.964534 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58ff8422-0a66-4bbc-9a3a-a0705ff74299 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:11:13.965199 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58ff8422-0a66-4bbc-9a3a-a0705ff74299 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:11:13.965517 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58ff8422-0a66-4bbc-9a3a-a0705ff74299 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:11:13.965828 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58ff8422-0a66-4bbc-9a3a-a0705ff74299 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:11:13.972790 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58ff8422-0a66-4bbc-9a3a-a0705ff74299 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:11:13.974237 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58ff8422-0a66-4bbc-9a3a-a0705ff74299 None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:11:13.986175 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application [None req-58ff8422-0a66-4bbc-9a3a-a0705ff74299 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 13 03:11:13.986175 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 03:11:13.986175 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 03:11:13.986175 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 03:11:13.986175 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 03:11:13.986175 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 03:11:13.986175 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 03:11:13.986175 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 03:11:13.986175 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 03:11:13.986175 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 03:11:13.986175 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 03:11:13.986175 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 03:11:13.986175 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 13 03:11:13.986175 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 13 03:11:13.986175 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 13 03:11:13.987396 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 13 03:11:13.987396 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 03:11:13.987396 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 03:11:13.987396 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 03:11:13.987396 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 03:11:13.987396 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 03:11:13.987396 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 03:11:13.987396 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 03:11:13.987396 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application with Lock( Sep 13 03:11:13.987396 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 03:11:13.987396 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._enter() Sep 13 03:11:13.987396 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 03:11:13.987396 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 03:11:13.987396 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 03:11:13.987396 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.creator() Sep 13 03:11:13.987396 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 03:11:13.987396 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application created_value = creator( Sep 13 03:11:13.988337 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 13 03:11:13.988337 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 13 03:11:13.988337 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 13 03:11:13.988337 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 13 03:11:13.988337 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 13 03:11:13.988337 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 13 03:11:13.988337 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 13 03:11:13.988337 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Sep 13 03:11:13.988337 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 619/1230] 10.222.0.18 () {56 vars in 1111 bytes} [Tue Sep 13 03:11:13 2022] GET /identity/v3/projects/service => generated 88 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 03:11:13.993325 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91cdfcb2-89a6-498c-8420-1be33ff21cf8 None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:11:14.104789 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91cdfcb2-89a6-498c-8420-1be33ff21cf8 None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:11:14.142779 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fda80a87-94ab-4d59-9baf-e1ef9ff44dc4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:11:14.143377 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda80a87-94ab-4d59-9baf-e1ef9ff44dc4 None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:11:14.143651 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda80a87-94ab-4d59-9baf-e1ef9ff44dc4 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:11:14.143915 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda80a87-94ab-4d59-9baf-e1ef9ff44dc4 None admin] PATH_INFO: `/v3/projects` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:11:14.144588 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fda80a87-94ab-4d59-9baf-e1ef9ff44dc4 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 03:11:14.144885 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fda80a87-94ab-4d59-9baf-e1ef9ff44dc4 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:11:14.146289 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fda80a87-94ab-4d59-9baf-e1ef9ff44dc4 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:11:14.158027 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 612/1231] 10.222.0.18 () {56 vars in 1148 bytes} [Tue Sep 13 03:11:13 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 03:11:14.164275 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58ff8422-0a66-4bbc-9a3a-a0705ff74299 None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:11:14.236934 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:11:14.277261 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58ff8422-0a66-4bbc-9a3a-a0705ff74299 None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:11:14.314836 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13c7a7a2-8ca4-4d00-ba0c-0aa199cef578 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:11:14.315762 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13c7a7a2-8ca4-4d00-ba0c-0aa199cef578 None admin] REQUEST_METHOD: `PUT` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:11:14.316174 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13c7a7a2-8ca4-4d00-ba0c-0aa199cef578 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:11:14.316448 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13c7a7a2-8ca4-4d00-ba0c-0aa199cef578 None admin] PATH_INFO: `/v3/projects/b9d94f6a68f7470d9494f112175620dc/users/4b99f1b2374d4d2f8db78306316180d9/roles/817d6837cd044890b41cf16b303a092b` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:11:14.332691 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13c7a7a2-8ca4-4d00-ba0c-0aa199cef578 None admin] RBAC: Authorizing `identity:create_grant(project_id=b9d94f6a68f7470d9494f112175620dc, user_id=4b99f1b2374d4d2f8db78306316180d9, role_id=817d6837cd044890b41cf16b303a092b)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:11:14.334039 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13c7a7a2-8ca4-4d00-ba0c-0aa199cef578 None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:11:14.347346 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 620/1232] 10.222.0.18 () {58 vars in 1439 bytes} [Tue Sep 13 03:11:14 2022] PUT /identity/v3/projects/b9d94f6a68f7470d9494f112175620dc/users/4b99f1b2374d4d2f8db78306316180d9/roles/817d6837cd044890b41cf16b303a092b => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 03:11:16.595295 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57b1874b-6c05-44c8-9d24-09ab71ed3c87 None None] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:11:16.595868 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57b1874b-6c05-44c8-9d24-09ab71ed3c87 None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:11:16.595868 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57b1874b-6c05-44c8-9d24-09ab71ed3c87 None None] PATH_INFO: `/` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:11:16.597462 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 613/1233] 10.222.0.18 () {54 vars in 927 bytes} [Tue Sep 13 03:11:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 03:11:16.603587 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b799e7-fed0-415b-aeb9-8db6fdafc194 None None] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:11:16.604112 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b799e7-fed0-415b-aeb9-8db6fdafc194 None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:11:16.604211 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b799e7-fed0-415b-aeb9-8db6fdafc194 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:11:16.625874 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.auth.core [None req-55b799e7-fed0-415b-aeb9-8db6fdafc194 None None] MFA Rules not processed for user `49f280ab086b4610855c4ffa378517d9`. Rule list: `[]` (Enabled: `True`). {{(pid=81400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 03:11:16.627101 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-55b799e7-fed0-415b-aeb9-8db6fdafc194 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:11:16.700768 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 621/1234] 10.222.0.18 () {58 vars in 1024 bytes} [Tue Sep 13 03:11:16 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 13 03:11:16.988158 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57b1874b-6c05-44c8-9d24-09ab71ed3c87 None None] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:11:17.015967 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:11:17.108520 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57b1874b-6c05-44c8-9d24-09ab71ed3c87 None None] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:11:17.148664 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-629715bc-900f-4c52-aee2-7c10115b8d65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:11:17.149323 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-629715bc-900f-4c52-aee2-7c10115b8d65 None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:11:17.149600 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-629715bc-900f-4c52-aee2-7c10115b8d65 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:11:17.150045 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-629715bc-900f-4c52-aee2-7c10115b8d65 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:11:17.160964 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-629715bc-900f-4c52-aee2-7c10115b8d65 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 03:11:17.165852 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application [None req-629715bc-900f-4c52-aee2-7c10115b8d65 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 13 03:11:17.165852 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 03:11:17.165852 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 03:11:17.165852 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 03:11:17.165852 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 03:11:17.165852 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 03:11:17.165852 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 03:11:17.165852 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 03:11:17.165852 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 03:11:17.165852 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 03:11:17.165852 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 03:11:17.165852 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 03:11:17.165852 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 03:11:17.165852 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 03:11:17.165852 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 03:11:17.165852 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 03:11:17.170704 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 03:11:17.170704 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 03:11:17.170704 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 03:11:17.170704 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 03:11:17.170704 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 03:11:17.170704 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 03:11:17.170704 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 03:11:17.170704 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 03:11:17.170704 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 03:11:17.170704 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 03:11:17.170704 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 03:11:17.170704 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application with Lock( Sep 13 03:11:17.170704 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 03:11:17.170704 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._enter() Sep 13 03:11:17.170704 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 03:11:17.170704 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 03:11:17.171906 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 03:11:17.171906 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.creator() Sep 13 03:11:17.171906 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 03:11:17.171906 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application created_value = creator( Sep 13 03:11:17.171906 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 03:11:17.171906 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 03:11:17.171906 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 03:11:17.171906 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 03:11:17.171906 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 03:11:17.171906 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 03:11:17.171906 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 13 03:11:17.171906 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Sep 13 03:11:17.171906 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 614/1235] 10.222.0.18 () {56 vars in 1102 bytes} [Tue Sep 13 03:11:16 2022] GET /identity/v3/roles/service => generated 85 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 03:11:17.173065 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55b799e7-fed0-415b-aeb9-8db6fdafc194 None None] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:11:17.221416 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:11:17.284673 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55b799e7-fed0-415b-aeb9-8db6fdafc194 None None] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:11:17.324726 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d34919ef-4934-4760-8b1a-05420467e248 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:11:17.325383 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d34919ef-4934-4760-8b1a-05420467e248 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:11:17.325708 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d34919ef-4934-4760-8b1a-05420467e248 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:11:17.326010 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d34919ef-4934-4760-8b1a-05420467e248 None admin] PATH_INFO: `/v3/roles` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:11:17.326711 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d34919ef-4934-4760-8b1a-05420467e248 None admin] RBAC: Adding query filter params (name=service) {{(pid=81400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 03:11:17.327018 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d34919ef-4934-4760-8b1a-05420467e248 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:11:17.328469 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d34919ef-4934-4760-8b1a-05420467e248 None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:11:17.337989 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 622/1236] 10.222.0.18 () {56 vars in 1103 bytes} [Tue Sep 13 03:11:17 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 03:11:17.344673 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-629715bc-900f-4c52-aee2-7c10115b8d65 None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:11:17.447211 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:11:17.468190 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-629715bc-900f-4c52-aee2-7c10115b8d65 None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:11:17.508304 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc4cddde-63d7-45a9-bbcc-3aeeecadceec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:11:17.509057 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc4cddde-63d7-45a9-bbcc-3aeeecadceec None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:11:17.509388 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc4cddde-63d7-45a9-bbcc-3aeeecadceec None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:11:17.509689 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc4cddde-63d7-45a9-bbcc-3aeeecadceec None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:11:17.521981 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc4cddde-63d7-45a9-bbcc-3aeeecadceec None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:11:17.523809 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc4cddde-63d7-45a9-bbcc-3aeeecadceec None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:11:17.541967 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application [None req-bc4cddde-63d7-45a9-bbcc-3aeeecadceec None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 03:11:17.541967 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 03:11:17.541967 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 03:11:17.541967 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 03:11:17.541967 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 03:11:17.541967 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 03:11:17.541967 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 03:11:17.541967 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 03:11:17.541967 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 03:11:17.541967 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 03:11:17.541967 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 03:11:17.541967 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 03:11:17.541967 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 13 03:11:17.541967 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 13 03:11:17.541967 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 13 03:11:17.541967 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 13 03:11:17.543637 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 03:11:17.543637 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 03:11:17.543637 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 03:11:17.543637 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 03:11:17.543637 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 03:11:17.543637 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 03:11:17.543637 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 03:11:17.543637 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application with Lock( Sep 13 03:11:17.543637 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 03:11:17.543637 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._enter() Sep 13 03:11:17.543637 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 03:11:17.543637 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 03:11:17.543637 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 03:11:17.543637 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.creator() Sep 13 03:11:17.543637 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 03:11:17.543637 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application created_value = creator( Sep 13 03:11:17.543637 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 13 03:11:17.544554 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 13 03:11:17.544554 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 03:11:17.544554 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Sep 13 03:11:17.545926 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 615/1237] 10.222.0.18 () {56 vars in 1108 bytes} [Tue Sep 13 03:11:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 03:11:17.552155 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d34919ef-4934-4760-8b1a-05420467e248 None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:11:17.664615 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d34919ef-4934-4760-8b1a-05420467e248 None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:11:17.676845 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:11:17.706959 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70fbceeb-07ea-4bd4-886c-5f904b6c8c5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:11:17.707726 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70fbceeb-07ea-4bd4-886c-5f904b6c8c5a None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:11:17.707958 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70fbceeb-07ea-4bd4-886c-5f904b6c8c5a None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:11:17.708312 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70fbceeb-07ea-4bd4-886c-5f904b6c8c5a None admin] PATH_INFO: `/v3/domains` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:11:17.709006 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70fbceeb-07ea-4bd4-886c-5f904b6c8c5a None admin] RBAC: Adding query filter params (name=Default) {{(pid=81400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 03:11:17.709308 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70fbceeb-07ea-4bd4-886c-5f904b6c8c5a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:11:17.710897 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70fbceeb-07ea-4bd4-886c-5f904b6c8c5a None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:11:17.725193 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 623/1238] 10.222.0.18 () {56 vars in 1109 bytes} [Tue Sep 13 03:11:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 03:11:17.730903 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc4cddde-63d7-45a9-bbcc-3aeeecadceec None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:11:17.850753 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc4cddde-63d7-45a9-bbcc-3aeeecadceec None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:11:17.887537 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e91ed33-157e-4972-90c4-e2d11aadb3e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:11:17.888172 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e91ed33-157e-4972-90c4-e2d11aadb3e7 None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:11:17.888533 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e91ed33-157e-4972-90c4-e2d11aadb3e7 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:11:17.888720 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e91ed33-157e-4972-90c4-e2d11aadb3e7 None admin] PATH_INFO: `/v3/users/placement` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:11:17.891632 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:11:17.896989 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e91ed33-157e-4972-90c4-e2d11aadb3e7 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:11:17.898620 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e91ed33-157e-4972-90c4-e2d11aadb3e7 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:11:17.912170 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application [None req-9e91ed33-157e-4972-90c4-e2d11aadb3e7 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Sep 13 03:11:17.912170 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 03:11:17.912170 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 03:11:17.912170 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 03:11:17.912170 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 03:11:17.912170 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 03:11:17.912170 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 03:11:17.912170 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 03:11:17.912170 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 03:11:17.912170 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 03:11:17.912170 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 03:11:17.912170 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 03:11:17.912170 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 13 03:11:17.912170 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 13 03:11:17.912170 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 13 03:11:17.912170 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 13 03:11:17.913329 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 03:11:17.913329 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 03:11:17.913329 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 13 03:11:17.913329 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 13 03:11:17.913329 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 13 03:11:17.913329 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 13 03:11:17.913329 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 03:11:17.913329 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 03:11:17.913329 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 03:11:17.913329 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 03:11:17.913329 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 03:11:17.913329 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application with Lock( Sep 13 03:11:17.913329 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 03:11:17.913329 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._enter() Sep 13 03:11:17.913329 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 03:11:17.913329 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 03:11:17.913329 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 03:11:17.913329 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.creator() Sep 13 03:11:17.914478 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 03:11:17.914478 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application created_value = creator( Sep 13 03:11:17.914478 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 13 03:11:17.914478 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 13 03:11:17.914478 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 13 03:11:17.914478 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 13 03:11:17.914478 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 13 03:11:17.914478 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 13 03:11:17.914478 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Sep 13 03:11:17.914478 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Sep 13 03:11:17.919156 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70fbceeb-07ea-4bd4-886c-5f904b6c8c5a None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:11:17.919597 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 616/1239] 10.222.0.18 () {56 vars in 1108 bytes} [Tue Sep 13 03:11:17 2022] GET /identity/v3/users/placement => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 13 03:11:18.029080 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70fbceeb-07ea-4bd4-886c-5f904b6c8c5a None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:11:18.068289 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43756890-e465-4c2d-8973-dba963f73687 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:11:18.068895 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43756890-e465-4c2d-8973-dba963f73687 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:11:18.069174 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43756890-e465-4c2d-8973-dba963f73687 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:11:18.069442 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43756890-e465-4c2d-8973-dba963f73687 None admin] PATH_INFO: `/v3/users` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:11:18.070170 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43756890-e465-4c2d-8973-dba963f73687 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 03:11:18.070500 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43756890-e465-4c2d-8973-dba963f73687 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:11:18.071957 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43756890-e465-4c2d-8973-dba963f73687 None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:11:18.091201 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 624/1240] 10.222.0.18 () {56 vars in 1143 bytes} [Tue Sep 13 03:11:17 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 03:11:18.097353 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e91ed33-157e-4972-90c4-e2d11aadb3e7 None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:11:18.214387 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e91ed33-157e-4972-90c4-e2d11aadb3e7 None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:11:18.252088 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1534971-a7f1-4566-8089-1a5d93052481 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:11:18.252846 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1534971-a7f1-4566-8089-1a5d93052481 None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:11:18.253108 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1534971-a7f1-4566-8089-1a5d93052481 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:11:18.253412 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1534971-a7f1-4566-8089-1a5d93052481 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:11:18.329652 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:11:18.337857 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1534971-a7f1-4566-8089-1a5d93052481 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:11:18.339583 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1534971-a7f1-4566-8089-1a5d93052481 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:11:18.446799 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 617/1241] 10.222.0.18 () {58 vars in 1282 bytes} [Tue Sep 13 03:11:18 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 350 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 13 03:11:18.453371 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43756890-e465-4c2d-8973-dba963f73687 None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:11:18.482349 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:11:18.566219 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43756890-e465-4c2d-8973-dba963f73687 None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:11:18.603918 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6d5bdec-2cdd-427c-8d7b-67b298827389 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:11:18.604540 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6d5bdec-2cdd-427c-8d7b-67b298827389 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:11:18.604812 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6d5bdec-2cdd-427c-8d7b-67b298827389 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:11:18.605135 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6d5bdec-2cdd-427c-8d7b-67b298827389 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:11:18.616567 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6d5bdec-2cdd-427c-8d7b-67b298827389 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:11:18.618331 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6d5bdec-2cdd-427c-8d7b-67b298827389 None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:11:18.640840 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application [None req-c6d5bdec-2cdd-427c-8d7b-67b298827389 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 03:11:18.640840 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 03:11:18.640840 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 03:11:18.640840 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 03:11:18.640840 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 03:11:18.640840 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 03:11:18.640840 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 03:11:18.640840 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 03:11:18.640840 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 03:11:18.640840 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 03:11:18.640840 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 03:11:18.640840 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 03:11:18.640840 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 13 03:11:18.640840 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 13 03:11:18.640840 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 13 03:11:18.640840 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 13 03:11:18.642129 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 03:11:18.642129 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 03:11:18.642129 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 03:11:18.642129 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 03:11:18.642129 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 03:11:18.642129 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 03:11:18.642129 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 03:11:18.642129 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application with Lock( Sep 13 03:11:18.642129 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 03:11:18.642129 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._enter() Sep 13 03:11:18.642129 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 03:11:18.642129 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 03:11:18.642129 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 03:11:18.642129 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.creator() Sep 13 03:11:18.642129 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 03:11:18.642129 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application created_value = creator( Sep 13 03:11:18.642129 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 13 03:11:18.643322 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 13 03:11:18.643322 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 03:11:18.643322 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Sep 13 03:11:18.643322 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 625/1242] 10.222.0.18 () {56 vars in 1108 bytes} [Tue Sep 13 03:11:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 13 03:11:18.647907 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1534971-a7f1-4566-8089-1a5d93052481 None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:11:18.753228 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:11:18.760043 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1534971-a7f1-4566-8089-1a5d93052481 None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:11:18.803188 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8416f97-8ecc-4548-af2c-29d8021bd3b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:11:18.803900 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8416f97-8ecc-4548-af2c-29d8021bd3b1 None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:11:18.804287 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8416f97-8ecc-4548-af2c-29d8021bd3b1 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:11:18.804670 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8416f97-8ecc-4548-af2c-29d8021bd3b1 None admin] PATH_INFO: `/v3/domains` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:11:18.805344 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8416f97-8ecc-4548-af2c-29d8021bd3b1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 03:11:18.805717 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8416f97-8ecc-4548-af2c-29d8021bd3b1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:11:18.807433 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8416f97-8ecc-4548-af2c-29d8021bd3b1 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:11:18.825713 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 618/1243] 10.222.0.18 () {56 vars in 1109 bytes} [Tue Sep 13 03:11:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 03:11:18.826970 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6d5bdec-2cdd-427c-8d7b-67b298827389 None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:11:18.913984 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:11:18.939049 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6d5bdec-2cdd-427c-8d7b-67b298827389 None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:11:18.976995 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3179860c-54bb-4c15-ae6c-1eb5f159bfaa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:11:18.977664 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3179860c-54bb-4c15-ae6c-1eb5f159bfaa None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:11:18.977963 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3179860c-54bb-4c15-ae6c-1eb5f159bfaa None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:11:18.978256 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3179860c-54bb-4c15-ae6c-1eb5f159bfaa None admin] PATH_INFO: `/v3/projects/service` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:11:18.985875 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3179860c-54bb-4c15-ae6c-1eb5f159bfaa None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:11:18.987359 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3179860c-54bb-4c15-ae6c-1eb5f159bfaa None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:11:18.999459 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application [None req-3179860c-54bb-4c15-ae6c-1eb5f159bfaa None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 13 03:11:18.999459 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 03:11:18.999459 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 03:11:18.999459 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 03:11:18.999459 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 03:11:18.999459 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 03:11:18.999459 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 03:11:18.999459 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 03:11:18.999459 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 03:11:18.999459 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 03:11:18.999459 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 03:11:18.999459 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 03:11:18.999459 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 13 03:11:18.999459 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 13 03:11:18.999459 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 13 03:11:19.000787 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 13 03:11:19.000787 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 03:11:19.000787 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 03:11:19.000787 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 03:11:19.000787 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 03:11:19.000787 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 03:11:19.000787 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 03:11:19.000787 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 03:11:19.000787 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application with Lock( Sep 13 03:11:19.000787 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 03:11:19.000787 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._enter() Sep 13 03:11:19.000787 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 03:11:19.000787 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 03:11:19.000787 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 03:11:19.000787 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.creator() Sep 13 03:11:19.000787 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 03:11:19.000787 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application created_value = creator( Sep 13 03:11:19.002562 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 13 03:11:19.002562 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 13 03:11:19.002562 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 13 03:11:19.002562 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 13 03:11:19.002562 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 13 03:11:19.002562 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 13 03:11:19.002562 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 13 03:11:19.002562 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Sep 13 03:11:19.002562 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 626/1244] 10.222.0.18 () {56 vars in 1111 bytes} [Tue Sep 13 03:11:18 2022] GET /identity/v3/projects/service => generated 88 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 03:11:19.007180 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8416f97-8ecc-4548-af2c-29d8021bd3b1 None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:11:19.130728 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8416f97-8ecc-4548-af2c-29d8021bd3b1 None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:11:19.169933 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e09d7df7-2ebf-4a17-95ff-5a38561750d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:11:19.170556 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e09d7df7-2ebf-4a17-95ff-5a38561750d1 None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:11:19.170866 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e09d7df7-2ebf-4a17-95ff-5a38561750d1 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:11:19.171139 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e09d7df7-2ebf-4a17-95ff-5a38561750d1 None admin] PATH_INFO: `/v3/projects` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:11:19.171860 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e09d7df7-2ebf-4a17-95ff-5a38561750d1 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 03:11:19.172160 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e09d7df7-2ebf-4a17-95ff-5a38561750d1 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:11:19.173590 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e09d7df7-2ebf-4a17-95ff-5a38561750d1 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:11:19.187358 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 619/1245] 10.222.0.18 () {56 vars in 1148 bytes} [Tue Sep 13 03:11:19 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 03:11:19.194825 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3179860c-54bb-4c15-ae6c-1eb5f159bfaa None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:11:19.315796 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3179860c-54bb-4c15-ae6c-1eb5f159bfaa None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:11:19.355460 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-007a1e94-c7d2-40af-ae9f-1af4a6da0a8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:11:19.356057 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-007a1e94-c7d2-40af-ae9f-1af4a6da0a8d None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:11:19.356336 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-007a1e94-c7d2-40af-ae9f-1af4a6da0a8d None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:11:19.356603 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-007a1e94-c7d2-40af-ae9f-1af4a6da0a8d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:11:19.357319 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-007a1e94-c7d2-40af-ae9f-1af4a6da0a8d None admin] RBAC: Adding query filter params (role.id=817d6837cd044890b41cf16b303a092b, scope.project.id=b9d94f6a68f7470d9494f112175620dc, user.id=4b99f1b2374d4d2f8db78306316180d9) {{(pid=81400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 03:11:19.357804 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-007a1e94-c7d2-40af-ae9f-1af4a6da0a8d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:11:19.359693 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-007a1e94-c7d2-40af-ae9f-1af4a6da0a8d None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:11:19.365870 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 627/1246] 10.222.0.18 () {56 vars in 1374 bytes} [Tue Sep 13 03:11:19 2022] GET /identity/v3/role_assignments?user.id=4b99f1b2374d4d2f8db78306316180d9&scope.project.id=b9d94f6a68f7470d9494f112175620dc&role.id=817d6837cd044890b41cf16b303a092b => generated 608 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 03:11:21.672408 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48f118cc-8c55-48c0-95c8-e7f5674a22cb None None] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:11:21.672988 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48f118cc-8c55-48c0-95c8-e7f5674a22cb None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:11:21.672988 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48f118cc-8c55-48c0-95c8-e7f5674a22cb None None] PATH_INFO: `/` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:11:21.674289 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 620/1247] 10.222.0.18 () {54 vars in 927 bytes} [Tue Sep 13 03:11:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 03:11:21.680161 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d863225a-6a9b-4230-890d-f2a1deb44200 None None] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:11:21.680594 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d863225a-6a9b-4230-890d-f2a1deb44200 None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:11:21.680718 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d863225a-6a9b-4230-890d-f2a1deb44200 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:11:21.688022 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:11:21.702105 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.auth.core [None req-d863225a-6a9b-4230-890d-f2a1deb44200 None None] MFA Rules not processed for user `49f280ab086b4610855c4ffa378517d9`. Rule list: `[]` (Enabled: `True`). {{(pid=81400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 03:11:21.703650 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-d863225a-6a9b-4230-890d-f2a1deb44200 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:11:21.782688 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 628/1248] 10.222.0.18 () {58 vars in 1024 bytes} [Tue Sep 13 03:11:21 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 13 03:11:22.093363 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48f118cc-8c55-48c0-95c8-e7f5674a22cb None None] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:11:22.115469 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:11:22.203906 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48f118cc-8c55-48c0-95c8-e7f5674a22cb None None] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:11:22.238631 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7e8664a-b308-48be-bc6c-b19f368063ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:11:22.239285 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e8664a-b308-48be-bc6c-b19f368063ae None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:11:22.239562 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e8664a-b308-48be-bc6c-b19f368063ae None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:11:22.239831 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e8664a-b308-48be-bc6c-b19f368063ae None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:11:22.252202 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7e8664a-b308-48be-bc6c-b19f368063ae None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 03:11:22.257120 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application [None req-f7e8664a-b308-48be-bc6c-b19f368063ae None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 13 03:11:22.257120 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 03:11:22.257120 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 03:11:22.257120 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 03:11:22.257120 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 03:11:22.257120 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 03:11:22.257120 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 03:11:22.257120 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 03:11:22.257120 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 03:11:22.257120 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 03:11:22.257120 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 03:11:22.257120 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 03:11:22.257120 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 03:11:22.257120 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 03:11:22.257120 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 03:11:22.257120 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 03:11:22.258496 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 03:11:22.258496 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 03:11:22.258496 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 03:11:22.258496 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 03:11:22.258496 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 03:11:22.258496 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 03:11:22.258496 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 03:11:22.258496 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 03:11:22.258496 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 03:11:22.258496 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 03:11:22.258496 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 03:11:22.258496 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application with Lock( Sep 13 03:11:22.258496 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 03:11:22.258496 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._enter() Sep 13 03:11:22.258496 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 03:11:22.258496 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 03:11:22.259792 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 03:11:22.259792 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.creator() Sep 13 03:11:22.259792 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 03:11:22.259792 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application created_value = creator( Sep 13 03:11:22.259792 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 03:11:22.259792 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 03:11:22.259792 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 03:11:22.259792 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 03:11:22.259792 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 03:11:22.259792 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 03:11:22.259792 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 13 03:11:22.259792 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Sep 13 03:11:22.259792 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 621/1249] 10.222.0.18 () {56 vars in 1096 bytes} [Tue Sep 13 03:11:22 2022] GET /identity/v3/roles/admin => generated 83 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 03:11:22.265812 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d863225a-6a9b-4230-890d-f2a1deb44200 None None] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:11:22.378016 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d863225a-6a9b-4230-890d-f2a1deb44200 None None] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:11:22.411915 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:11:22.417707 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95fbb46c-6efd-451e-8041-c0a030e11d70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:11:22.418350 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95fbb46c-6efd-451e-8041-c0a030e11d70 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:11:22.418647 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95fbb46c-6efd-451e-8041-c0a030e11d70 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:11:22.418918 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95fbb46c-6efd-451e-8041-c0a030e11d70 None admin] PATH_INFO: `/v3/roles` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:11:22.419587 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95fbb46c-6efd-451e-8041-c0a030e11d70 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 03:11:22.419887 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95fbb46c-6efd-451e-8041-c0a030e11d70 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:11:22.421293 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95fbb46c-6efd-451e-8041-c0a030e11d70 None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:11:22.431000 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 629/1250] 10.222.0.18 () {56 vars in 1099 bytes} [Tue Sep 13 03:11:22 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 03:11:22.437205 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7e8664a-b308-48be-bc6c-b19f368063ae None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:11:22.518941 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:11:22.542463 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7e8664a-b308-48be-bc6c-b19f368063ae None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:11:22.578310 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48073f89-b89a-4a71-82c6-436b85d08038 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:11:22.578988 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48073f89-b89a-4a71-82c6-436b85d08038 None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:11:22.579267 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48073f89-b89a-4a71-82c6-436b85d08038 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:11:22.579536 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48073f89-b89a-4a71-82c6-436b85d08038 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:11:22.591938 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48073f89-b89a-4a71-82c6-436b85d08038 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:11:22.593405 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48073f89-b89a-4a71-82c6-436b85d08038 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:11:22.608221 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application [None req-48073f89-b89a-4a71-82c6-436b85d08038 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 03:11:22.608221 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 03:11:22.608221 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 03:11:22.608221 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 03:11:22.608221 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 03:11:22.608221 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 03:11:22.608221 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 03:11:22.608221 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 03:11:22.608221 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 03:11:22.608221 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 03:11:22.608221 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 03:11:22.608221 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 03:11:22.608221 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 13 03:11:22.608221 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 13 03:11:22.608221 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 13 03:11:22.608221 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 13 03:11:22.609329 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 03:11:22.609329 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 03:11:22.609329 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 03:11:22.609329 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 03:11:22.609329 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 03:11:22.609329 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 03:11:22.609329 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 03:11:22.609329 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application with Lock( Sep 13 03:11:22.609329 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 03:11:22.609329 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._enter() Sep 13 03:11:22.609329 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 03:11:22.609329 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 03:11:22.609329 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 03:11:22.609329 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.creator() Sep 13 03:11:22.609329 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 03:11:22.609329 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application created_value = creator( Sep 13 03:11:22.609329 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 13 03:11:22.610257 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 13 03:11:22.610257 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 03:11:22.610257 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Sep 13 03:11:22.610257 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 622/1251] 10.222.0.18 () {56 vars in 1108 bytes} [Tue Sep 13 03:11:22 2022] GET /identity/v3/domains/Default => generated 87 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 03:11:22.615288 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95fbb46c-6efd-451e-8041-c0a030e11d70 None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:11:22.731368 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95fbb46c-6efd-451e-8041-c0a030e11d70 None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:11:22.775806 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e2942dd-f7dd-4fe2-94e9-58ccd46b3847 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:11:22.776456 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e2942dd-f7dd-4fe2-94e9-58ccd46b3847 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:11:22.776801 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e2942dd-f7dd-4fe2-94e9-58ccd46b3847 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:11:22.777108 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e2942dd-f7dd-4fe2-94e9-58ccd46b3847 None admin] PATH_INFO: `/v3/domains` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:11:22.778067 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e2942dd-f7dd-4fe2-94e9-58ccd46b3847 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 03:11:22.778400 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e2942dd-f7dd-4fe2-94e9-58ccd46b3847 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:11:22.780178 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e2942dd-f7dd-4fe2-94e9-58ccd46b3847 None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:11:22.798532 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 630/1252] 10.222.0.18 () {56 vars in 1109 bytes} [Tue Sep 13 03:11:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 03:11:22.804806 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48073f89-b89a-4a71-82c6-436b85d08038 None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:11:22.906707 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48073f89-b89a-4a71-82c6-436b85d08038 None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:11:22.932038 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:11:22.940795 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-884b88e4-db29-4b32-a9fe-aefcec1f62f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:11:22.941434 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-884b88e4-db29-4b32-a9fe-aefcec1f62f4 None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:11:22.941711 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-884b88e4-db29-4b32-a9fe-aefcec1f62f4 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:11:22.941982 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-884b88e4-db29-4b32-a9fe-aefcec1f62f4 None admin] PATH_INFO: `/v3/users/placement` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:11:22.950183 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-884b88e4-db29-4b32-a9fe-aefcec1f62f4 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:11:22.951642 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-884b88e4-db29-4b32-a9fe-aefcec1f62f4 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:11:22.963039 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application [None req-884b88e4-db29-4b32-a9fe-aefcec1f62f4 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Sep 13 03:11:22.963039 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 03:11:22.963039 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 03:11:22.963039 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 03:11:22.963039 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 03:11:22.963039 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 03:11:22.963039 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 03:11:22.963039 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 03:11:22.963039 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 03:11:22.963039 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 03:11:22.963039 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 03:11:22.963039 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 03:11:22.963039 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 13 03:11:22.963039 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 13 03:11:22.963039 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 13 03:11:22.963039 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 13 03:11:22.964195 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 03:11:22.964195 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 03:11:22.964195 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 13 03:11:22.964195 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 13 03:11:22.964195 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 13 03:11:22.964195 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 13 03:11:22.964195 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 03:11:22.964195 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 03:11:22.964195 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 03:11:22.964195 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 03:11:22.964195 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 03:11:22.964195 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application with Lock( Sep 13 03:11:22.964195 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 03:11:22.964195 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._enter() Sep 13 03:11:22.964195 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 03:11:22.964195 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 03:11:22.964195 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 03:11:22.964195 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.creator() Sep 13 03:11:22.965288 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 03:11:22.965288 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application created_value = creator( Sep 13 03:11:22.965288 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 13 03:11:22.965288 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 13 03:11:22.965288 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 13 03:11:22.965288 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 13 03:11:22.965288 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 13 03:11:22.965288 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 13 03:11:22.965288 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Sep 13 03:11:22.965288 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Sep 13 03:11:22.966788 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 623/1253] 10.222.0.18 () {56 vars in 1108 bytes} [Tue Sep 13 03:11:22 2022] GET /identity/v3/users/placement => generated 87 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 03:11:22.969964 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e2942dd-f7dd-4fe2-94e9-58ccd46b3847 None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:11:23.028652 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:11:23.075892 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e2942dd-f7dd-4fe2-94e9-58ccd46b3847 None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:11:23.108892 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de4ec9f7-da7b-47cd-b2f6-20296a3add66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:11:23.109481 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de4ec9f7-da7b-47cd-b2f6-20296a3add66 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:11:23.109753 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de4ec9f7-da7b-47cd-b2f6-20296a3add66 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:11:23.110014 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de4ec9f7-da7b-47cd-b2f6-20296a3add66 None admin] PATH_INFO: `/v3/users` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:11:23.110755 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de4ec9f7-da7b-47cd-b2f6-20296a3add66 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 03:11:23.111051 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de4ec9f7-da7b-47cd-b2f6-20296a3add66 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:11:23.112447 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de4ec9f7-da7b-47cd-b2f6-20296a3add66 None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:11:23.129332 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 631/1254] 10.222.0.18 () {56 vars in 1143 bytes} [Tue Sep 13 03:11:22 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 03:11:23.135088 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-884b88e4-db29-4b32-a9fe-aefcec1f62f4 None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:11:23.232020 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-884b88e4-db29-4b32-a9fe-aefcec1f62f4 None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:11:23.263838 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-701ee617-aaa0-4a71-b565-1765f8141414 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:11:23.264431 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-701ee617-aaa0-4a71-b565-1765f8141414 None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:11:23.264709 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-701ee617-aaa0-4a71-b565-1765f8141414 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:11:23.264978 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-701ee617-aaa0-4a71-b565-1765f8141414 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:11:23.317532 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:11:23.329682 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-701ee617-aaa0-4a71-b565-1765f8141414 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:11:23.331151 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-701ee617-aaa0-4a71-b565-1765f8141414 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:11:23.424724 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 624/1255] 10.222.0.18 () {58 vars in 1282 bytes} [Tue Sep 13 03:11:23 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 290 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 13 03:11:23.430714 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de4ec9f7-da7b-47cd-b2f6-20296a3add66 None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:11:23.524596 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de4ec9f7-da7b-47cd-b2f6-20296a3add66 None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:11:23.537549 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:11:23.559334 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90004010-96cc-4ed9-835e-c6c05ba0fd72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:11:23.559915 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90004010-96cc-4ed9-835e-c6c05ba0fd72 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:11:23.560190 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90004010-96cc-4ed9-835e-c6c05ba0fd72 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:11:23.560459 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90004010-96cc-4ed9-835e-c6c05ba0fd72 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:11:23.570659 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90004010-96cc-4ed9-835e-c6c05ba0fd72 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:11:23.572035 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90004010-96cc-4ed9-835e-c6c05ba0fd72 None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:11:23.586437 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application [None req-90004010-96cc-4ed9-835e-c6c05ba0fd72 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 03:11:23.586437 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 03:11:23.586437 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 03:11:23.586437 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 03:11:23.586437 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 03:11:23.586437 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 03:11:23.586437 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 03:11:23.586437 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 03:11:23.586437 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 03:11:23.586437 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 03:11:23.586437 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 03:11:23.586437 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 03:11:23.586437 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 13 03:11:23.586437 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 13 03:11:23.586437 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 13 03:11:23.586437 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 13 03:11:23.587530 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 03:11:23.587530 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 03:11:23.587530 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 03:11:23.587530 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 03:11:23.587530 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 03:11:23.587530 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 03:11:23.587530 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 03:11:23.587530 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application with Lock( Sep 13 03:11:23.587530 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 03:11:23.587530 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._enter() Sep 13 03:11:23.587530 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 03:11:23.587530 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 03:11:23.587530 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 03:11:23.587530 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.creator() Sep 13 03:11:23.587530 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 03:11:23.587530 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application created_value = creator( Sep 13 03:11:23.587530 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 13 03:11:23.588568 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 13 03:11:23.588568 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 03:11:23.588568 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Sep 13 03:11:23.588568 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 632/1256] 10.222.0.18 () {56 vars in 1108 bytes} [Tue Sep 13 03:11:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 13 03:11:23.593649 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-701ee617-aaa0-4a71-b565-1765f8141414 None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:11:23.677081 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:11:23.689273 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-701ee617-aaa0-4a71-b565-1765f8141414 None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:11:23.722727 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4e3aa4c-fedd-4882-a369-6d6c6c2c3081 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:11:23.723314 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4e3aa4c-fedd-4882-a369-6d6c6c2c3081 None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:11:23.723591 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4e3aa4c-fedd-4882-a369-6d6c6c2c3081 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:11:23.723856 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4e3aa4c-fedd-4882-a369-6d6c6c2c3081 None admin] PATH_INFO: `/v3/domains` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:11:23.724477 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4e3aa4c-fedd-4882-a369-6d6c6c2c3081 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 03:11:23.724774 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4e3aa4c-fedd-4882-a369-6d6c6c2c3081 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:11:23.726116 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4e3aa4c-fedd-4882-a369-6d6c6c2c3081 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:11:23.737011 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 625/1257] 10.222.0.18 () {56 vars in 1109 bytes} [Tue Sep 13 03:11:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 03:11:23.744033 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90004010-96cc-4ed9-835e-c6c05ba0fd72 None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:11:23.848510 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90004010-96cc-4ed9-835e-c6c05ba0fd72 None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:11:23.880282 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22368cb1-8d44-4c9b-8f31-bafc33beadb2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:11:23.880891 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22368cb1-8d44-4c9b-8f31-bafc33beadb2 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:11:23.881165 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22368cb1-8d44-4c9b-8f31-bafc33beadb2 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:11:23.881436 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22368cb1-8d44-4c9b-8f31-bafc33beadb2 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:11:23.888340 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22368cb1-8d44-4c9b-8f31-bafc33beadb2 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:11:23.889769 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22368cb1-8d44-4c9b-8f31-bafc33beadb2 None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:11:23.900706 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application [None req-22368cb1-8d44-4c9b-8f31-bafc33beadb2 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 13 03:11:23.900706 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 03:11:23.900706 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 03:11:23.900706 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 03:11:23.900706 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 03:11:23.900706 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 03:11:23.900706 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 03:11:23.900706 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 03:11:23.900706 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 03:11:23.900706 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 03:11:23.900706 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 03:11:23.900706 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 03:11:23.900706 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 13 03:11:23.900706 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 13 03:11:23.900706 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 13 03:11:23.901718 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 13 03:11:23.901718 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 03:11:23.901718 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 03:11:23.901718 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 03:11:23.901718 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 03:11:23.901718 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 03:11:23.901718 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 03:11:23.901718 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 03:11:23.901718 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application with Lock( Sep 13 03:11:23.901718 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 03:11:23.901718 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._enter() Sep 13 03:11:23.901718 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 03:11:23.901718 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 03:11:23.901718 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 03:11:23.901718 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.creator() Sep 13 03:11:23.901718 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 03:11:23.901718 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application created_value = creator( Sep 13 03:11:23.903318 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 13 03:11:23.903318 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 13 03:11:23.903318 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 13 03:11:23.903318 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 13 03:11:23.903318 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 13 03:11:23.903318 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 13 03:11:23.903318 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 13 03:11:23.903318 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Sep 13 03:11:23.903318 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 633/1258] 10.222.0.18 () {56 vars in 1111 bytes} [Tue Sep 13 03:11:23 2022] GET /identity/v3/projects/service => generated 88 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 03:11:23.908284 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4e3aa4c-fedd-4882-a369-6d6c6c2c3081 None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:11:24.003699 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4e3aa4c-fedd-4882-a369-6d6c6c2c3081 None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:11:24.038422 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad7cb3cd-c026-47b2-971d-0c230fffb603 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:11:24.039068 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad7cb3cd-c026-47b2-971d-0c230fffb603 None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:11:24.039349 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad7cb3cd-c026-47b2-971d-0c230fffb603 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:11:24.039622 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad7cb3cd-c026-47b2-971d-0c230fffb603 None admin] PATH_INFO: `/v3/projects` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:11:24.040301 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad7cb3cd-c026-47b2-971d-0c230fffb603 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 03:11:24.040607 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad7cb3cd-c026-47b2-971d-0c230fffb603 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:11:24.042022 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad7cb3cd-c026-47b2-971d-0c230fffb603 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:11:24.053911 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 626/1259] 10.222.0.18 () {56 vars in 1148 bytes} [Tue Sep 13 03:11:23 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 03:11:24.060016 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22368cb1-8d44-4c9b-8f31-bafc33beadb2 None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:11:24.085407 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:11:24.157944 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22368cb1-8d44-4c9b-8f31-bafc33beadb2 None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:11:24.190450 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5921fef9-68dc-4d3c-9ff2-669a332bf447 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:11:24.191070 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5921fef9-68dc-4d3c-9ff2-669a332bf447 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:11:24.191350 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5921fef9-68dc-4d3c-9ff2-669a332bf447 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:11:24.191621 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5921fef9-68dc-4d3c-9ff2-669a332bf447 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:11:24.192326 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5921fef9-68dc-4d3c-9ff2-669a332bf447 None admin] RBAC: Adding query filter params (role.id=425d4b481c7846e0bc07e7e805d5b15f, scope.project.id=b9d94f6a68f7470d9494f112175620dc, user.id=4b99f1b2374d4d2f8db78306316180d9) {{(pid=81400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 03:11:24.192628 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5921fef9-68dc-4d3c-9ff2-669a332bf447 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:11:24.194039 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5921fef9-68dc-4d3c-9ff2-669a332bf447 None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:11:24.199849 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 634/1260] 10.222.0.18 () {56 vars in 1374 bytes} [Tue Sep 13 03:11:24 2022] GET /identity/v3/role_assignments?user.id=4b99f1b2374d4d2f8db78306316180d9&scope.project.id=b9d94f6a68f7470d9494f112175620dc&role.id=425d4b481c7846e0bc07e7e805d5b15f => generated 259 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 03:11:26.300097 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fcdec90-8330-4119-9748-1dd3c4a816cf None None] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:11:26.300634 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fcdec90-8330-4119-9748-1dd3c4a816cf None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:11:26.300634 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fcdec90-8330-4119-9748-1dd3c4a816cf None None] PATH_INFO: `/` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:11:26.301990 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 627/1261] 10.222.0.18 () {54 vars in 927 bytes} [Tue Sep 13 03:11:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 03:11:26.307211 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3447e3cd-bd09-4524-8e4e-cac04427581d None None] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:11:26.307588 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3447e3cd-bd09-4524-8e4e-cac04427581d None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:11:26.307753 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3447e3cd-bd09-4524-8e4e-cac04427581d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:11:26.326828 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.auth.core [None req-3447e3cd-bd09-4524-8e4e-cac04427581d None None] MFA Rules not processed for user `49f280ab086b4610855c4ffa378517d9`. Rule list: `[]` (Enabled: `True`). {{(pid=81400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 03:11:26.327936 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-3447e3cd-bd09-4524-8e4e-cac04427581d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:11:26.391746 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 635/1262] 10.222.0.18 () {58 vars in 1024 bytes} [Tue Sep 13 03:11:26 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 13 03:11:26.658358 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fcdec90-8330-4119-9748-1dd3c4a816cf None None] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:11:26.669776 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:11:26.762538 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fcdec90-8330-4119-9748-1dd3c4a816cf None None] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:11:26.801558 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd6683f6-6e3e-4bad-a82e-89e43ff76e88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:11:26.802548 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd6683f6-6e3e-4bad-a82e-89e43ff76e88 None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:11:26.802921 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd6683f6-6e3e-4bad-a82e-89e43ff76e88 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:11:26.803180 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd6683f6-6e3e-4bad-a82e-89e43ff76e88 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:11:26.814553 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd6683f6-6e3e-4bad-a82e-89e43ff76e88 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 03:11:26.820243 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application [None req-bd6683f6-6e3e-4bad-a82e-89e43ff76e88 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 13 03:11:26.820243 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 03:11:26.820243 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 03:11:26.820243 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 03:11:26.820243 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 03:11:26.820243 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 03:11:26.820243 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 03:11:26.820243 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 03:11:26.820243 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 03:11:26.820243 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 03:11:26.820243 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 03:11:26.820243 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 03:11:26.820243 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 03:11:26.820243 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 03:11:26.820243 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 03:11:26.820243 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 03:11:26.821404 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 03:11:26.821404 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 03:11:26.821404 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 03:11:26.821404 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 03:11:26.821404 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 03:11:26.821404 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 03:11:26.821404 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 03:11:26.821404 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 03:11:26.821404 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 03:11:26.821404 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 03:11:26.821404 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 03:11:26.821404 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application with Lock( Sep 13 03:11:26.821404 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 03:11:26.821404 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._enter() Sep 13 03:11:26.821404 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 03:11:26.821404 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 03:11:26.822267 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 03:11:26.822267 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.creator() Sep 13 03:11:26.822267 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 03:11:26.822267 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application created_value = creator( Sep 13 03:11:26.822267 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 03:11:26.822267 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 03:11:26.822267 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 03:11:26.822267 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 03:11:26.822267 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 03:11:26.822267 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 03:11:26.822267 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 13 03:11:26.822267 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Sep 13 03:11:26.822267 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 628/1263] 10.222.0.18 () {56 vars in 1096 bytes} [Tue Sep 13 03:11:26 2022] GET /identity/v3/roles/admin => generated 83 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 03:11:26.827942 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3447e3cd-bd09-4524-8e4e-cac04427581d None None] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:11:26.846295 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:11:26.923906 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3447e3cd-bd09-4524-8e4e-cac04427581d None None] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:11:26.956516 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a2412f1-b91d-402f-9268-21a2a543d707 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:11:26.957103 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a2412f1-b91d-402f-9268-21a2a543d707 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:11:26.957374 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a2412f1-b91d-402f-9268-21a2a543d707 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:11:26.957637 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a2412f1-b91d-402f-9268-21a2a543d707 None admin] PATH_INFO: `/v3/roles` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:11:26.958276 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a2412f1-b91d-402f-9268-21a2a543d707 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 03:11:26.958568 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a2412f1-b91d-402f-9268-21a2a543d707 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:11:26.959964 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a2412f1-b91d-402f-9268-21a2a543d707 None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:11:26.968234 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 636/1264] 10.222.0.18 () {56 vars in 1099 bytes} [Tue Sep 13 03:11:26 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 03:11:26.973820 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd6683f6-6e3e-4bad-a82e-89e43ff76e88 None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:11:27.037261 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:11:27.071611 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd6683f6-6e3e-4bad-a82e-89e43ff76e88 None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:11:27.104893 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c7fc1e6-6bc0-4266-82d6-e0657f3f1afc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:11:27.105516 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c7fc1e6-6bc0-4266-82d6-e0657f3f1afc None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:11:27.105795 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c7fc1e6-6bc0-4266-82d6-e0657f3f1afc None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:11:27.106066 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c7fc1e6-6bc0-4266-82d6-e0657f3f1afc None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:11:27.115777 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c7fc1e6-6bc0-4266-82d6-e0657f3f1afc None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:11:27.117208 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c7fc1e6-6bc0-4266-82d6-e0657f3f1afc None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:11:27.131735 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application [None req-0c7fc1e6-6bc0-4266-82d6-e0657f3f1afc None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 03:11:27.131735 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 03:11:27.131735 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 03:11:27.131735 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 03:11:27.131735 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 03:11:27.131735 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 03:11:27.131735 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 03:11:27.131735 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 03:11:27.131735 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 03:11:27.131735 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 03:11:27.131735 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 03:11:27.131735 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 03:11:27.131735 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 13 03:11:27.131735 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 13 03:11:27.131735 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 13 03:11:27.131735 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 13 03:11:27.132685 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 03:11:27.132685 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 03:11:27.132685 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 03:11:27.132685 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 03:11:27.132685 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 03:11:27.132685 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 03:11:27.132685 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 03:11:27.132685 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application with Lock( Sep 13 03:11:27.132685 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 03:11:27.132685 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._enter() Sep 13 03:11:27.132685 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 03:11:27.132685 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 03:11:27.132685 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 03:11:27.132685 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.creator() Sep 13 03:11:27.132685 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 03:11:27.132685 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application created_value = creator( Sep 13 03:11:27.132685 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 13 03:11:27.134037 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 13 03:11:27.134037 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 03:11:27.134037 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Sep 13 03:11:27.134037 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 629/1265] 10.222.0.18 () {56 vars in 1108 bytes} [Tue Sep 13 03:11:26 2022] GET /identity/v3/domains/Default => generated 87 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 13 03:11:27.139054 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a2412f1-b91d-402f-9268-21a2a543d707 None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:11:27.217852 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:11:27.236375 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a2412f1-b91d-402f-9268-21a2a543d707 None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:11:27.269045 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed037aea-7157-4de5-bc02-65c09c55bf18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:11:27.269630 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed037aea-7157-4de5-bc02-65c09c55bf18 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:11:27.269903 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed037aea-7157-4de5-bc02-65c09c55bf18 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:11:27.270171 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed037aea-7157-4de5-bc02-65c09c55bf18 None admin] PATH_INFO: `/v3/domains` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:11:27.270819 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed037aea-7157-4de5-bc02-65c09c55bf18 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 03:11:27.271118 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed037aea-7157-4de5-bc02-65c09c55bf18 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:11:27.272470 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed037aea-7157-4de5-bc02-65c09c55bf18 None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:11:27.283646 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 637/1266] 10.222.0.18 () {56 vars in 1109 bytes} [Tue Sep 13 03:11:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 03:11:27.289206 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c7fc1e6-6bc0-4266-82d6-e0657f3f1afc None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:11:27.385524 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c7fc1e6-6bc0-4266-82d6-e0657f3f1afc None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:11:27.404796 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:11:27.418925 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa066f93-7332-4821-9d65-a48fcfbf9f2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:11:27.419540 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa066f93-7332-4821-9d65-a48fcfbf9f2e None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:11:27.419819 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa066f93-7332-4821-9d65-a48fcfbf9f2e None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:11:27.420092 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa066f93-7332-4821-9d65-a48fcfbf9f2e None admin] PATH_INFO: `/v3/users/placement` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:11:27.428287 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa066f93-7332-4821-9d65-a48fcfbf9f2e None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:11:27.429710 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa066f93-7332-4821-9d65-a48fcfbf9f2e None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:11:27.441602 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application [None req-aa066f93-7332-4821-9d65-a48fcfbf9f2e None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Sep 13 03:11:27.441602 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 03:11:27.441602 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 03:11:27.441602 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 03:11:27.441602 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 03:11:27.441602 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 03:11:27.441602 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 03:11:27.441602 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 03:11:27.441602 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 03:11:27.441602 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 03:11:27.441602 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 03:11:27.441602 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 03:11:27.441602 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 13 03:11:27.441602 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 13 03:11:27.441602 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 13 03:11:27.441602 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 13 03:11:27.442717 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 03:11:27.442717 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 03:11:27.442717 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 13 03:11:27.442717 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 13 03:11:27.442717 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 13 03:11:27.442717 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 13 03:11:27.442717 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 03:11:27.442717 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 03:11:27.442717 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 03:11:27.442717 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 03:11:27.442717 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 03:11:27.442717 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application with Lock( Sep 13 03:11:27.442717 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 03:11:27.442717 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._enter() Sep 13 03:11:27.442717 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 03:11:27.442717 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 03:11:27.442717 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 03:11:27.442717 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.creator() Sep 13 03:11:27.443714 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 03:11:27.443714 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application created_value = creator( Sep 13 03:11:27.443714 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 13 03:11:27.443714 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 13 03:11:27.443714 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 13 03:11:27.443714 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 13 03:11:27.443714 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 13 03:11:27.443714 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 13 03:11:27.443714 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Sep 13 03:11:27.443714 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Sep 13 03:11:27.443714 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 630/1267] 10.222.0.18 () {56 vars in 1108 bytes} [Tue Sep 13 03:11:27 2022] GET /identity/v3/users/placement => generated 87 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 03:11:27.448411 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed037aea-7157-4de5-bc02-65c09c55bf18 None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:11:27.545104 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed037aea-7157-4de5-bc02-65c09c55bf18 None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:11:27.578125 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81002926-75b9-4e68-bf3c-c94a684b1925 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:11:27.578744 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81002926-75b9-4e68-bf3c-c94a684b1925 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:11:27.579025 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81002926-75b9-4e68-bf3c-c94a684b1925 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:11:27.579295 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81002926-75b9-4e68-bf3c-c94a684b1925 None admin] PATH_INFO: `/v3/users` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:11:27.580008 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81002926-75b9-4e68-bf3c-c94a684b1925 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 03:11:27.580307 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81002926-75b9-4e68-bf3c-c94a684b1925 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:11:27.581721 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81002926-75b9-4e68-bf3c-c94a684b1925 None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:11:27.590024 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:11:27.598448 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 638/1268] 10.222.0.18 () {56 vars in 1143 bytes} [Tue Sep 13 03:11:27 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 03:11:27.604814 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa066f93-7332-4821-9d65-a48fcfbf9f2e None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:11:27.700586 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa066f93-7332-4821-9d65-a48fcfbf9f2e None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:11:27.732101 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff7f40ac-3c40-4c5d-9036-7bd640cebe81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:11:27.732702 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff7f40ac-3c40-4c5d-9036-7bd640cebe81 None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:11:27.732977 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff7f40ac-3c40-4c5d-9036-7bd640cebe81 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:11:27.733239 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff7f40ac-3c40-4c5d-9036-7bd640cebe81 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:11:27.745000 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff7f40ac-3c40-4c5d-9036-7bd640cebe81 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:11:27.747266 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff7f40ac-3c40-4c5d-9036-7bd640cebe81 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:11:27.765733 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application [None req-ff7f40ac-3c40-4c5d-9036-7bd640cebe81 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 03:11:27.765733 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 03:11:27.765733 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 03:11:27.765733 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 03:11:27.765733 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 03:11:27.765733 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 03:11:27.765733 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 03:11:27.765733 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 03:11:27.765733 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 03:11:27.765733 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 03:11:27.765733 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 03:11:27.765733 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 03:11:27.765733 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 13 03:11:27.765733 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 13 03:11:27.765733 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 13 03:11:27.765733 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 13 03:11:27.766908 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 03:11:27.766908 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 03:11:27.766908 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 03:11:27.766908 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 03:11:27.766908 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 03:11:27.766908 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 03:11:27.766908 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 03:11:27.766908 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application with Lock( Sep 13 03:11:27.766908 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 03:11:27.766908 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._enter() Sep 13 03:11:27.766908 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 03:11:27.766908 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 03:11:27.766908 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 03:11:27.766908 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.creator() Sep 13 03:11:27.766908 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 03:11:27.766908 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application created_value = creator( Sep 13 03:11:27.766908 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 13 03:11:27.767909 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 13 03:11:27.767909 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 03:11:27.767909 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Sep 13 03:11:27.767909 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 631/1269] 10.222.0.18 () {56 vars in 1108 bytes} [Tue Sep 13 03:11:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 03:11:27.772852 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81002926-75b9-4e68-bf3c-c94a684b1925 None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:11:27.869914 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81002926-75b9-4e68-bf3c-c94a684b1925 None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:11:27.902718 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1358a1fd-1caf-4bf3-9cbd-2b6ce3aeed4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:11:27.903306 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1358a1fd-1caf-4bf3-9cbd-2b6ce3aeed4b None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:11:27.903576 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1358a1fd-1caf-4bf3-9cbd-2b6ce3aeed4b None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:11:27.903841 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1358a1fd-1caf-4bf3-9cbd-2b6ce3aeed4b None admin] PATH_INFO: `/v3/domains` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:11:27.904463 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1358a1fd-1caf-4bf3-9cbd-2b6ce3aeed4b None admin] RBAC: Adding query filter params (name=Default) {{(pid=81400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 03:11:27.904757 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1358a1fd-1caf-4bf3-9cbd-2b6ce3aeed4b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:11:27.906104 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1358a1fd-1caf-4bf3-9cbd-2b6ce3aeed4b None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:11:27.916739 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 639/1270] 10.222.0.18 () {56 vars in 1109 bytes} [Tue Sep 13 03:11:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 03:11:27.923006 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff7f40ac-3c40-4c5d-9036-7bd640cebe81 None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:11:27.953150 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:11:28.017939 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff7f40ac-3c40-4c5d-9036-7bd640cebe81 None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:11:28.050494 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aab0d9cb-d91c-4e28-8515-2577924c9c5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:11:28.051125 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab0d9cb-d91c-4e28-8515-2577924c9c5d None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:11:28.051399 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab0d9cb-d91c-4e28-8515-2577924c9c5d None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:11:28.051665 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab0d9cb-d91c-4e28-8515-2577924c9c5d None admin] PATH_INFO: `/v3/projects/service` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:11:28.058196 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aab0d9cb-d91c-4e28-8515-2577924c9c5d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:11:28.059665 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aab0d9cb-d91c-4e28-8515-2577924c9c5d None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:11:28.070740 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application [None req-aab0d9cb-d91c-4e28-8515-2577924c9c5d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 13 03:11:28.070740 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 03:11:28.070740 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 03:11:28.070740 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 03:11:28.070740 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 03:11:28.070740 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 03:11:28.070740 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 03:11:28.070740 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 03:11:28.070740 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 03:11:28.070740 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 03:11:28.070740 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 03:11:28.070740 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 03:11:28.070740 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 13 03:11:28.070740 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 13 03:11:28.070740 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 13 03:11:28.071769 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 13 03:11:28.071769 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 03:11:28.071769 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 03:11:28.071769 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 03:11:28.071769 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 03:11:28.071769 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 03:11:28.071769 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 03:11:28.071769 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 03:11:28.071769 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application with Lock( Sep 13 03:11:28.071769 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 03:11:28.071769 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._enter() Sep 13 03:11:28.071769 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 03:11:28.071769 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 03:11:28.071769 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 03:11:28.071769 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.creator() Sep 13 03:11:28.071769 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 03:11:28.071769 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application created_value = creator( Sep 13 03:11:28.072731 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 13 03:11:28.072731 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 13 03:11:28.072731 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 13 03:11:28.072731 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 13 03:11:28.072731 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 13 03:11:28.072731 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 13 03:11:28.072731 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 13 03:11:28.072731 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Sep 13 03:11:28.072731 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 632/1271] 10.222.0.18 () {56 vars in 1111 bytes} [Tue Sep 13 03:11:27 2022] GET /identity/v3/projects/service => generated 88 bytes in 149 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 03:11:28.077533 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1358a1fd-1caf-4bf3-9cbd-2b6ce3aeed4b None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:11:28.135676 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:11:28.176501 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1358a1fd-1caf-4bf3-9cbd-2b6ce3aeed4b None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:11:28.208731 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d03610d7-177f-44c2-9355-2689d4b2ef4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:11:28.209321 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d03610d7-177f-44c2-9355-2689d4b2ef4e None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:11:28.209592 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d03610d7-177f-44c2-9355-2689d4b2ef4e None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:11:28.209856 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d03610d7-177f-44c2-9355-2689d4b2ef4e None admin] PATH_INFO: `/v3/projects` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:11:28.210528 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d03610d7-177f-44c2-9355-2689d4b2ef4e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 03:11:28.210854 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d03610d7-177f-44c2-9355-2689d4b2ef4e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:11:28.212260 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d03610d7-177f-44c2-9355-2689d4b2ef4e None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:11:28.224288 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 640/1272] 10.222.0.18 () {56 vars in 1148 bytes} [Tue Sep 13 03:11:28 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 03:11:28.230464 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aab0d9cb-d91c-4e28-8515-2577924c9c5d None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:11:28.316257 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:11:28.325226 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aab0d9cb-d91c-4e28-8515-2577924c9c5d None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:11:28.356949 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e213b25d-c588-4fec-9779-b31b622d723c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:11:28.357575 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e213b25d-c588-4fec-9779-b31b622d723c None admin] REQUEST_METHOD: `PUT` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:11:28.357851 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e213b25d-c588-4fec-9779-b31b622d723c None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:11:28.358117 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e213b25d-c588-4fec-9779-b31b622d723c None admin] PATH_INFO: `/v3/projects/b9d94f6a68f7470d9494f112175620dc/users/4b99f1b2374d4d2f8db78306316180d9/roles/425d4b481c7846e0bc07e7e805d5b15f` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:11:28.359993 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e213b25d-c588-4fec-9779-b31b622d723c None admin] RBAC: Authorizing `identity:create_grant(project_id=b9d94f6a68f7470d9494f112175620dc, user_id=4b99f1b2374d4d2f8db78306316180d9, role_id=425d4b481c7846e0bc07e7e805d5b15f)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:11:28.361401 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e213b25d-c588-4fec-9779-b31b622d723c None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:11:28.373068 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 633/1273] 10.222.0.18 () {58 vars in 1439 bytes} [Tue Sep 13 03:11:28 2022] PUT /identity/v3/projects/b9d94f6a68f7470d9494f112175620dc/users/4b99f1b2374d4d2f8db78306316180d9/roles/425d4b481c7846e0bc07e7e805d5b15f => generated 0 bytes in 143 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 03:11:30.466406 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c97f729-160d-4ca0-94fc-ba820ef3fb3b None None] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:11:30.466945 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c97f729-160d-4ca0-94fc-ba820ef3fb3b None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:11:30.467057 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c97f729-160d-4ca0-94fc-ba820ef3fb3b None None] PATH_INFO: `/` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:11:30.468300 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 641/1274] 10.222.0.18 () {54 vars in 927 bytes} [Tue Sep 13 03:11:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 03:11:30.473510 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecb2dce3-3cde-4bd4-94d8-d040586a9d38 None None] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:11:30.473917 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecb2dce3-3cde-4bd4-94d8-d040586a9d38 None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:11:30.474045 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecb2dce3-3cde-4bd4-94d8-d040586a9d38 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:11:30.492688 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.auth.core [None req-ecb2dce3-3cde-4bd4-94d8-d040586a9d38 None None] MFA Rules not processed for user `49f280ab086b4610855c4ffa378517d9`. Rule list: `[]` (Enabled: `True`). {{(pid=81399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 03:11:30.493852 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-ecb2dce3-3cde-4bd4-94d8-d040586a9d38 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:11:30.562025 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 634/1275] 10.222.0.18 () {58 vars in 1024 bytes} [Tue Sep 13 03:11:30 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 13 03:11:30.831529 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c97f729-160d-4ca0-94fc-ba820ef3fb3b None None] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:11:30.930865 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c97f729-160d-4ca0-94fc-ba820ef3fb3b None None] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:11:30.947543 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:11:30.963967 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21824bd4-3885-4e1f-a950-679ff0ec3e03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:11:30.965107 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21824bd4-3885-4e1f-a950-679ff0ec3e03 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:11:30.965384 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21824bd4-3885-4e1f-a950-679ff0ec3e03 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:11:30.965657 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21824bd4-3885-4e1f-a950-679ff0ec3e03 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:11:30.975462 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21824bd4-3885-4e1f-a950-679ff0ec3e03 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 03:11:30.981007 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application [None req-21824bd4-3885-4e1f-a950-679ff0ec3e03 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 13 03:11:30.981007 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 03:11:30.981007 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 03:11:30.981007 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 03:11:30.981007 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 03:11:30.981007 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 03:11:30.981007 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 03:11:30.981007 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 03:11:30.981007 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 03:11:30.981007 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 03:11:30.981007 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 03:11:30.981007 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 03:11:30.981007 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 03:11:30.981007 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 03:11:30.981007 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 03:11:30.981007 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 03:11:30.982122 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 03:11:30.982122 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 03:11:30.982122 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 03:11:30.982122 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 03:11:30.982122 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 03:11:30.982122 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 03:11:30.982122 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 03:11:30.982122 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 03:11:30.982122 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 03:11:30.982122 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 03:11:30.982122 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 03:11:30.982122 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application with Lock( Sep 13 03:11:30.982122 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 03:11:30.982122 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._enter() Sep 13 03:11:30.982122 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 03:11:30.982122 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 03:11:30.983198 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 03:11:30.983198 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.creator() Sep 13 03:11:30.983198 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 03:11:30.983198 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application created_value = creator( Sep 13 03:11:30.983198 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 03:11:30.983198 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 03:11:30.983198 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 03:11:30.983198 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 03:11:30.983198 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 03:11:30.983198 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 03:11:30.983198 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 13 03:11:30.983198 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Sep 13 03:11:30.983198 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 642/1276] 10.222.0.18 () {56 vars in 1096 bytes} [Tue Sep 13 03:11:30 2022] GET /identity/v3/roles/admin => generated 83 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 03:11:30.987948 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecb2dce3-3cde-4bd4-94d8-d040586a9d38 None None] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:11:31.064840 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:11:31.083330 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecb2dce3-3cde-4bd4-94d8-d040586a9d38 None None] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:11:31.116512 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a0e5da7-bd18-4f71-aedb-30cec1651d12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:11:31.117100 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a0e5da7-bd18-4f71-aedb-30cec1651d12 None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:11:31.117371 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a0e5da7-bd18-4f71-aedb-30cec1651d12 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:11:31.117635 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a0e5da7-bd18-4f71-aedb-30cec1651d12 None admin] PATH_INFO: `/v3/roles` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:11:31.118279 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a0e5da7-bd18-4f71-aedb-30cec1651d12 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 03:11:31.118573 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a0e5da7-bd18-4f71-aedb-30cec1651d12 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:11:31.119946 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a0e5da7-bd18-4f71-aedb-30cec1651d12 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:11:31.127605 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 635/1277] 10.222.0.18 () {56 vars in 1099 bytes} [Tue Sep 13 03:11:30 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 03:11:31.133421 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21824bd4-3885-4e1f-a950-679ff0ec3e03 None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:11:31.229646 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21824bd4-3885-4e1f-a950-679ff0ec3e03 None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:11:31.261809 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-106e8a81-43b6-4b2d-99b7-8de40f844f2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:11:31.262480 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-106e8a81-43b6-4b2d-99b7-8de40f844f2b None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:11:31.262782 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-106e8a81-43b6-4b2d-99b7-8de40f844f2b None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:11:31.263046 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-106e8a81-43b6-4b2d-99b7-8de40f844f2b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:11:31.272522 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-106e8a81-43b6-4b2d-99b7-8de40f844f2b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:11:31.273935 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-106e8a81-43b6-4b2d-99b7-8de40f844f2b None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:11:31.288591 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application [None req-106e8a81-43b6-4b2d-99b7-8de40f844f2b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 03:11:31.288591 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 03:11:31.288591 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 03:11:31.288591 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 03:11:31.288591 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 03:11:31.288591 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 03:11:31.288591 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 03:11:31.288591 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 03:11:31.288591 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 03:11:31.288591 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 03:11:31.288591 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 03:11:31.288591 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 03:11:31.288591 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 13 03:11:31.288591 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 13 03:11:31.288591 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 13 03:11:31.288591 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 13 03:11:31.289698 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 03:11:31.289698 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 03:11:31.289698 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 03:11:31.289698 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 03:11:31.289698 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 03:11:31.289698 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 03:11:31.289698 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 03:11:31.289698 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application with Lock( Sep 13 03:11:31.289698 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 03:11:31.289698 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._enter() Sep 13 03:11:31.289698 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 03:11:31.289698 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 03:11:31.289698 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 03:11:31.289698 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.creator() Sep 13 03:11:31.289698 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 03:11:31.289698 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application created_value = creator( Sep 13 03:11:31.289698 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 13 03:11:31.290839 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 13 03:11:31.290839 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 03:11:31.290839 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Sep 13 03:11:31.290839 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 643/1278] 10.222.0.18 () {56 vars in 1108 bytes} [Tue Sep 13 03:11:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 03:11:31.297382 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a0e5da7-bd18-4f71-aedb-30cec1651d12 None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:11:31.394166 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a0e5da7-bd18-4f71-aedb-30cec1651d12 None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:11:31.436213 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6dd3956-054e-4b06-aafd-f782f3bf537b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:11:31.436811 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6dd3956-054e-4b06-aafd-f782f3bf537b None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:11:31.437088 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6dd3956-054e-4b06-aafd-f782f3bf537b None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:11:31.437354 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6dd3956-054e-4b06-aafd-f782f3bf537b None admin] PATH_INFO: `/v3/domains` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:11:31.437983 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6dd3956-054e-4b06-aafd-f782f3bf537b None admin] RBAC: Adding query filter params (name=Default) {{(pid=81399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 03:11:31.438284 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6dd3956-054e-4b06-aafd-f782f3bf537b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:11:31.439670 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6dd3956-054e-4b06-aafd-f782f3bf537b None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:11:31.451599 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 636/1279] 10.222.0.18 () {56 vars in 1109 bytes} [Tue Sep 13 03:11:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 03:11:31.457284 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-106e8a81-43b6-4b2d-99b7-8de40f844f2b None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:11:31.487324 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:11:31.554060 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-106e8a81-43b6-4b2d-99b7-8de40f844f2b None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:11:31.586074 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fddd9b7c-4a8a-409f-bf4b-1ca1d76bf716 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:11:31.586699 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fddd9b7c-4a8a-409f-bf4b-1ca1d76bf716 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:11:31.586975 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fddd9b7c-4a8a-409f-bf4b-1ca1d76bf716 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:11:31.587243 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fddd9b7c-4a8a-409f-bf4b-1ca1d76bf716 None admin] PATH_INFO: `/v3/users/placement` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:11:31.594895 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fddd9b7c-4a8a-409f-bf4b-1ca1d76bf716 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:11:31.596317 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fddd9b7c-4a8a-409f-bf4b-1ca1d76bf716 None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:11:31.607119 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application [None req-fddd9b7c-4a8a-409f-bf4b-1ca1d76bf716 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Sep 13 03:11:31.607119 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 03:11:31.607119 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 03:11:31.607119 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 03:11:31.607119 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 03:11:31.607119 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 03:11:31.607119 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 03:11:31.607119 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 03:11:31.607119 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 03:11:31.607119 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 03:11:31.607119 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 03:11:31.607119 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 03:11:31.607119 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 13 03:11:31.607119 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 13 03:11:31.607119 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 13 03:11:31.607119 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 13 03:11:31.608232 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 03:11:31.608232 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 03:11:31.608232 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 13 03:11:31.608232 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 13 03:11:31.608232 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 13 03:11:31.608232 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 13 03:11:31.608232 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 03:11:31.608232 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 03:11:31.608232 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 03:11:31.608232 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 03:11:31.608232 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 03:11:31.608232 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application with Lock( Sep 13 03:11:31.608232 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 03:11:31.608232 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._enter() Sep 13 03:11:31.608232 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 03:11:31.608232 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 03:11:31.608232 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 03:11:31.608232 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.creator() Sep 13 03:11:31.609591 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 03:11:31.609591 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application created_value = creator( Sep 13 03:11:31.609591 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 13 03:11:31.609591 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 13 03:11:31.609591 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 13 03:11:31.609591 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 13 03:11:31.609591 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 13 03:11:31.609591 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 13 03:11:31.609591 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Sep 13 03:11:31.609591 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Sep 13 03:11:31.609591 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 644/1280] 10.222.0.18 () {56 vars in 1108 bytes} [Tue Sep 13 03:11:31 2022] GET /identity/v3/users/placement => generated 87 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 03:11:31.613783 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6dd3956-054e-4b06-aafd-f782f3bf537b None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:11:31.617505 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:11:31.711018 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6dd3956-054e-4b06-aafd-f782f3bf537b None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:11:31.744959 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f879f26a-6e93-4b55-9b56-c01b56c21166 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:11:31.745558 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f879f26a-6e93-4b55-9b56-c01b56c21166 None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:11:31.746326 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f879f26a-6e93-4b55-9b56-c01b56c21166 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:11:31.746597 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f879f26a-6e93-4b55-9b56-c01b56c21166 None admin] PATH_INFO: `/v3/users` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:11:31.747347 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f879f26a-6e93-4b55-9b56-c01b56c21166 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 03:11:31.747651 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f879f26a-6e93-4b55-9b56-c01b56c21166 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:11:31.749068 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f879f26a-6e93-4b55-9b56-c01b56c21166 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:11:31.766476 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 637/1281] 10.222.0.18 () {56 vars in 1143 bytes} [Tue Sep 13 03:11:31 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 03:11:31.772312 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fddd9b7c-4a8a-409f-bf4b-1ca1d76bf716 None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:11:31.860228 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:11:31.869986 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fddd9b7c-4a8a-409f-bf4b-1ca1d76bf716 None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:11:31.903377 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a324b6ea-cf22-45d3-842b-5a7c61136d14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:11:31.903961 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a324b6ea-cf22-45d3-842b-5a7c61136d14 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:11:31.904234 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a324b6ea-cf22-45d3-842b-5a7c61136d14 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:11:31.904499 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a324b6ea-cf22-45d3-842b-5a7c61136d14 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:11:31.969493 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a324b6ea-cf22-45d3-842b-5a7c61136d14 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:11:31.970873 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a324b6ea-cf22-45d3-842b-5a7c61136d14 None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:11:32.055366 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:11:32.065774 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 645/1282] 10.222.0.18 () {58 vars in 1282 bytes} [Tue Sep 13 03:11:31 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 294 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 13 03:11:32.071690 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f879f26a-6e93-4b55-9b56-c01b56c21166 None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:11:32.130233 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:11:32.167795 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f879f26a-6e93-4b55-9b56-c01b56c21166 None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:11:32.200391 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c60569d3-0f7b-477b-be24-729de3330188 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:11:32.201002 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c60569d3-0f7b-477b-be24-729de3330188 None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:11:32.201280 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c60569d3-0f7b-477b-be24-729de3330188 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:11:32.201552 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c60569d3-0f7b-477b-be24-729de3330188 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:11:32.211145 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c60569d3-0f7b-477b-be24-729de3330188 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:11:32.212572 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c60569d3-0f7b-477b-be24-729de3330188 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:11:32.227325 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application [None req-c60569d3-0f7b-477b-be24-729de3330188 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 03:11:32.227325 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 03:11:32.227325 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 03:11:32.227325 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 03:11:32.227325 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 03:11:32.227325 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 03:11:32.227325 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 03:11:32.227325 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 03:11:32.227325 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 03:11:32.227325 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 03:11:32.227325 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 03:11:32.227325 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 03:11:32.227325 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 13 03:11:32.227325 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 13 03:11:32.227325 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 13 03:11:32.227325 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 13 03:11:32.228419 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 03:11:32.228419 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 03:11:32.228419 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 03:11:32.228419 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 03:11:32.228419 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 03:11:32.228419 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 03:11:32.228419 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 03:11:32.228419 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application with Lock( Sep 13 03:11:32.228419 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 03:11:32.228419 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._enter() Sep 13 03:11:32.228419 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 03:11:32.228419 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 03:11:32.228419 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 03:11:32.228419 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.creator() Sep 13 03:11:32.228419 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 03:11:32.228419 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application created_value = creator( Sep 13 03:11:32.228419 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 13 03:11:32.229693 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 13 03:11:32.229693 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 03:11:32.229693 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Sep 13 03:11:32.229693 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 638/1283] 10.222.0.18 () {56 vars in 1108 bytes} [Tue Sep 13 03:11:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 13 03:11:32.233992 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a324b6ea-cf22-45d3-842b-5a7c61136d14 None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:11:32.329268 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a324b6ea-cf22-45d3-842b-5a7c61136d14 None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:11:32.361938 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-957fd356-d732-422a-acf2-8f1a05e7e45c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:11:32.362525 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-957fd356-d732-422a-acf2-8f1a05e7e45c None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:11:32.362827 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-957fd356-d732-422a-acf2-8f1a05e7e45c None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:11:32.363099 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-957fd356-d732-422a-acf2-8f1a05e7e45c None admin] PATH_INFO: `/v3/domains` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:11:32.363719 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-957fd356-d732-422a-acf2-8f1a05e7e45c None admin] RBAC: Adding query filter params (name=Default) {{(pid=81400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 03:11:32.364016 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-957fd356-d732-422a-acf2-8f1a05e7e45c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:11:32.365366 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-957fd356-d732-422a-acf2-8f1a05e7e45c None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:11:32.376718 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 646/1284] 10.222.0.18 () {56 vars in 1109 bytes} [Tue Sep 13 03:11:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 03:11:32.382195 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c60569d3-0f7b-477b-be24-729de3330188 None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:11:32.478603 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c60569d3-0f7b-477b-be24-729de3330188 None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:11:32.491978 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:11:32.511348 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6e0b85e-ac01-4b55-ba2d-e0dad8c9fe62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:11:32.511958 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e0b85e-ac01-4b55-ba2d-e0dad8c9fe62 None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:11:32.512235 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e0b85e-ac01-4b55-ba2d-e0dad8c9fe62 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:11:32.512506 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e0b85e-ac01-4b55-ba2d-e0dad8c9fe62 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:11:32.518837 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6e0b85e-ac01-4b55-ba2d-e0dad8c9fe62 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:11:32.520246 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6e0b85e-ac01-4b55-ba2d-e0dad8c9fe62 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:11:32.532418 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application [None req-b6e0b85e-ac01-4b55-ba2d-e0dad8c9fe62 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 13 03:11:32.532418 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 03:11:32.532418 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 03:11:32.532418 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 03:11:32.532418 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 03:11:32.532418 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 03:11:32.532418 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 03:11:32.532418 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 03:11:32.532418 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 03:11:32.532418 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 03:11:32.532418 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 03:11:32.532418 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 03:11:32.532418 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 13 03:11:32.532418 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 13 03:11:32.532418 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 13 03:11:32.533440 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 13 03:11:32.533440 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 03:11:32.533440 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 03:11:32.533440 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 03:11:32.533440 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 03:11:32.533440 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 03:11:32.533440 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 03:11:32.533440 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 03:11:32.533440 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application with Lock( Sep 13 03:11:32.533440 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 03:11:32.533440 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._enter() Sep 13 03:11:32.533440 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 03:11:32.533440 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 03:11:32.533440 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 03:11:32.533440 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.creator() Sep 13 03:11:32.533440 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 03:11:32.533440 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application created_value = creator( Sep 13 03:11:32.534373 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 13 03:11:32.534373 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 13 03:11:32.534373 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 13 03:11:32.534373 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 13 03:11:32.534373 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 13 03:11:32.534373 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 13 03:11:32.534373 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 13 03:11:32.534373 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: ERROR keystone.server.flask.application Sep 13 03:11:32.534373 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 639/1285] 10.222.0.18 () {56 vars in 1111 bytes} [Tue Sep 13 03:11:32 2022] GET /identity/v3/projects/service => generated 88 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 03:11:32.539169 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-957fd356-d732-422a-acf2-8f1a05e7e45c None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:11:32.587184 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:11:32.635378 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-957fd356-d732-422a-acf2-8f1a05e7e45c None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:11:32.667552 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2259b6f-f47c-457b-8904-16a87f871d2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:11:32.668720 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2259b6f-f47c-457b-8904-16a87f871d2e None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:11:32.668995 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2259b6f-f47c-457b-8904-16a87f871d2e None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:11:32.669353 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2259b6f-f47c-457b-8904-16a87f871d2e None admin] PATH_INFO: `/v3/projects` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:11:32.669943 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2259b6f-f47c-457b-8904-16a87f871d2e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 03:11:32.670250 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2259b6f-f47c-457b-8904-16a87f871d2e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:11:32.671698 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2259b6f-f47c-457b-8904-16a87f871d2e None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:11:32.683474 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 647/1286] 10.222.0.18 () {56 vars in 1148 bytes} [Tue Sep 13 03:11:32 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 03:11:32.689307 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6e0b85e-ac01-4b55-ba2d-e0dad8c9fe62 None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:11:32.789114 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6e0b85e-ac01-4b55-ba2d-e0dad8c9fe62 None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:11:32.820934 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35857cf4-798d-4723-8bce-65d2e17d74a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:11:32.821528 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35857cf4-798d-4723-8bce-65d2e17d74a4 None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:11:32.821806 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35857cf4-798d-4723-8bce-65d2e17d74a4 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:11:32.822080 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35857cf4-798d-4723-8bce-65d2e17d74a4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:11:32.822818 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35857cf4-798d-4723-8bce-65d2e17d74a4 None admin] RBAC: Adding query filter params (role.id=425d4b481c7846e0bc07e7e805d5b15f, scope.project.id=b9d94f6a68f7470d9494f112175620dc, user.id=4b99f1b2374d4d2f8db78306316180d9) {{(pid=81399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 03:11:32.823128 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35857cf4-798d-4723-8bce-65d2e17d74a4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:11:32.825057 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35857cf4-798d-4723-8bce-65d2e17d74a4 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:11:32.830633 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 640/1287] 10.222.0.18 () {56 vars in 1374 bytes} [Tue Sep 13 03:11:32 2022] GET /identity/v3/role_assignments?user.id=4b99f1b2374d4d2f8db78306316180d9&scope.project.id=b9d94f6a68f7470d9494f112175620dc&role.id=425d4b481c7846e0bc07e7e805d5b15f => generated 608 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 03:11:34.960203 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-166eed2c-b7ce-4ead-aff3-f1b6934ca574 None None] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:11:34.960770 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-166eed2c-b7ce-4ead-aff3-f1b6934ca574 None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:11:34.960770 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-166eed2c-b7ce-4ead-aff3-f1b6934ca574 None None] PATH_INFO: `/` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:11:34.962090 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 648/1288] 10.222.0.18 () {54 vars in 927 bytes} [Tue Sep 13 03:11:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 03:11:34.967880 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88273d51-3a58-4019-a40d-d897fbb14c7b None None] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:11:34.968304 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88273d51-3a58-4019-a40d-d897fbb14c7b None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:11:34.968425 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88273d51-3a58-4019-a40d-d897fbb14c7b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:11:34.986101 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.auth.core [None req-88273d51-3a58-4019-a40d-d897fbb14c7b None None] MFA Rules not processed for user `49f280ab086b4610855c4ffa378517d9`. Rule list: `[]` (Enabled: `True`). {{(pid=81399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 03:11:34.987309 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-88273d51-3a58-4019-a40d-d897fbb14c7b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:11:35.015763 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:11:35.052391 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 641/1289] 10.222.0.18 () {58 vars in 1024 bytes} [Tue Sep 13 03:11:34 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 13 03:11:35.319032 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-166eed2c-b7ce-4ead-aff3-f1b6934ca574 None None] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:11:35.417281 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-166eed2c-b7ce-4ead-aff3-f1b6934ca574 None None] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:11:35.427755 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:11:35.451547 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-972ccffd-1166-4b8b-b08d-98f1ceb1ee80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:11:35.452162 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-972ccffd-1166-4b8b-b08d-98f1ceb1ee80 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:11:35.452441 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-972ccffd-1166-4b8b-b08d-98f1ceb1ee80 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:11:35.452710 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-972ccffd-1166-4b8b-b08d-98f1ceb1ee80 None admin] PATH_INFO: `/v3/services/placement` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:11:35.453277 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-972ccffd-1166-4b8b-b08d-98f1ceb1ee80 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:11:35.454664 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-972ccffd-1166-4b8b-b08d-98f1ceb1ee80 None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:11:35.463372 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application [None req-972ccffd-1166-4b8b-b08d-98f1ceb1ee80 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Sep 13 03:11:35.463372 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 03:11:35.463372 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 13 03:11:35.463372 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 13 03:11:35.463372 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 13 03:11:35.463372 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 13 03:11:35.463372 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 13 03:11:35.463372 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 13 03:11:35.463372 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Sep 13 03:11:35.463372 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Sep 13 03:11:35.463372 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 13 03:11:35.463372 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Sep 13 03:11:35.463372 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 03:11:35.463372 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 03:11:35.463372 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 03:11:35.463372 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 03:11:35.463372 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 03:11:35.464594 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 03:11:35.464594 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 03:11:35.464594 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 03:11:35.464594 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 03:11:35.464594 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 03:11:35.464594 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 03:11:35.464594 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 13 03:11:35.464594 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 13 03:11:35.464594 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 13 03:11:35.464594 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 13 03:11:35.464594 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 03:11:35.464594 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 03:11:35.464594 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 03:11:35.464594 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 03:11:35.464594 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 03:11:35.464594 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 03:11:35.465624 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 03:11:35.465624 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application with Lock( Sep 13 03:11:35.465624 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 03:11:35.465624 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._enter() Sep 13 03:11:35.465624 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 03:11:35.465624 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 03:11:35.465624 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 03:11:35.465624 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.creator() Sep 13 03:11:35.465624 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 03:11:35.465624 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application created_value = creator( Sep 13 03:11:35.465624 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 13 03:11:35.465624 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 13 03:11:35.465624 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Sep 13 03:11:35.465624 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Sep 13 03:11:35.465624 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 649/1290] 10.222.0.18 () {56 vars in 1117 bytes} [Tue Sep 13 03:11:35 2022] GET /identity/v3/services/placement => generated 90 bytes in 146 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 03:11:35.470354 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88273d51-3a58-4019-a40d-d897fbb14c7b None None] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:11:35.564439 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88273d51-3a58-4019-a40d-d897fbb14c7b None None] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:11:35.596651 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af1daf7f-44ff-4251-a8c9-8acaea71c924 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:11:35.597181 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af1daf7f-44ff-4251-a8c9-8acaea71c924 None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:11:35.597457 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af1daf7f-44ff-4251-a8c9-8acaea71c924 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:11:35.597725 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af1daf7f-44ff-4251-a8c9-8acaea71c924 None admin] PATH_INFO: `/v3/services` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:11:35.598341 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af1daf7f-44ff-4251-a8c9-8acaea71c924 None admin] RBAC: Adding query filter params (name=placement) {{(pid=81399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 03:11:35.598670 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af1daf7f-44ff-4251-a8c9-8acaea71c924 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:11:35.600018 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af1daf7f-44ff-4251-a8c9-8acaea71c924 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:11:35.604613 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 642/1291] 10.222.0.18 () {56 vars in 1116 bytes} [Tue Sep 13 03:11:35 2022] GET /identity/v3/services?name=placement => generated 126 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 03:11:35.609931 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-972ccffd-1166-4b8b-b08d-98f1ceb1ee80 None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:11:35.707642 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-972ccffd-1166-4b8b-b08d-98f1ceb1ee80 None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:11:35.741118 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-618707b9-35d3-4fb9-a730-af02b7ee863c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:11:35.741925 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-618707b9-35d3-4fb9-a730-af02b7ee863c None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:11:35.742276 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-618707b9-35d3-4fb9-a730-af02b7ee863c None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:11:35.742653 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-618707b9-35d3-4fb9-a730-af02b7ee863c None admin] PATH_INFO: `/v3/services` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:11:35.743569 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-618707b9-35d3-4fb9-a730-af02b7ee863c None admin] RBAC: Adding query filter params (type=placement) {{(pid=81400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 03:11:35.744001 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-618707b9-35d3-4fb9-a730-af02b7ee863c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:11:35.745919 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-618707b9-35d3-4fb9-a730-af02b7ee863c None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:11:35.752107 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 650/1292] 10.222.0.18 () {56 vars in 1116 bytes} [Tue Sep 13 03:11:35 2022] GET /identity/v3/services?type=placement => generated 126 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 03:11:37.859042 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a9b815b-8920-4f32-95a2-37ac78100e4b None None] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:11:37.859567 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a9b815b-8920-4f32-95a2-37ac78100e4b None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:11:37.859567 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a9b815b-8920-4f32-95a2-37ac78100e4b None None] PATH_INFO: `/` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:11:37.860925 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 643/1293] 10.222.0.18 () {54 vars in 927 bytes} [Tue Sep 13 03:11:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 03:11:37.865929 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c034ec9-4b30-475d-b2db-89cfb1edc04c None None] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:11:37.866364 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c034ec9-4b30-475d-b2db-89cfb1edc04c None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:11:37.866553 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c034ec9-4b30-475d-b2db-89cfb1edc04c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:11:37.884187 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.auth.core [None req-3c034ec9-4b30-475d-b2db-89cfb1edc04c None None] MFA Rules not processed for user `49f280ab086b4610855c4ffa378517d9`. Rule list: `[]` (Enabled: `True`). {{(pid=81400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 03:11:37.885354 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-3c034ec9-4b30-475d-b2db-89cfb1edc04c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:11:37.913986 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:11:37.951461 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 651/1294] 10.222.0.18 () {58 vars in 1024 bytes} [Tue Sep 13 03:11:37 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 13 03:11:38.214868 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a9b815b-8920-4f32-95a2-37ac78100e4b None None] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:11:38.246683 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:11:38.315141 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a9b815b-8920-4f32-95a2-37ac78100e4b None None] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:11:38.347589 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02a87eb0-efe9-4bad-909f-7e823255a563 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:11:38.348175 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02a87eb0-efe9-4bad-909f-7e823255a563 None admin] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:11:38.348456 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02a87eb0-efe9-4bad-909f-7e823255a563 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:11:38.348729 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02a87eb0-efe9-4bad-909f-7e823255a563 None admin] PATH_INFO: `/v3/services` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:11:38.349378 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02a87eb0-efe9-4bad-909f-7e823255a563 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:11:38.350767 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02a87eb0-efe9-4bad-909f-7e823255a563 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:11:38.364710 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 644/1295] 10.222.0.18 () {60 vars in 1141 bytes} [Tue Sep 13 03:11:38 2022] POST /identity/v3/services => generated 246 bytes in 150 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 03:11:40.496920 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b1aac9-a4cc-4ed1-9787-c662ce7d97f7 None None] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:11:40.497434 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b1aac9-a4cc-4ed1-9787-c662ce7d97f7 None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:11:40.497434 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b1aac9-a4cc-4ed1-9787-c662ce7d97f7 None None] PATH_INFO: `/` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:11:40.498819 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 652/1296] 10.222.0.18 () {54 vars in 927 bytes} [Tue Sep 13 03:11:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 03:11:40.503933 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-495c8268-c88a-409a-802c-60f31076c773 None None] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:11:40.504376 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-495c8268-c88a-409a-802c-60f31076c773 None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:11:40.504472 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-495c8268-c88a-409a-802c-60f31076c773 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:11:40.522529 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.auth.core [None req-495c8268-c88a-409a-802c-60f31076c773 None None] MFA Rules not processed for user `49f280ab086b4610855c4ffa378517d9`. Rule list: `[]` (Enabled: `True`). {{(pid=81399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 03:11:40.523721 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-495c8268-c88a-409a-802c-60f31076c773 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:11:40.593224 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 645/1297] 10.222.0.18 () {58 vars in 1024 bytes} [Tue Sep 13 03:11:40 2022] POST /identity/v3/auth/tokens => generated 2221 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 13 03:11:40.869817 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09b1aac9-a4cc-4ed1-9787-c662ce7d97f7 None None] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:11:40.967935 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09b1aac9-a4cc-4ed1-9787-c662ce7d97f7 None None] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:11:41.000936 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63fb086a-2873-41a7-b111-d61f675a55bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:11:41.001534 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63fb086a-2873-41a7-b111-d61f675a55bc None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:11:41.001807 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63fb086a-2873-41a7-b111-d61f675a55bc None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:11:41.002075 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63fb086a-2873-41a7-b111-d61f675a55bc None admin] PATH_INFO: `/v3/services/placement` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:11:41.002665 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63fb086a-2873-41a7-b111-d61f675a55bc None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:11:41.004015 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63fb086a-2873-41a7-b111-d61f675a55bc None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:11:41.012529 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application [None req-63fb086a-2873-41a7-b111-d61f675a55bc None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Sep 13 03:11:41.012529 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 03:11:41.012529 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 13 03:11:41.012529 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 13 03:11:41.012529 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 13 03:11:41.012529 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 13 03:11:41.012529 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 13 03:11:41.012529 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 13 03:11:41.012529 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Sep 13 03:11:41.012529 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Sep 13 03:11:41.012529 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 13 03:11:41.012529 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Sep 13 03:11:41.012529 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 03:11:41.012529 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 03:11:41.012529 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 03:11:41.012529 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 03:11:41.012529 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 03:11:41.013690 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 03:11:41.013690 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 03:11:41.013690 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 03:11:41.013690 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 03:11:41.013690 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 03:11:41.013690 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 03:11:41.013690 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 13 03:11:41.013690 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 13 03:11:41.013690 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 13 03:11:41.013690 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 13 03:11:41.013690 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 03:11:41.013690 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 03:11:41.013690 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 03:11:41.013690 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 03:11:41.013690 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 03:11:41.013690 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 03:11:41.014632 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 03:11:41.014632 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application with Lock( Sep 13 03:11:41.014632 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 03:11:41.014632 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._enter() Sep 13 03:11:41.014632 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 03:11:41.014632 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 03:11:41.014632 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 03:11:41.014632 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.creator() Sep 13 03:11:41.014632 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 03:11:41.014632 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application created_value = creator( Sep 13 03:11:41.014632 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 13 03:11:41.014632 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 13 03:11:41.014632 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Sep 13 03:11:41.014632 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Sep 13 03:11:41.014632 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 653/1298] 10.222.0.18 () {56 vars in 1117 bytes} [Tue Sep 13 03:11:40 2022] GET /identity/v3/services/placement => generated 90 bytes in 144 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 03:11:41.019470 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-495c8268-c88a-409a-802c-60f31076c773 None None] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:11:41.037027 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:11:41.115915 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-495c8268-c88a-409a-802c-60f31076c773 None None] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:11:41.149149 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4336f740-1b3a-4d1f-9fbd-390a697b14d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:11:41.149689 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4336f740-1b3a-4d1f-9fbd-390a697b14d5 None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:11:41.149969 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4336f740-1b3a-4d1f-9fbd-390a697b14d5 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:11:41.150241 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4336f740-1b3a-4d1f-9fbd-390a697b14d5 None admin] PATH_INFO: `/v3/services` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:11:41.150898 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4336f740-1b3a-4d1f-9fbd-390a697b14d5 None admin] RBAC: Adding query filter params (name=placement) {{(pid=81399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 03:11:41.151200 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4336f740-1b3a-4d1f-9fbd-390a697b14d5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:11:41.152599 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4336f740-1b3a-4d1f-9fbd-390a697b14d5 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:11:41.157773 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 646/1299] 10.222.0.18 () {56 vars in 1116 bytes} [Tue Sep 13 03:11:41 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 03:11:41.163810 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63fb086a-2873-41a7-b111-d61f675a55bc None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:11:41.195362 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:11:41.261943 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63fb086a-2873-41a7-b111-d61f675a55bc None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:11:41.294243 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d803bb9f-cdad-40ed-845f-781b8aca48d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:11:41.294862 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d803bb9f-cdad-40ed-845f-781b8aca48d0 None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:11:41.295135 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d803bb9f-cdad-40ed-845f-781b8aca48d0 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:11:41.295398 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d803bb9f-cdad-40ed-845f-781b8aca48d0 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:11:41.296080 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d803bb9f-cdad-40ed-845f-781b8aca48d0 None admin] RBAC: Adding query filter params (interface=public, service_id=a62f37f0ae8a4f9389e981c337195d4e, region_id=RegionOne) {{(pid=81400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 03:11:41.296380 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d803bb9f-cdad-40ed-845f-781b8aca48d0 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:11:41.297721 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d803bb9f-cdad-40ed-845f-781b8aca48d0 None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:11:41.302769 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 654/1300] 10.222.0.18 () {56 vars in 1251 bytes} [Tue Sep 13 03:11:41 2022] GET /identity/v3/endpoints?service_id=a62f37f0ae8a4f9389e981c337195d4e&interface=public®ion_id=RegionOne => generated 194 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 03:11:41.308937 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4336f740-1b3a-4d1f-9fbd-390a697b14d5 None admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:11:41.386572 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:11:41.405103 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4336f740-1b3a-4d1f-9fbd-390a697b14d5 None admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:11:41.437868 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad4599f8-13e6-47bd-8359-bf72f927e9a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:11:41.438461 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad4599f8-13e6-47bd-8359-bf72f927e9a8 None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:11:41.438766 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad4599f8-13e6-47bd-8359-bf72f927e9a8 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:11:41.439036 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad4599f8-13e6-47bd-8359-bf72f927e9a8 None admin] PATH_INFO: `/v3/services` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:11:41.439635 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad4599f8-13e6-47bd-8359-bf72f927e9a8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:11:41.440963 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad4599f8-13e6-47bd-8359-bf72f927e9a8 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:11:41.447531 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 647/1301] 10.222.0.18 () {56 vars in 1087 bytes} [Tue Sep 13 03:11:41 2022] GET /identity/v3/services => generated 2191 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 13 03:11:43.578328 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfba2cb2-f7f3-4107-afcc-0846c7f58824 None None] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:11:43.578883 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfba2cb2-f7f3-4107-afcc-0846c7f58824 None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:11:43.578883 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfba2cb2-f7f3-4107-afcc-0846c7f58824 None None] PATH_INFO: `/` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:11:43.580228 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 655/1302] 10.222.0.18 () {54 vars in 927 bytes} [Tue Sep 13 03:11:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 03:11:43.585401 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c104f968-0103-4594-ac4d-21a6b61b56fa None None] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:11:43.585816 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c104f968-0103-4594-ac4d-21a6b61b56fa None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:11:43.585945 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c104f968-0103-4594-ac4d-21a6b61b56fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:11:43.604154 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.auth.core [None req-c104f968-0103-4594-ac4d-21a6b61b56fa None None] MFA Rules not processed for user `49f280ab086b4610855c4ffa378517d9`. Rule list: `[]` (Enabled: `True`). {{(pid=81399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 03:11:43.605319 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-c104f968-0103-4594-ac4d-21a6b61b56fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:11:43.668966 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 648/1303] 10.222.0.18 () {58 vars in 1024 bytes} [Tue Sep 13 03:11:43 2022] POST /identity/v3/auth/tokens => generated 2221 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 13 03:11:43.942035 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfba2cb2-f7f3-4107-afcc-0846c7f58824 None None] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:11:44.034390 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:11:44.042754 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfba2cb2-f7f3-4107-afcc-0846c7f58824 None None] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:11:44.076486 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f1998ba-435e-4e6f-bb4b-1adde019c9fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:11:44.077109 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f1998ba-435e-4e6f-bb4b-1adde019c9fe None admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:11:44.077381 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f1998ba-435e-4e6f-bb4b-1adde019c9fe None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:11:44.077647 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f1998ba-435e-4e6f-bb4b-1adde019c9fe None admin] PATH_INFO: `/v3/services/placement` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:11:44.078214 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f1998ba-435e-4e6f-bb4b-1adde019c9fe None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:11:44.079633 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f1998ba-435e-4e6f-bb4b-1adde019c9fe None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:11:44.088193 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application [None req-7f1998ba-435e-4e6f-bb4b-1adde019c9fe None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Sep 13 03:11:44.088193 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 03:11:44.088193 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 13 03:11:44.088193 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 13 03:11:44.088193 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 13 03:11:44.088193 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 13 03:11:44.088193 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 13 03:11:44.088193 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 13 03:11:44.088193 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Sep 13 03:11:44.088193 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Sep 13 03:11:44.088193 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 13 03:11:44.088193 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Sep 13 03:11:44.088193 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 03:11:44.088193 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 03:11:44.088193 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 03:11:44.088193 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 03:11:44.088193 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 03:11:44.089385 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 03:11:44.089385 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 03:11:44.089385 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 03:11:44.089385 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 03:11:44.089385 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 03:11:44.089385 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 03:11:44.089385 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 13 03:11:44.089385 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 13 03:11:44.089385 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 13 03:11:44.089385 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 13 03:11:44.089385 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 03:11:44.089385 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 03:11:44.089385 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 03:11:44.089385 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 03:11:44.089385 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 03:11:44.089385 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 03:11:44.090593 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 03:11:44.090593 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application with Lock( Sep 13 03:11:44.090593 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 03:11:44.090593 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self._enter() Sep 13 03:11:44.090593 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 03:11:44.090593 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 03:11:44.090593 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 03:11:44.090593 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application return self.creator() Sep 13 03:11:44.090593 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 03:11:44.090593 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application created_value = creator( Sep 13 03:11:44.090593 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 13 03:11:44.090593 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 13 03:11:44.090593 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Sep 13 03:11:44.090593 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: ERROR keystone.server.flask.application Sep 13 03:11:44.090593 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 656/1304] 10.222.0.18 () {56 vars in 1117 bytes} [Tue Sep 13 03:11:43 2022] GET /identity/v3/services/placement => generated 90 bytes in 148 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 03:11:44.095190 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c104f968-0103-4594-ac4d-21a6b61b56fa None None] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:11:44.166287 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:11:44.189474 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c104f968-0103-4594-ac4d-21a6b61b56fa None None] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:11:44.220853 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcbe22b5-93cd-4cad-a7bd-c7af5d66b568 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:11:44.221386 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcbe22b5-93cd-4cad-a7bd-c7af5d66b568 None admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:11:44.221658 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcbe22b5-93cd-4cad-a7bd-c7af5d66b568 None admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:11:44.221925 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcbe22b5-93cd-4cad-a7bd-c7af5d66b568 None admin] PATH_INFO: `/v3/services` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:11:44.222532 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcbe22b5-93cd-4cad-a7bd-c7af5d66b568 None admin] RBAC: Adding query filter params (name=placement) {{(pid=81399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 03:11:44.222855 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcbe22b5-93cd-4cad-a7bd-c7af5d66b568 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:11:44.224202 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcbe22b5-93cd-4cad-a7bd-c7af5d66b568 None admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:11:44.229074 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 649/1305] 10.222.0.18 () {56 vars in 1116 bytes} [Tue Sep 13 03:11:44 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 03:11:44.235054 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f1998ba-435e-4e6f-bb4b-1adde019c9fe None admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:11:44.330925 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f1998ba-435e-4e6f-bb4b-1adde019c9fe None admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:11:44.364458 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58917f24-98a3-4a4c-acdc-ed1cb0311ed0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:11:44.365036 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58917f24-98a3-4a4c-acdc-ed1cb0311ed0 None admin] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:11:44.365308 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58917f24-98a3-4a4c-acdc-ed1cb0311ed0 None admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:11:44.365574 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58917f24-98a3-4a4c-acdc-ed1cb0311ed0 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:11:44.366223 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58917f24-98a3-4a4c-acdc-ed1cb0311ed0 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:11:44.367663 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58917f24-98a3-4a4c-acdc-ed1cb0311ed0 None admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:11:44.386531 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 657/1306] 10.222.0.18 () {60 vars in 1144 bytes} [Tue Sep 13 03:11:44 2022] POST /identity/v3/endpoints => generated 331 bytes in 152 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 13 03:11:54.226352 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=30 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:11:54.382725 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=91,keystone:INSERT=1 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:12:47.872293 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-645da67c-674e-484c-8d5e-3075aa4de533 None None] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:12:47.872765 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-645da67c-674e-484c-8d5e-3075aa4de533 None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:12:47.873014 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-645da67c-674e-484c-8d5e-3075aa4de533 None None] PATH_INFO: `/` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:12:47.878415 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 650/1307] 10.222.0.18 () {54 vars in 927 bytes} [Tue Sep 13 03:12:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 03:12:47.879580 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65d0a989-8c23-4045-9fbd-14c058fe5595 None None] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:12:47.879975 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65d0a989-8c23-4045-9fbd-14c058fe5595 None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:12:47.880225 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65d0a989-8c23-4045-9fbd-14c058fe5595 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:12:48.081080 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.auth.core [None req-65d0a989-8c23-4045-9fbd-14c058fe5595 None None] MFA Rules not processed for user `49f280ab086b4610855c4ffa378517d9`. Rule list: `[]` (Enabled: `True`). {{(pid=81400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 03:12:48.082520 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-65d0a989-8c23-4045-9fbd-14c058fe5595 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:12:48.228860 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 658/1308] 10.222.0.18 () {58 vars in 1024 bytes} [Tue Sep 13 03:12:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 350 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 03:12:48.526288 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a720040-d819-4db6-9bd5-8d4166bcd2d7 None None] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:12:48.526763 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a720040-d819-4db6-9bd5-8d4166bcd2d7 None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:12:48.527150 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a720040-d819-4db6-9bd5-8d4166bcd2d7 None None] PATH_INFO: `/` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:12:48.529068 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 651/1309] 10.222.0.18 () {54 vars in 965 bytes} [Tue Sep 13 03:12:48 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 03:12:48.534640 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d159374d-1356-4cd9-a115-fd11620b565d None None] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:12:48.534936 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d159374d-1356-4cd9-a115-fd11620b565d None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:12:48.535208 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d159374d-1356-4cd9-a115-fd11620b565d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:12:48.649030 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.auth.core [None req-d159374d-1356-4cd9-a115-fd11620b565d None None] MFA Rules not processed for user `cdffbc851086441ebd01022d1e4c2609`. Rule list: `[]` (Enabled: `True`). {{(pid=81400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 03:12:48.651250 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-d159374d-1356-4cd9-a115-fd11620b565d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:12:48.675688 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:12:48.749492 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 659/1310] 10.222.0.18 () {58 vars in 1062 bytes} [Tue Sep 13 03:12:48 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 216 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 03:12:48.759090 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a720040-d819-4db6-9bd5-8d4166bcd2d7 None None] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:12:48.761779 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-6a720040-d819-4db6-9bd5-8d4166bcd2d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:12:48.887945 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a720040-d819-4db6-9bd5-8d4166bcd2d7 None None] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:12:48.969970 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d55648a5-e463-4072-854a-8729b8d15931 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdffbc851086441ebd01022d1e4c2609', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b9d94f6a68f7470d9494f112175620dc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:12:48.970723 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d55648a5-e463-4072-854a-8729b8d15931 service neutron] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:12:48.971073 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d55648a5-e463-4072-854a-8729b8d15931 service neutron] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:12:48.971415 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d55648a5-e463-4072-854a-8729b8d15931 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:12:48.973115 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-d55648a5-e463-4072-854a-8729b8d15931 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:12:49.040926 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:12:49.105298 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d55648a5-e463-4072-854a-8729b8d15931 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:12:49.106869 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d55648a5-e463-4072-854a-8729b8d15931 service neutron] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:12:49.127810 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 652/1311] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Sep 13 03:12:48 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 369 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 03:12:51.616170 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-589378e2-a754-4168-8a8f-b4f02336d2af None None] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:12:51.616731 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-589378e2-a754-4168-8a8f-b4f02336d2af None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:12:51.616731 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-589378e2-a754-4168-8a8f-b4f02336d2af None None] PATH_INFO: `/` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:12:51.618039 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 660/1312] 10.222.0.18 () {54 vars in 927 bytes} [Tue Sep 13 03:12:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 03:12:51.623420 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8e206fc-626e-40f6-a9ac-a92f8f04c35f None None] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:12:51.623803 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8e206fc-626e-40f6-a9ac-a92f8f04c35f None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:12:51.623973 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8e206fc-626e-40f6-a9ac-a92f8f04c35f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:12:51.704180 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.auth.core [None req-b8e206fc-626e-40f6-a9ac-a92f8f04c35f None None] MFA Rules not processed for user `49f280ab086b4610855c4ffa378517d9`. Rule list: `[]` (Enabled: `True`). {{(pid=81399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 03:12:51.705307 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-b8e206fc-626e-40f6-a9ac-a92f8f04c35f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:12:51.801292 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 653/1313] 10.222.0.18 () {58 vars in 1024 bytes} [Tue Sep 13 03:12:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 03:12:52.091150 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-589378e2-a754-4168-8a8f-b4f02336d2af None None] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:12:52.092709 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-589378e2-a754-4168-8a8f-b4f02336d2af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:12:52.194115 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-589378e2-a754-4168-8a8f-b4f02336d2af None None] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:12:52.264895 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:12:52.267180 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3ca21ac-3f39-47a3-be30-7660b83b9013 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdffbc851086441ebd01022d1e4c2609', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b9d94f6a68f7470d9494f112175620dc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:12:52.267960 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ca21ac-3f39-47a3-be30-7660b83b9013 service neutron] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:12:52.268440 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ca21ac-3f39-47a3-be30-7660b83b9013 service neutron] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:12:52.268899 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ca21ac-3f39-47a3-be30-7660b83b9013 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:12:52.270525 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-f3ca21ac-3f39-47a3-be30-7660b83b9013 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:12:52.377536 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3ca21ac-3f39-47a3-be30-7660b83b9013 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:12:52.378933 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3ca21ac-3f39-47a3-be30-7660b83b9013 service neutron] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:12:52.396215 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 661/1314] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Sep 13 03:12:52 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 306 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 03:12:54.840998 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ec9ff7f-798e-47b2-b835-ba6e49ef3f20 None None] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:12:54.841579 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ec9ff7f-798e-47b2-b835-ba6e49ef3f20 None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:12:54.841579 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ec9ff7f-798e-47b2-b835-ba6e49ef3f20 None None] PATH_INFO: `/` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:12:54.842925 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 654/1315] 10.222.0.18 () {54 vars in 927 bytes} [Tue Sep 13 03:12:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 03:12:54.848180 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec8ad91f-5200-44cd-b2f5-6b3e82e8d9d1 None None] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:12:54.848570 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec8ad91f-5200-44cd-b2f5-6b3e82e8d9d1 None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:12:54.848721 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec8ad91f-5200-44cd-b2f5-6b3e82e8d9d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:12:54.913425 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.auth.core [None req-ec8ad91f-5200-44cd-b2f5-6b3e82e8d9d1 None None] MFA Rules not processed for user `06632bd878b1474f8d5432f99e899155`. Rule list: `[]` (Enabled: `True`). {{(pid=81400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 03:12:54.914635 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-ec8ad91f-5200-44cd-b2f5-6b3e82e8d9d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:12:54.965657 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:12:55.009723 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 662/1316] 10.222.0.18 () {58 vars in 1024 bytes} [Tue Sep 13 03:12:54 2022] POST /identity/v3/auth/tokens => generated 3243 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 03:12:55.298855 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ec9ff7f-798e-47b2-b835-ba6e49ef3f20 None None] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:12:55.300419 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-5ec9ff7f-798e-47b2-b835-ba6e49ef3f20 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:12:55.310346 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:12:55.403343 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ec9ff7f-798e-47b2-b835-ba6e49ef3f20 None None] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:12:55.476667 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4f4a09c-485a-4e2b-8452-23e983d5fa2f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdffbc851086441ebd01022d1e4c2609', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b9d94f6a68f7470d9494f112175620dc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:12:55.477267 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f4a09c-485a-4e2b-8452-23e983d5fa2f service neutron] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:12:55.477549 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f4a09c-485a-4e2b-8452-23e983d5fa2f service neutron] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:12:55.477814 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f4a09c-485a-4e2b-8452-23e983d5fa2f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:12:55.479466 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-c4f4a09c-485a-4e2b-8452-23e983d5fa2f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:12:55.540075 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:12:55.587840 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4f4a09c-485a-4e2b-8452-23e983d5fa2f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:12:55.589202 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4f4a09c-485a-4e2b-8452-23e983d5fa2f service neutron] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:12:55.607045 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 655/1317] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Sep 13 03:12:55 2022] GET /identity/v3/auth/tokens => generated 3243 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 03:12:58.265385 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbf430a8-acc6-4a7e-93e1-aeec9549f0bc None None] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:12:58.265908 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbf430a8-acc6-4a7e-93e1-aeec9549f0bc None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:12:58.265908 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbf430a8-acc6-4a7e-93e1-aeec9549f0bc None None] PATH_INFO: `/` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:12:58.267338 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 663/1318] 10.222.0.18 () {54 vars in 927 bytes} [Tue Sep 13 03:12:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 03:12:58.272401 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52f96283-eef7-4ab8-a7a5-6eadaad21173 None None] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:12:58.272769 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52f96283-eef7-4ab8-a7a5-6eadaad21173 None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:12:58.272945 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52f96283-eef7-4ab8-a7a5-6eadaad21173 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:12:58.337458 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.auth.core [None req-52f96283-eef7-4ab8-a7a5-6eadaad21173 None None] MFA Rules not processed for user `49f280ab086b4610855c4ffa378517d9`. Rule list: `[]` (Enabled: `True`). {{(pid=81399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 03:12:58.338567 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-52f96283-eef7-4ab8-a7a5-6eadaad21173 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:12:58.433679 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 656/1319] 10.222.0.18 () {58 vars in 1024 bytes} [Tue Sep 13 03:12:58 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 03:12:58.718071 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbf430a8-acc6-4a7e-93e1-aeec9549f0bc None None] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:12:58.719681 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-dbf430a8-acc6-4a7e-93e1-aeec9549f0bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:12:58.822927 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbf430a8-acc6-4a7e-93e1-aeec9549f0bc None None] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:12:58.895034 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d963bbb-a042-4e04-860c-4d2f448b955e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdffbc851086441ebd01022d1e4c2609', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b9d94f6a68f7470d9494f112175620dc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:12:58.895635 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d963bbb-a042-4e04-860c-4d2f448b955e service neutron] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:12:58.895913 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d963bbb-a042-4e04-860c-4d2f448b955e service neutron] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:12:58.896189 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d963bbb-a042-4e04-860c-4d2f448b955e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:12:58.897763 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-6d963bbb-a042-4e04-860c-4d2f448b955e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:12:58.904981 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:12:59.004726 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d963bbb-a042-4e04-860c-4d2f448b955e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:12:59.006084 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d963bbb-a042-4e04-860c-4d2f448b955e service neutron] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:12:59.024498 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 664/1320] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Sep 13 03:12:58 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 307 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 03:13:01.806479 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46630479-2015-4ebf-a7b6-0f4f66b45234 None None] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:13:01.807008 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46630479-2015-4ebf-a7b6-0f4f66b45234 None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:13:01.807116 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46630479-2015-4ebf-a7b6-0f4f66b45234 None None] PATH_INFO: `/` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:13:01.808481 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 657/1321] 10.222.0.18 () {54 vars in 927 bytes} [Tue Sep 13 03:13:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 03:13:01.813656 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed25caa0-721e-4bc4-80fe-2e3ebea5ff0e None None] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:13:01.814064 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed25caa0-721e-4bc4-80fe-2e3ebea5ff0e None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:13:01.814197 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed25caa0-721e-4bc4-80fe-2e3ebea5ff0e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:13:01.887512 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.auth.core [None req-ed25caa0-721e-4bc4-80fe-2e3ebea5ff0e None None] MFA Rules not processed for user `49f280ab086b4610855c4ffa378517d9`. Rule list: `[]` (Enabled: `True`). {{(pid=81400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 03:13:01.888751 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-ed25caa0-721e-4bc4-80fe-2e3ebea5ff0e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:13:01.951719 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:13:01.993274 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 665/1322] 10.222.0.18 () {58 vars in 1024 bytes} [Tue Sep 13 03:13:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 03:13:02.274789 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46630479-2015-4ebf-a7b6-0f4f66b45234 None None] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:13:02.276882 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-46630479-2015-4ebf-a7b6-0f4f66b45234 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:13:02.296441 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:13:02.381742 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46630479-2015-4ebf-a7b6-0f4f66b45234 None None] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:13:02.456401 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-578b31a4-fcde-4010-9bd1-473afdbe631c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdffbc851086441ebd01022d1e4c2609', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b9d94f6a68f7470d9494f112175620dc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:13:02.457007 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-578b31a4-fcde-4010-9bd1-473afdbe631c service neutron] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:13:02.457285 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-578b31a4-fcde-4010-9bd1-473afdbe631c service neutron] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:13:02.457555 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-578b31a4-fcde-4010-9bd1-473afdbe631c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:13:02.459218 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-578b31a4-fcde-4010-9bd1-473afdbe631c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:13:02.532724 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:13:02.572251 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-578b31a4-fcde-4010-9bd1-473afdbe631c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:13:02.574364 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-578b31a4-fcde-4010-9bd1-473afdbe631c service neutron] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:13:02.592175 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 658/1323] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Sep 13 03:13:02 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 318 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 03:13:04.802375 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca3d59e-3d91-48bd-8965-1a8c04e2eea8 None None] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:13:04.802961 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca3d59e-3d91-48bd-8965-1a8c04e2eea8 None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:13:04.802961 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca3d59e-3d91-48bd-8965-1a8c04e2eea8 None None] PATH_INFO: `/` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:13:04.804320 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 666/1324] 10.222.0.18 () {54 vars in 927 bytes} [Tue Sep 13 03:13:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 03:13:04.809765 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e33c12-7771-4381-ac31-2bcb3445f7aa None None] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:13:04.810223 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e33c12-7771-4381-ac31-2bcb3445f7aa None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:13:04.810321 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e33c12-7771-4381-ac31-2bcb3445f7aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:13:04.882348 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.auth.core [None req-f4e33c12-7771-4381-ac31-2bcb3445f7aa None None] MFA Rules not processed for user `49f280ab086b4610855c4ffa378517d9`. Rule list: `[]` (Enabled: `True`). {{(pid=81399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 03:13:04.883676 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-f4e33c12-7771-4381-ac31-2bcb3445f7aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:13:04.980570 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 659/1325] 10.222.0.18 () {58 vars in 1024 bytes} [Tue Sep 13 03:13:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 03:13:05.271225 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ce0a16-5850-42e7-ac75-5c20529c4d7e None None] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:13:05.271504 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ce0a16-5850-42e7-ac75-5c20529c4d7e None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:13:05.271771 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ce0a16-5850-42e7-ac75-5c20529c4d7e None None] PATH_INFO: `/` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:13:05.273136 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 667/1326] 10.222.0.18 () {54 vars in 965 bytes} [Tue Sep 13 03:13:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 13 03:13:05.279063 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b712f651-2463-49e7-915f-9367c6126ce6 None None] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:13:05.279351 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b712f651-2463-49e7-915f-9367c6126ce6 None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:13:05.279626 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b712f651-2463-49e7-915f-9367c6126ce6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:13:05.311518 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:13:05.357936 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.auth.core [None req-b712f651-2463-49e7-915f-9367c6126ce6 None None] MFA Rules not processed for user `cdffbc851086441ebd01022d1e4c2609`. Rule list: `[]` (Enabled: `True`). {{(pid=81399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 03:13:05.359235 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-b712f651-2463-49e7-915f-9367c6126ce6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:13:05.432603 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 660/1327] 10.222.0.18 () {58 vars in 1062 bytes} [Tue Sep 13 03:13:05 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 03:13:05.441483 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6ce0a16-5850-42e7-ac75-5c20529c4d7e None None] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:13:05.443088 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-a6ce0a16-5850-42e7-ac75-5c20529c4d7e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:13:05.546469 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6ce0a16-5850-42e7-ac75-5c20529c4d7e None None] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:13:05.620802 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8fe5df9-f1cd-4d5d-98d9-c265004c5d11 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdffbc851086441ebd01022d1e4c2609', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b9d94f6a68f7470d9494f112175620dc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:13:05.621398 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8fe5df9-f1cd-4d5d-98d9-c265004c5d11 service neutron] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:13:05.621670 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8fe5df9-f1cd-4d5d-98d9-c265004c5d11 service neutron] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:13:05.621935 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8fe5df9-f1cd-4d5d-98d9-c265004c5d11 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:13:05.623519 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-c8fe5df9-f1cd-4d5d-98d9-c265004c5d11 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:13:05.641640 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:13:05.734975 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8fe5df9-f1cd-4d5d-98d9-c265004c5d11 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:13:05.736399 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8fe5df9-f1cd-4d5d-98d9-c265004c5d11 service neutron] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:13:05.755040 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 668/1328] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Sep 13 03:13:05 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 03:13:08.205776 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-527bd79e-3ff0-4646-934d-999c707db53d None None] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:13:08.206309 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-527bd79e-3ff0-4646-934d-999c707db53d None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:13:08.206309 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-527bd79e-3ff0-4646-934d-999c707db53d None None] PATH_INFO: `/` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:13:08.207728 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 661/1329] 10.222.0.18 () {54 vars in 927 bytes} [Tue Sep 13 03:13:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 03:13:08.213339 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed09bc23-2cfb-4300-bdf8-4fd3531c378b None None] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:13:08.213787 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed09bc23-2cfb-4300-bdf8-4fd3531c378b None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:13:08.213895 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed09bc23-2cfb-4300-bdf8-4fd3531c378b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:13:08.282743 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.auth.core [None req-ed09bc23-2cfb-4300-bdf8-4fd3531c378b None None] MFA Rules not processed for user `06632bd878b1474f8d5432f99e899155`. Rule list: `[]` (Enabled: `True`). {{(pid=81400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 03:13:08.283964 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-ed09bc23-2cfb-4300-bdf8-4fd3531c378b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:13:08.349661 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:13:08.382534 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 669/1330] 10.222.0.18 () {58 vars in 1024 bytes} [Tue Sep 13 03:13:08 2022] POST /identity/v3/auth/tokens => generated 3243 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 03:13:08.674193 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-527bd79e-3ff0-4646-934d-999c707db53d None None] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:13:08.675763 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-527bd79e-3ff0-4646-934d-999c707db53d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:13:08.780681 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-527bd79e-3ff0-4646-934d-999c707db53d None None] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:13:08.801126 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:13:08.855210 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-526e0247-9baa-49ea-9df2-7603d0ad0bf7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdffbc851086441ebd01022d1e4c2609', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b9d94f6a68f7470d9494f112175620dc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:13:08.855815 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-526e0247-9baa-49ea-9df2-7603d0ad0bf7 service neutron] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:13:08.856094 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-526e0247-9baa-49ea-9df2-7603d0ad0bf7 service neutron] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:13:08.856362 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-526e0247-9baa-49ea-9df2-7603d0ad0bf7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:13:08.857936 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-526e0247-9baa-49ea-9df2-7603d0ad0bf7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:13:08.975063 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-526e0247-9baa-49ea-9df2-7603d0ad0bf7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:13:08.977707 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-526e0247-9baa-49ea-9df2-7603d0ad0bf7 service neutron] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:13:08.998322 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 662/1331] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Sep 13 03:13:08 2022] GET /identity/v3/auth/tokens => generated 3243 bytes in 325 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 03:13:11.540572 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d9c9c8f-f58a-4193-b542-88ca7a7bb5fe None None] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:13:11.541075 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d9c9c8f-f58a-4193-b542-88ca7a7bb5fe None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:13:11.541187 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d9c9c8f-f58a-4193-b542-88ca7a7bb5fe None None] PATH_INFO: `/` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:13:11.542496 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 670/1332] 10.222.0.18 () {54 vars in 927 bytes} [Tue Sep 13 03:13:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 03:13:11.547693 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-951f7d7d-af96-4043-a620-4bf230dfae55 None None] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:13:11.548073 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-951f7d7d-af96-4043-a620-4bf230dfae55 None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:13:11.548277 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-951f7d7d-af96-4043-a620-4bf230dfae55 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:13:11.610442 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:13:11.613487 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.auth.core [None req-951f7d7d-af96-4043-a620-4bf230dfae55 None None] MFA Rules not processed for user `49f280ab086b4610855c4ffa378517d9`. Rule list: `[]` (Enabled: `True`). {{(pid=81399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 03:13:11.615227 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-951f7d7d-af96-4043-a620-4bf230dfae55 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:13:11.709699 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 663/1333] 10.222.0.18 () {58 vars in 1024 bytes} [Tue Sep 13 03:13:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 03:13:11.995093 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d9c9c8f-f58a-4193-b542-88ca7a7bb5fe None None] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:13:11.996609 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-7d9c9c8f-f58a-4193-b542-88ca7a7bb5fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:13:12.099192 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d9c9c8f-f58a-4193-b542-88ca7a7bb5fe None None] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:13:12.171649 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6e45644-f3b2-454f-9c1e-69814dd4e811 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdffbc851086441ebd01022d1e4c2609', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b9d94f6a68f7470d9494f112175620dc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:13:12.172250 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6e45644-f3b2-454f-9c1e-69814dd4e811 service neutron] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:13:12.172522 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6e45644-f3b2-454f-9c1e-69814dd4e811 service neutron] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:13:12.172788 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6e45644-f3b2-454f-9c1e-69814dd4e811 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:13:12.174356 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-e6e45644-f3b2-454f-9c1e-69814dd4e811 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:13:12.201072 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:13:12.283197 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6e45644-f3b2-454f-9c1e-69814dd4e811 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:13:12.284557 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6e45644-f3b2-454f-9c1e-69814dd4e811 service neutron] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:13:12.302979 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 671/1334] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Sep 13 03:13:11 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 03:13:14.501724 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a4b1e4f-5f47-44f8-b9a9-ffa401af0c57 None None] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:13:14.502269 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a4b1e4f-5f47-44f8-b9a9-ffa401af0c57 None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:13:14.502269 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a4b1e4f-5f47-44f8-b9a9-ffa401af0c57 None None] PATH_INFO: `/` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:13:14.503729 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 664/1335] 10.222.0.18 () {54 vars in 927 bytes} [Tue Sep 13 03:13:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 03:13:14.508992 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da6b8780-d6dc-440f-9643-3b7ef64c6c04 None None] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:13:14.509783 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da6b8780-d6dc-440f-9643-3b7ef64c6c04 None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:13:14.510789 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da6b8780-d6dc-440f-9643-3b7ef64c6c04 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:13:14.575779 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.auth.core [None req-da6b8780-d6dc-440f-9643-3b7ef64c6c04 None None] MFA Rules not processed for user `49f280ab086b4610855c4ffa378517d9`. Rule list: `[]` (Enabled: `True`). {{(pid=81400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 03:13:14.577008 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-da6b8780-d6dc-440f-9643-3b7ef64c6c04 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:13:14.649559 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:13:14.671784 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 672/1336] 10.222.0.18 () {58 vars in 1024 bytes} [Tue Sep 13 03:13:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 03:13:14.955836 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a4b1e4f-5f47-44f8-b9a9-ffa401af0c57 None None] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:13:14.957386 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-4a4b1e4f-5f47-44f8-b9a9-ffa401af0c57 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:13:15.059055 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a4b1e4f-5f47-44f8-b9a9-ffa401af0c57 None None] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:13:15.088830 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:13:15.132911 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a93bc821-7f4e-419e-8f04-9f8a6a82ebf3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdffbc851086441ebd01022d1e4c2609', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b9d94f6a68f7470d9494f112175620dc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:13:15.133507 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a93bc821-7f4e-419e-8f04-9f8a6a82ebf3 service neutron] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:13:15.133780 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a93bc821-7f4e-419e-8f04-9f8a6a82ebf3 service neutron] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:13:15.134051 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a93bc821-7f4e-419e-8f04-9f8a6a82ebf3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:13:15.135706 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-a93bc821-7f4e-419e-8f04-9f8a6a82ebf3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:13:15.240185 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a93bc821-7f4e-419e-8f04-9f8a6a82ebf3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:13:15.241534 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a93bc821-7f4e-419e-8f04-9f8a6a82ebf3 service neutron] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:13:15.258743 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 665/1337] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Sep 13 03:13:14 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 303 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 03:13:17.737616 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17608711-53e9-4aca-b110-4cddfe7eb51e None None] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:13:17.738141 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17608711-53e9-4aca-b110-4cddfe7eb51e None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:13:17.738141 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17608711-53e9-4aca-b110-4cddfe7eb51e None None] PATH_INFO: `/` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:13:17.739450 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 673/1338] 10.222.0.18 () {54 vars in 927 bytes} [Tue Sep 13 03:13:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 13 03:13:17.745323 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22b94e24-ad16-41e7-8679-68e119a0b405 None None] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:13:17.745744 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22b94e24-ad16-41e7-8679-68e119a0b405 None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:13:17.745856 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22b94e24-ad16-41e7-8679-68e119a0b405 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:13:17.813958 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.auth.core [None req-22b94e24-ad16-41e7-8679-68e119a0b405 None None] MFA Rules not processed for user `49f280ab086b4610855c4ffa378517d9`. Rule list: `[]` (Enabled: `True`). {{(pid=81399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 03:13:17.815410 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-22b94e24-ad16-41e7-8679-68e119a0b405 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:13:17.828670 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:13:17.919037 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 666/1339] 10.222.0.18 () {58 vars in 1024 bytes} [Tue Sep 13 03:13:17 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 03:13:18.209536 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17608711-53e9-4aca-b110-4cddfe7eb51e None None] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:13:18.211253 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-17608711-53e9-4aca-b110-4cddfe7eb51e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:13:18.311544 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17608711-53e9-4aca-b110-4cddfe7eb51e None None] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:13:18.382472 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edb99040-c2bb-4aa7-8f4f-a85171778f34 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdffbc851086441ebd01022d1e4c2609', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b9d94f6a68f7470d9494f112175620dc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:13:18.383019 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edb99040-c2bb-4aa7-8f4f-a85171778f34 service neutron] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:13:18.383291 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edb99040-c2bb-4aa7-8f4f-a85171778f34 service neutron] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:13:18.383558 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edb99040-c2bb-4aa7-8f4f-a85171778f34 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:13:18.385126 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-edb99040-c2bb-4aa7-8f4f-a85171778f34 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:13:18.419850 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:13:18.493330 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edb99040-c2bb-4aa7-8f4f-a85171778f34 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:13:18.494815 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edb99040-c2bb-4aa7-8f4f-a85171778f34 service neutron] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:13:18.513728 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 674/1340] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Sep 13 03:13:18 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 305 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 03:13:22.590920 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b806f239-dae3-46df-908d-09a69b7b7668 None None] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:13:22.591476 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b806f239-dae3-46df-908d-09a69b7b7668 None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:13:22.591476 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b806f239-dae3-46df-908d-09a69b7b7668 None None] PATH_INFO: `/` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:13:22.592782 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 667/1341] 10.222.0.18 () {54 vars in 927 bytes} [Tue Sep 13 03:13:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 03:13:22.598336 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b957c83d-b8e1-44b3-89a9-dc90e0499cdd None None] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:13:22.598792 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b957c83d-b8e1-44b3-89a9-dc90e0499cdd None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:13:22.598895 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b957c83d-b8e1-44b3-89a9-dc90e0499cdd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:13:22.665019 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.auth.core [None req-b957c83d-b8e1-44b3-89a9-dc90e0499cdd None None] MFA Rules not processed for user `49f280ab086b4610855c4ffa378517d9`. Rule list: `[]` (Enabled: `True`). {{(pid=81400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 03:13:22.666134 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-b957c83d-b8e1-44b3-89a9-dc90e0499cdd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:13:22.752962 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:13:22.765097 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 675/1342] 10.222.0.18 () {58 vars in 1024 bytes} [Tue Sep 13 03:13:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 03:13:23.046836 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b806f239-dae3-46df-908d-09a69b7b7668 None None] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:13:23.048412 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-b806f239-dae3-46df-908d-09a69b7b7668 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:13:23.152406 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b806f239-dae3-46df-908d-09a69b7b7668 None None] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:13:23.191389 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:13:23.228234 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55128cf9-f2eb-403f-bcfd-62f4f084bd92 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdffbc851086441ebd01022d1e4c2609', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b9d94f6a68f7470d9494f112175620dc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:13:23.229333 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55128cf9-f2eb-403f-bcfd-62f4f084bd92 service neutron] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:13:23.229604 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55128cf9-f2eb-403f-bcfd-62f4f084bd92 service neutron] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:13:23.229869 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55128cf9-f2eb-403f-bcfd-62f4f084bd92 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:13:23.231488 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-55128cf9-f2eb-403f-bcfd-62f4f084bd92 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:13:23.338238 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55128cf9-f2eb-403f-bcfd-62f4f084bd92 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:13:23.339623 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55128cf9-f2eb-403f-bcfd-62f4f084bd92 service neutron] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:13:23.358987 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 668/1343] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Sep 13 03:13:23 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 03:13:26.180853 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2db5b983-eb79-4f27-b199-d5cfad66fea9 None None] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:13:26.181412 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2db5b983-eb79-4f27-b199-d5cfad66fea9 None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:13:26.181412 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2db5b983-eb79-4f27-b199-d5cfad66fea9 None None] PATH_INFO: `/` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:13:26.182787 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 676/1344] 10.222.0.18 () {54 vars in 927 bytes} [Tue Sep 13 03:13:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 03:13:26.188047 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-894841b3-ba3f-4323-bd50-36964b4ca64a None None] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:13:26.188451 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-894841b3-ba3f-4323-bd50-36964b4ca64a None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:13:26.188602 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-894841b3-ba3f-4323-bd50-36964b4ca64a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:13:26.255106 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.auth.core [None req-894841b3-ba3f-4323-bd50-36964b4ca64a None None] MFA Rules not processed for user `49f280ab086b4610855c4ffa378517d9`. Rule list: `[]` (Enabled: `True`). {{(pid=81399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 03:13:26.256318 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-894841b3-ba3f-4323-bd50-36964b4ca64a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:13:26.276049 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:13:26.356110 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 669/1345] 10.222.0.18 () {58 vars in 1024 bytes} [Tue Sep 13 03:13:26 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 03:13:26.639877 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2db5b983-eb79-4f27-b199-d5cfad66fea9 None None] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:13:26.641384 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-2db5b983-eb79-4f27-b199-d5cfad66fea9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:13:26.747844 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2db5b983-eb79-4f27-b199-d5cfad66fea9 None None] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:13:26.826503 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9b1ac1b-e4c8-49c3-b6c9-818a61c085d9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdffbc851086441ebd01022d1e4c2609', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b9d94f6a68f7470d9494f112175620dc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:13:26.827131 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b1ac1b-e4c8-49c3-b6c9-818a61c085d9 service neutron] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:13:26.827404 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b1ac1b-e4c8-49c3-b6c9-818a61c085d9 service neutron] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:13:26.827669 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b1ac1b-e4c8-49c3-b6c9-818a61c085d9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:13:26.829233 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-e9b1ac1b-e4c8-49c3-b6c9-818a61c085d9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:13:26.876226 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:13:26.943898 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9b1ac1b-e4c8-49c3-b6c9-818a61c085d9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:13:26.945332 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9b1ac1b-e4c8-49c3-b6c9-818a61c085d9 service neutron] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:13:26.964063 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 677/1346] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Sep 13 03:13:26 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 325 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 03:13:31.085834 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c47a069-ee49-493d-9463-c2040ae1e6ca None None] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:13:31.086260 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c47a069-ee49-493d-9463-c2040ae1e6ca None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:13:31.086366 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c47a069-ee49-493d-9463-c2040ae1e6ca None None] PATH_INFO: `/` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:13:31.088296 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 670/1347] 10.222.0.18 () {54 vars in 927 bytes} [Tue Sep 13 03:13:31 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 03:13:31.093697 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e9ca24a-6c03-4197-b3d3-bb84aec3c335 None None] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:13:31.094108 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e9ca24a-6c03-4197-b3d3-bb84aec3c335 None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:13:31.094244 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e9ca24a-6c03-4197-b3d3-bb84aec3c335 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:13:31.162815 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.auth.core [None req-0e9ca24a-6c03-4197-b3d3-bb84aec3c335 None None] MFA Rules not processed for user `49f280ab086b4610855c4ffa378517d9`. Rule list: `[]` (Enabled: `True`). {{(pid=81400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 03:13:31.164012 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-0e9ca24a-6c03-4197-b3d3-bb84aec3c335 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:13:31.263637 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:13:31.266254 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 678/1348] 10.222.0.18 () {58 vars in 1024 bytes} [Tue Sep 13 03:13:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 03:13:31.549730 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c47a069-ee49-493d-9463-c2040ae1e6ca None None] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:13:31.551643 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-8c47a069-ee49-493d-9463-c2040ae1e6ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:13:31.655923 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c47a069-ee49-493d-9463-c2040ae1e6ca None None] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:13:31.706808 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:13:31.735320 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0957b7f5-329c-43ec-ba27-ef8e444a8938 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdffbc851086441ebd01022d1e4c2609', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b9d94f6a68f7470d9494f112175620dc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:13:31.735927 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0957b7f5-329c-43ec-ba27-ef8e444a8938 service neutron] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:13:31.736224 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0957b7f5-329c-43ec-ba27-ef8e444a8938 service neutron] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:13:31.736470 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0957b7f5-329c-43ec-ba27-ef8e444a8938 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:13:31.738116 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-0957b7f5-329c-43ec-ba27-ef8e444a8938 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:13:31.850025 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0957b7f5-329c-43ec-ba27-ef8e444a8938 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:13:31.851434 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0957b7f5-329c-43ec-ba27-ef8e444a8938 service neutron] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:13:31.869717 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 671/1349] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Sep 13 03:13:31 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 320 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 03:13:36.739634 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbcf8a53-8e97-4e19-ac20-728cf6729a4e None None] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:13:36.740124 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbcf8a53-8e97-4e19-ac20-728cf6729a4e None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:13:36.740396 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbcf8a53-8e97-4e19-ac20-728cf6729a4e None None] PATH_INFO: `/` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:13:36.742034 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 679/1350] 10.222.0.18 () {54 vars in 921 bytes} [Tue Sep 13 03:13:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 03:13:36.747257 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6882086-15f0-4c5f-904f-d0360ab4989d None None] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:13:36.747998 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6882086-15f0-4c5f-904f-d0360ab4989d None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:13:36.748557 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6882086-15f0-4c5f-904f-d0360ab4989d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:13:36.860961 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.auth.core [None req-c6882086-15f0-4c5f-904f-d0360ab4989d None None] MFA Rules not processed for user `4b99f1b2374d4d2f8db78306316180d9`. Rule list: `[]` (Enabled: `True`). {{(pid=81399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 03:13:36.862172 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-c6882086-15f0-4c5f-904f-d0360ab4989d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:13:36.880794 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:13:37.032249 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 672/1351] 10.222.0.18 () {58 vars in 1018 bytes} [Tue Sep 13 03:13:36 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 286 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 03:13:41.299163 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f9e19a-63d2-4d40-9504-77fbbdf0adfa None None] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:13:41.299636 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f9e19a-63d2-4d40-9504-77fbbdf0adfa None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:13:41.299743 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f9e19a-63d2-4d40-9504-77fbbdf0adfa None None] PATH_INFO: `/` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:13:41.301440 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 680/1352] 10.222.0.18 () {54 vars in 921 bytes} [Tue Sep 13 03:13:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 03:13:41.306977 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf826d89-dfaf-4e4d-a3d2-fb7f12807825 None None] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:13:41.307404 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf826d89-dfaf-4e4d-a3d2-fb7f12807825 None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:13:41.307523 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf826d89-dfaf-4e4d-a3d2-fb7f12807825 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:13:41.387004 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.auth.core [None req-bf826d89-dfaf-4e4d-a3d2-fb7f12807825 None None] MFA Rules not processed for user `4b99f1b2374d4d2f8db78306316180d9`. Rule list: `[]` (Enabled: `True`). {{(pid=81399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 03:13:41.388121 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-bf826d89-dfaf-4e4d-a3d2-fb7f12807825 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:13:41.470905 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:13:41.511387 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 673/1353] 10.222.0.18 () {58 vars in 1018 bytes} [Tue Sep 13 03:13:41 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 206 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 03:13:42.700475 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4735be5a-fc2d-48b0-8eae-2adb889c7ec4 None None] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:13:42.701040 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4735be5a-fc2d-48b0-8eae-2adb889c7ec4 None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:13:42.701040 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4735be5a-fc2d-48b0-8eae-2adb889c7ec4 None None] PATH_INFO: `/` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:13:42.702405 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 681/1354] 10.222.0.18 () {54 vars in 921 bytes} [Tue Sep 13 03:13:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 03:13:42.707683 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76a53681-7d50-4b96-bf89-2dce40724497 None None] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:13:42.708115 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76a53681-7d50-4b96-bf89-2dce40724497 None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:13:42.708365 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76a53681-7d50-4b96-bf89-2dce40724497 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:13:42.788108 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.auth.core [None req-76a53681-7d50-4b96-bf89-2dce40724497 None None] MFA Rules not processed for user `4b99f1b2374d4d2f8db78306316180d9`. Rule list: `[]` (Enabled: `True`). {{(pid=81399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 03:13:42.789363 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-76a53681-7d50-4b96-bf89-2dce40724497 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:13:42.896031 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 674/1355] 10.222.0.18 () {58 vars in 1018 bytes} [Tue Sep 13 03:13:42 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 13 03:13:46.477213 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75578700-2ca0-4d5d-8726-d93865aff2e5 None None] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:13:46.477834 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75578700-2ca0-4d5d-8726-d93865aff2e5 None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:13:46.477834 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75578700-2ca0-4d5d-8726-d93865aff2e5 None None] PATH_INFO: `/` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:13:46.479203 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 682/1356] 10.222.0.18 () {54 vars in 927 bytes} [Tue Sep 13 03:13:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 03:13:46.484509 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcd5fd2b-b716-4f39-b3f9-604b496857a7 None None] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:13:46.485586 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcd5fd2b-b716-4f39-b3f9-604b496857a7 None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:13:46.485864 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcd5fd2b-b716-4f39-b3f9-604b496857a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:13:46.528734 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:13:46.567249 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.auth.core [None req-dcd5fd2b-b716-4f39-b3f9-604b496857a7 None None] MFA Rules not processed for user `49f280ab086b4610855c4ffa378517d9`. Rule list: `[]` (Enabled: `True`). {{(pid=81399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 03:13:46.568431 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-dcd5fd2b-b716-4f39-b3f9-604b496857a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:13:46.670540 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 675/1357] 10.222.0.18 () {58 vars in 1024 bytes} [Tue Sep 13 03:13:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 03:13:46.974584 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-079c4107-e079-4418-83bb-1963f47ed624 None None] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:13:46.975273 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-079c4107-e079-4418-83bb-1963f47ed624 None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:13:46.976789 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-079c4107-e079-4418-83bb-1963f47ed624 None None] PATH_INFO: `/` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:13:46.979105 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 683/1358] 10.222.0.18 () {54 vars in 962 bytes} [Tue Sep 13 03:13:46 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 03:13:46.987015 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e51e9710-8a17-4b91-92ff-37ace82c7b16 None None] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:13:46.987528 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e51e9710-8a17-4b91-92ff-37ace82c7b16 None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:13:46.988166 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e51e9710-8a17-4b91-92ff-37ace82c7b16 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:13:47.100520 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.auth.core [None req-e51e9710-8a17-4b91-92ff-37ace82c7b16 None None] MFA Rules not processed for user `c7377f5bee4a49959182e1bd34bc97a4`. Rule list: `[]` (Enabled: `True`). {{(pid=81399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 03:13:47.101694 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-e51e9710-8a17-4b91-92ff-37ace82c7b16 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:13:47.153352 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:13:47.224997 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 676/1359] 10.222.0.18 () {58 vars in 1059 bytes} [Tue Sep 13 03:13:46 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 239 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 13 03:13:47.233011 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-079c4107-e079-4418-83bb-1963f47ed624 None None] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:13:47.234678 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-079c4107-e079-4418-83bb-1963f47ed624 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:13:47.390341 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-079c4107-e079-4418-83bb-1963f47ed624 None None] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:13:47.466567 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:13:47.514910 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9526f89-e547-4c5c-bef7-343c0e9845b4 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7377f5bee4a49959182e1bd34bc97a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b9d94f6a68f7470d9494f112175620dc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:13:47.515632 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9526f89-e547-4c5c-bef7-343c0e9845b4 service nova] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:13:47.516004 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9526f89-e547-4c5c-bef7-343c0e9845b4 service nova] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:13:47.518804 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9526f89-e547-4c5c-bef7-343c0e9845b4 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:13:47.518804 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-b9526f89-e547-4c5c-bef7-343c0e9845b4 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:13:47.620561 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9526f89-e547-4c5c-bef7-343c0e9845b4 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:13:47.622373 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9526f89-e547-4c5c-bef7-343c0e9845b4 service nova] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:13:47.643260 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 684/1360] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Sep 13 03:13:47 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 411 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 03:13:50.022281 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bce6ddc-d27a-4a1c-9ff1-84297ded7314 None None] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:13:50.022950 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bce6ddc-d27a-4a1c-9ff1-84297ded7314 None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:13:50.022950 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bce6ddc-d27a-4a1c-9ff1-84297ded7314 None None] PATH_INFO: `/` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:13:50.024184 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 677/1361] 10.222.0.18 () {54 vars in 927 bytes} [Tue Sep 13 03:13:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 03:13:50.029860 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a7db8ac-5087-4e67-a6f9-86ae761f82b7 None None] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:13:50.029860 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a7db8ac-5087-4e67-a6f9-86ae761f82b7 None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:13:50.030720 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a7db8ac-5087-4e67-a6f9-86ae761f82b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:13:50.100302 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.auth.core [None req-4a7db8ac-5087-4e67-a6f9-86ae761f82b7 None None] MFA Rules not processed for user `49f280ab086b4610855c4ffa378517d9`. Rule list: `[]` (Enabled: `True`). {{(pid=81400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 03:13:50.101583 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-4a7db8ac-5087-4e67-a6f9-86ae761f82b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:13:50.143121 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:13:50.208199 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 685/1362] 10.222.0.18 () {58 vars in 1024 bytes} [Tue Sep 13 03:13:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 03:13:50.500984 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bce6ddc-d27a-4a1c-9ff1-84297ded7314 None None] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:13:50.503080 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-1bce6ddc-d27a-4a1c-9ff1-84297ded7314 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:13:50.684633 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bce6ddc-d27a-4a1c-9ff1-84297ded7314 None None] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:13:50.695343 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:13:50.809564 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0f0aab4-9fb3-4151-8581-28e898bdd1bd service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7377f5bee4a49959182e1bd34bc97a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b9d94f6a68f7470d9494f112175620dc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:13:50.810170 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f0aab4-9fb3-4151-8581-28e898bdd1bd service nova] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:13:50.810440 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f0aab4-9fb3-4151-8581-28e898bdd1bd service nova] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:13:50.810739 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f0aab4-9fb3-4151-8581-28e898bdd1bd service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:13:50.812330 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-a0f0aab4-9fb3-4151-8581-28e898bdd1bd service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:13:50.904798 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0f0aab4-9fb3-4151-8581-28e898bdd1bd service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:13:50.906426 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0f0aab4-9fb3-4151-8581-28e898bdd1bd service nova] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:13:50.924478 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 678/1363] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Sep 13 03:13:50 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 424 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 03:13:53.408687 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-084fa69f-ef86-4a28-94f6-8087561e0d34 None None] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:13:53.409275 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-084fa69f-ef86-4a28-94f6-8087561e0d34 None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:13:53.409275 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-084fa69f-ef86-4a28-94f6-8087561e0d34 None None] PATH_INFO: `/` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:13:53.410480 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 686/1364] 10.222.0.18 () {54 vars in 927 bytes} [Tue Sep 13 03:13:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 03:13:53.416814 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc6727ac-c71d-48cb-b55b-a0bd11fe387a None None] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:13:53.417250 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc6727ac-c71d-48cb-b55b-a0bd11fe387a None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:13:53.417346 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc6727ac-c71d-48cb-b55b-a0bd11fe387a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:13:53.432982 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:13:53.487127 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.auth.core [None req-cc6727ac-c71d-48cb-b55b-a0bd11fe387a None None] MFA Rules not processed for user `49f280ab086b4610855c4ffa378517d9`. Rule list: `[]` (Enabled: `True`). {{(pid=81399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 03:13:53.488337 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-cc6727ac-c71d-48cb-b55b-a0bd11fe387a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:13:53.601873 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 679/1365] 10.222.0.18 () {58 vars in 1024 bytes} [Tue Sep 13 03:13:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 03:13:53.891244 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0948964-b57d-486f-b267-3bcb2de27a68 None None] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:13:53.891517 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0948964-b57d-486f-b267-3bcb2de27a68 None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:13:53.892053 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0948964-b57d-486f-b267-3bcb2de27a68 None None] PATH_INFO: `/` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:13:53.893398 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 687/1366] 10.222.0.18 () {54 vars in 962 bytes} [Tue Sep 13 03:13:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 03:13:53.899387 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-950c0c5a-6b91-4861-ae64-53c9abc15c88 None None] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:13:53.900457 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-950c0c5a-6b91-4861-ae64-53c9abc15c88 None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:13:53.900882 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-950c0c5a-6b91-4861-ae64-53c9abc15c88 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:13:53.980075 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.auth.core [None req-950c0c5a-6b91-4861-ae64-53c9abc15c88 None None] MFA Rules not processed for user `c7377f5bee4a49959182e1bd34bc97a4`. Rule list: `[]` (Enabled: `True`). {{(pid=81399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 03:13:53.981548 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-950c0c5a-6b91-4861-ae64-53c9abc15c88 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:13:54.017644 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:13:54.112818 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 680/1367] 10.222.0.18 () {58 vars in 1059 bytes} [Tue Sep 13 03:13:53 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 214 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 03:13:54.126067 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0948964-b57d-486f-b267-3bcb2de27a68 None None] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:13:54.127888 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-f0948964-b57d-486f-b267-3bcb2de27a68 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:13:54.282007 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0948964-b57d-486f-b267-3bcb2de27a68 None None] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:13:54.304948 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:13:54.407089 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ef9b36c-1859-4026-8b48-68fa0e866aa1 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7377f5bee4a49959182e1bd34bc97a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b9d94f6a68f7470d9494f112175620dc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:13:54.407816 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef9b36c-1859-4026-8b48-68fa0e866aa1 service nova] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:13:54.408094 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef9b36c-1859-4026-8b48-68fa0e866aa1 service nova] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:13:54.408468 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef9b36c-1859-4026-8b48-68fa0e866aa1 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:13:54.410043 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-3ef9b36c-1859-4026-8b48-68fa0e866aa1 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:13:54.507511 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ef9b36c-1859-4026-8b48-68fa0e866aa1 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:13:54.510439 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ef9b36c-1859-4026-8b48-68fa0e866aa1 service nova] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:13:54.532812 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 688/1368] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Sep 13 03:13:54 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 407 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 03:13:54.643627 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0854356-77c4-4ba1-ab2c-284731549228 None None] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:13:54.644202 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0854356-77c4-4ba1-ab2c-284731549228 None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:13:54.644202 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0854356-77c4-4ba1-ab2c-284731549228 None None] PATH_INFO: `/` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:13:54.645799 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 681/1369] 10.222.0.18 () {54 vars in 927 bytes} [Tue Sep 13 03:13:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 03:13:54.652675 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54394866-d839-4d80-948c-dda95b6eb576 None None] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:13:54.653492 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54394866-d839-4d80-948c-dda95b6eb576 None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:13:54.654004 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54394866-d839-4d80-948c-dda95b6eb576 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:13:54.693926 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:13:54.740789 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.auth.core [None req-54394866-d839-4d80-948c-dda95b6eb576 None None] MFA Rules not processed for user `49f280ab086b4610855c4ffa378517d9`. Rule list: `[]` (Enabled: `True`). {{(pid=81400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 03:13:54.742470 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-54394866-d839-4d80-948c-dda95b6eb576 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:13:54.862948 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 689/1370] 10.222.0.18 () {58 vars in 1024 bytes} [Tue Sep 13 03:13:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 212 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 03:13:55.195103 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da250703-c467-458e-843c-a6b402ff607d None None] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:13:55.195103 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da250703-c467-458e-843c-a6b402ff607d None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:13:55.195103 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da250703-c467-458e-843c-a6b402ff607d None None] PATH_INFO: `/` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:13:55.196624 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 682/1371] 10.222.0.18 () {54 vars in 964 bytes} [Tue Sep 13 03:13:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 03:13:55.201447 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89781bda-e49d-4938-9a72-129ec8dc4c9c None None] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:13:55.201729 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89781bda-e49d-4938-9a72-129ec8dc4c9c None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:13:55.202001 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89781bda-e49d-4938-9a72-129ec8dc4c9c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:13:55.279509 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.auth.core [None req-89781bda-e49d-4938-9a72-129ec8dc4c9c None None] MFA Rules not processed for user `924ee3f608d34a18b0ddc4695f102a56`. Rule list: `[]` (Enabled: `True`). {{(pid=81400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 03:13:55.280699 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-89781bda-e49d-4938-9a72-129ec8dc4c9c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:13:55.336580 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:13:55.369564 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 690/1372] 10.222.0.18 () {58 vars in 1061 bytes} [Tue Sep 13 03:13:55 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 03:13:55.377108 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da250703-c467-458e-843c-a6b402ff607d None None] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:13:55.378656 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-da250703-c467-458e-843c-a6b402ff607d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:13:55.487680 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da250703-c467-458e-843c-a6b402ff607d None None] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:13:55.539786 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:13:55.577822 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8f61579-b22d-4da7-956d-d369b4fcff10 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '924ee3f608d34a18b0ddc4695f102a56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b9d94f6a68f7470d9494f112175620dc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:13:55.578471 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f61579-b22d-4da7-956d-d369b4fcff10 service cinder] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:13:55.578908 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f61579-b22d-4da7-956d-d369b4fcff10 service cinder] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:13:55.579030 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f61579-b22d-4da7-956d-d369b4fcff10 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:13:55.580623 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-c8f61579-b22d-4da7-956d-d369b4fcff10 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:13:55.722467 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8f61579-b22d-4da7-956d-d369b4fcff10 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:13:55.724515 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8f61579-b22d-4da7-956d-d369b4fcff10 service cinder] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:13:55.748148 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 683/1373] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Sep 13 03:13:55 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 371 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 03:13:56.826452 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f7f29f-e9a1-4a3e-bbd6-b98b3e8fa85c None None] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:13:56.827087 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f7f29f-e9a1-4a3e-bbd6-b98b3e8fa85c None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:13:56.827087 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f7f29f-e9a1-4a3e-bbd6-b98b3e8fa85c None None] PATH_INFO: `/` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:13:56.828483 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 691/1374] 10.222.0.18 () {54 vars in 927 bytes} [Tue Sep 13 03:13:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 03:13:56.833751 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16fbf396-dba3-4ba4-9ad4-a9bad42b5f39 None None] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:13:56.834162 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16fbf396-dba3-4ba4-9ad4-a9bad42b5f39 None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:13:56.834416 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16fbf396-dba3-4ba4-9ad4-a9bad42b5f39 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:13:56.900138 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.auth.core [None req-16fbf396-dba3-4ba4-9ad4-a9bad42b5f39 None None] MFA Rules not processed for user `49f280ab086b4610855c4ffa378517d9`. Rule list: `[]` (Enabled: `True`). {{(pid=81399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 03:13:56.901338 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-16fbf396-dba3-4ba4-9ad4-a9bad42b5f39 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:13:56.922442 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:13:56.999680 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 684/1375] 10.222.0.18 () {58 vars in 1024 bytes} [Tue Sep 13 03:13:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 03:13:57.301728 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0f7f29f-e9a1-4a3e-bbd6-b98b3e8fa85c None None] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:13:57.303441 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-d0f7f29f-e9a1-4a3e-bbd6-b98b3e8fa85c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:13:57.460281 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0f7f29f-e9a1-4a3e-bbd6-b98b3e8fa85c None None] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:13:57.515108 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:13:57.579939 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9db4e306-d125-403a-beea-8d1c944b4c79 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7377f5bee4a49959182e1bd34bc97a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b9d94f6a68f7470d9494f112175620dc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:13:57.580543 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db4e306-d125-403a-beea-8d1c944b4c79 service nova] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:13:57.580827 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db4e306-d125-403a-beea-8d1c944b4c79 service nova] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:13:57.581100 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db4e306-d125-403a-beea-8d1c944b4c79 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:13:57.582716 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-9db4e306-d125-403a-beea-8d1c944b4c79 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:13:57.677517 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9db4e306-d125-403a-beea-8d1c944b4c79 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:13:57.679150 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9db4e306-d125-403a-beea-8d1c944b4c79 service nova] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:13:57.698472 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 692/1376] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Sep 13 03:13:57 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 398 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 03:13:59.905930 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-986caa7b-fb0d-4cf9-b149-b84c5f562f23 None None] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:13:59.906460 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-986caa7b-fb0d-4cf9-b149-b84c5f562f23 None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:13:59.906460 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-986caa7b-fb0d-4cf9-b149-b84c5f562f23 None None] PATH_INFO: `/` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:13:59.908005 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 685/1377] 10.222.0.18 () {54 vars in 927 bytes} [Tue Sep 13 03:13:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 13 03:13:59.912939 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ed5c556-513d-462e-93e3-f4a1c9f2fa53 None None] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:13:59.913328 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ed5c556-513d-462e-93e3-f4a1c9f2fa53 None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:13:59.913513 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ed5c556-513d-462e-93e3-f4a1c9f2fa53 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:13:59.980762 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.auth.core [None req-3ed5c556-513d-462e-93e3-f4a1c9f2fa53 None None] MFA Rules not processed for user `49f280ab086b4610855c4ffa378517d9`. Rule list: `[]` (Enabled: `True`). {{(pid=81400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 03:13:59.981939 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-3ed5c556-513d-462e-93e3-f4a1c9f2fa53 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:13:59.989085 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:14:00.098347 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 693/1378] 10.222.0.18 () {58 vars in 1024 bytes} [Tue Sep 13 03:13:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 03:14:00.385525 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-986caa7b-fb0d-4cf9-b149-b84c5f562f23 None None] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:14:00.387224 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-986caa7b-fb0d-4cf9-b149-b84c5f562f23 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:14:00.536854 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-986caa7b-fb0d-4cf9-b149-b84c5f562f23 None None] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:14:00.540773 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:14:00.653313 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95118a55-68f0-4e27-aadb-ed59e0c93abd service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7377f5bee4a49959182e1bd34bc97a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b9d94f6a68f7470d9494f112175620dc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:14:00.653832 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95118a55-68f0-4e27-aadb-ed59e0c93abd service nova] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:14:00.654107 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95118a55-68f0-4e27-aadb-ed59e0c93abd service nova] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:14:00.654402 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95118a55-68f0-4e27-aadb-ed59e0c93abd service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:14:00.656056 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-95118a55-68f0-4e27-aadb-ed59e0c93abd service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:14:00.747297 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95118a55-68f0-4e27-aadb-ed59e0c93abd service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:14:00.748875 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95118a55-68f0-4e27-aadb-ed59e0c93abd service nova] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:14:00.771309 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 686/1379] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Sep 13 03:14:00 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 386 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 03:14:02.968612 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e68950-56cb-4aad-9020-3f9760dec1f3 None None] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:14:02.969176 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e68950-56cb-4aad-9020-3f9760dec1f3 None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:14:02.969176 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e68950-56cb-4aad-9020-3f9760dec1f3 None None] PATH_INFO: `/` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:14:02.970658 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 694/1380] 10.222.0.18 () {54 vars in 927 bytes} [Tue Sep 13 03:14:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 03:14:02.975749 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e1e35d-ca4a-4843-819b-cddfa57e37cc None None] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:14:02.976140 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e1e35d-ca4a-4843-819b-cddfa57e37cc None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:14:02.976347 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e1e35d-ca4a-4843-819b-cddfa57e37cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:14:02.987677 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:14:03.046730 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.auth.core [None req-e3e1e35d-ca4a-4843-819b-cddfa57e37cc None None] MFA Rules not processed for user `49f280ab086b4610855c4ffa378517d9`. Rule list: `[]` (Enabled: `True`). {{(pid=81399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 03:14:03.047942 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-e3e1e35d-ca4a-4843-819b-cddfa57e37cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:14:03.183238 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 687/1381] 10.222.0.18 () {58 vars in 1024 bytes} [Tue Sep 13 03:14:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 208 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 03:14:03.478758 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6e68950-56cb-4aad-9020-3f9760dec1f3 None None] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:14:03.480200 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-a6e68950-56cb-4aad-9020-3f9760dec1f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:14:03.619509 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:14:03.630237 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6e68950-56cb-4aad-9020-3f9760dec1f3 None None] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:14:03.746179 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a33f87bb-bc02-4342-82dd-4d4bfa4e5c5c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7377f5bee4a49959182e1bd34bc97a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b9d94f6a68f7470d9494f112175620dc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:14:03.746781 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a33f87bb-bc02-4342-82dd-4d4bfa4e5c5c service nova] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:14:03.747071 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a33f87bb-bc02-4342-82dd-4d4bfa4e5c5c service nova] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:14:03.747298 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a33f87bb-bc02-4342-82dd-4d4bfa4e5c5c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:14:03.748892 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-a33f87bb-bc02-4342-82dd-4d4bfa4e5c5c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:14:03.844558 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a33f87bb-bc02-4342-82dd-4d4bfa4e5c5c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:14:03.846179 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a33f87bb-bc02-4342-82dd-4d4bfa4e5c5c service nova] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:14:03.858187 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:14:03.869050 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 695/1382] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Sep 13 03:14:03 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 391 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 03:14:06.057027 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-418517e5-c909-4cf5-b207-c9a03a23dc67 None None] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:14:06.057968 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-418517e5-c909-4cf5-b207-c9a03a23dc67 None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:14:06.058300 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-418517e5-c909-4cf5-b207-c9a03a23dc67 None None] PATH_INFO: `/` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:14:06.059874 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 688/1383] 10.222.0.18 () {54 vars in 927 bytes} [Tue Sep 13 03:14:06 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 03:14:06.065232 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93799144-ea02-474e-87b6-dd9e44555d32 None None] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:14:06.066284 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93799144-ea02-474e-87b6-dd9e44555d32 None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:14:06.066284 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93799144-ea02-474e-87b6-dd9e44555d32 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:14:06.134142 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.auth.core [None req-93799144-ea02-474e-87b6-dd9e44555d32 None None] MFA Rules not processed for user `49f280ab086b4610855c4ffa378517d9`. Rule list: `[]` (Enabled: `True`). {{(pid=81400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 03:14:06.135340 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-93799144-ea02-474e-87b6-dd9e44555d32 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:14:06.233696 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 696/1384] 10.222.0.18 () {58 vars in 1024 bytes} [Tue Sep 13 03:14:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 03:14:06.524311 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-418517e5-c909-4cf5-b207-c9a03a23dc67 None None] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:14:06.526455 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-418517e5-c909-4cf5-b207-c9a03a23dc67 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:14:06.623397 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:14:06.702167 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-418517e5-c909-4cf5-b207-c9a03a23dc67 None None] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:14:06.823653 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be93fa6c-93ac-4dd9-8c81-c613de217cbd service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7377f5bee4a49959182e1bd34bc97a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b9d94f6a68f7470d9494f112175620dc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:14:06.824194 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be93fa6c-93ac-4dd9-8c81-c613de217cbd service nova] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:14:06.824474 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be93fa6c-93ac-4dd9-8c81-c613de217cbd service nova] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:14:06.824749 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be93fa6c-93ac-4dd9-8c81-c613de217cbd service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:14:06.826348 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-be93fa6c-93ac-4dd9-8c81-c613de217cbd service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:14:06.888400 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:14:06.936105 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be93fa6c-93ac-4dd9-8c81-c613de217cbd service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:14:06.937713 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be93fa6c-93ac-4dd9-8c81-c613de217cbd service nova] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:14:06.955458 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 689/1385] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Sep 13 03:14:06 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 432 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 03:14:09.229395 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02be4e7a-dae5-4b82-acbf-a3f25f01a47a None None] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:14:09.229920 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02be4e7a-dae5-4b82-acbf-a3f25f01a47a None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:14:09.229920 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02be4e7a-dae5-4b82-acbf-a3f25f01a47a None None] PATH_INFO: `/` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:14:09.231339 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 697/1386] 10.222.0.18 () {54 vars in 927 bytes} [Tue Sep 13 03:14:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 03:14:09.236685 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a2401c8-d793-40fe-b047-98c332b1b502 None None] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:14:09.237102 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a2401c8-d793-40fe-b047-98c332b1b502 None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:14:09.237234 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a2401c8-d793-40fe-b047-98c332b1b502 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:14:09.303351 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.auth.core [None req-4a2401c8-d793-40fe-b047-98c332b1b502 None None] MFA Rules not processed for user `49f280ab086b4610855c4ffa378517d9`. Rule list: `[]` (Enabled: `True`). {{(pid=81399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 03:14:09.304539 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-4a2401c8-d793-40fe-b047-98c332b1b502 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:14:09.402045 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 690/1387] 10.222.0.18 () {58 vars in 1024 bytes} [Tue Sep 13 03:14:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 03:14:09.693190 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02be4e7a-dae5-4b82-acbf-a3f25f01a47a None None] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:14:09.694734 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-02be4e7a-dae5-4b82-acbf-a3f25f01a47a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:14:09.782431 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:14:09.859471 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02be4e7a-dae5-4b82-acbf-a3f25f01a47a None None] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:14:09.974135 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fe6587e-f710-42ac-ae5a-a18cb7ac543d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7377f5bee4a49959182e1bd34bc97a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b9d94f6a68f7470d9494f112175620dc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:14:09.974682 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fe6587e-f710-42ac-ae5a-a18cb7ac543d service nova] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:14:09.974957 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fe6587e-f710-42ac-ae5a-a18cb7ac543d service nova] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:14:09.975226 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fe6587e-f710-42ac-ae5a-a18cb7ac543d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:14:09.976803 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-6fe6587e-f710-42ac-ae5a-a18cb7ac543d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:14:10.015621 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:14:10.072285 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fe6587e-f710-42ac-ae5a-a18cb7ac543d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:14:10.073882 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fe6587e-f710-42ac-ae5a-a18cb7ac543d service nova] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:14:10.092334 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 698/1388] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Sep 13 03:14:09 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 400 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 03:14:11.165914 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca7d56d-81fa-47c9-8bc5-f129b316704e None None] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:14:11.166450 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca7d56d-81fa-47c9-8bc5-f129b316704e None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:14:11.166547 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca7d56d-81fa-47c9-8bc5-f129b316704e None None] PATH_INFO: `/` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:14:11.168091 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 691/1389] 10.222.0.18 () {54 vars in 927 bytes} [Tue Sep 13 03:14:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 03:14:11.173036 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd55e63-e774-4058-91c1-abe887a03db1 None None] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:14:11.173439 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd55e63-e774-4058-91c1-abe887a03db1 None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:14:11.173564 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd55e63-e774-4058-91c1-abe887a03db1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:14:11.238415 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.auth.core [None req-0fd55e63-e774-4058-91c1-abe887a03db1 None None] MFA Rules not processed for user `49f280ab086b4610855c4ffa378517d9`. Rule list: `[]` (Enabled: `True`). {{(pid=81400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 03:14:11.239616 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-0fd55e63-e774-4058-91c1-abe887a03db1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:14:11.335521 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 699/1390] 10.222.0.18 () {58 vars in 1024 bytes} [Tue Sep 13 03:14:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 03:14:11.633766 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-978e57dd-89d7-438e-adeb-9835435bb0dd None None] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:14:11.634046 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-978e57dd-89d7-438e-adeb-9835435bb0dd None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:14:11.634317 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-978e57dd-89d7-438e-adeb-9835435bb0dd None None] PATH_INFO: `/` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:14:11.635706 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 692/1391] 10.222.0.18 () {54 vars in 966 bytes} [Tue Sep 13 03:14:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 03:14:11.640927 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f63d50af-846a-466a-8a09-5e39c4c80d50 None None] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:14:11.641327 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f63d50af-846a-466a-8a09-5e39c4c80d50 None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:14:11.641662 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f63d50af-846a-466a-8a09-5e39c4c80d50 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:14:11.657786 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:14:11.722021 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.auth.core [None req-f63d50af-846a-466a-8a09-5e39c4c80d50 None None] MFA Rules not processed for user `b5fdc2ff46a14638b409a13e73ba5500`. Rule list: `[]` (Enabled: `True`). {{(pid=81400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 03:14:11.723234 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-f63d50af-846a-466a-8a09-5e39c4c80d50 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:14:11.796632 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 700/1392] 10.222.0.18 () {58 vars in 1063 bytes} [Tue Sep 13 03:14:11 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 03:14:11.805954 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-978e57dd-89d7-438e-adeb-9835435bb0dd None None] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:14:11.807563 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-978e57dd-89d7-438e-adeb-9835435bb0dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:14:11.839642 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:14:11.913358 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-978e57dd-89d7-438e-adeb-9835435bb0dd None None] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:14:11.984935 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf486ba3-e6cf-48cd-813f-409c88985291 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b5fdc2ff46a14638b409a13e73ba5500', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b9d94f6a68f7470d9494f112175620dc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:14:11.985536 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf486ba3-e6cf-48cd-813f-409c88985291 service glance] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:14:11.985811 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf486ba3-e6cf-48cd-813f-409c88985291 service glance] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:14:11.986079 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf486ba3-e6cf-48cd-813f-409c88985291 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:14:11.987699 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-cf486ba3-e6cf-48cd-813f-409c88985291 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:14:12.064418 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:14:12.100368 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf486ba3-e6cf-48cd-813f-409c88985291 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:14:12.101812 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf486ba3-e6cf-48cd-813f-409c88985291 service glance] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:14:12.120130 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 693/1393] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Sep 13 03:14:11 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 315 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 13 03:14:12.275683 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f29329f0-cdfc-4137-8ec0-44cc6a47c5a7 None None] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:14:12.276154 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f29329f0-cdfc-4137-8ec0-44cc6a47c5a7 None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:14:12.276249 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f29329f0-cdfc-4137-8ec0-44cc6a47c5a7 None None] PATH_INFO: `/` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:14:12.277752 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 701/1394] 10.222.0.18 () {54 vars in 927 bytes} [Tue Sep 13 03:14:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 03:14:12.282849 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-536da02f-f029-4f11-9472-06242e3275e2 None None] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:14:12.283271 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-536da02f-f029-4f11-9472-06242e3275e2 None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:14:12.283405 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-536da02f-f029-4f11-9472-06242e3275e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:14:12.356666 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.auth.core [None req-536da02f-f029-4f11-9472-06242e3275e2 None None] MFA Rules not processed for user `49f280ab086b4610855c4ffa378517d9`. Rule list: `[]` (Enabled: `True`). {{(pid=81399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 03:14:12.357898 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-536da02f-f029-4f11-9472-06242e3275e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:14:12.465306 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 694/1395] 10.222.0.18 () {58 vars in 1024 bytes} [Tue Sep 13 03:14:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 13 03:14:12.761618 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f29329f0-cdfc-4137-8ec0-44cc6a47c5a7 None None] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:14:12.763215 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-f29329f0-cdfc-4137-8ec0-44cc6a47c5a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:14:12.875428 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:14:12.882917 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6bb345e-9f10-45a6-bb07-2e2eee35c7f3 None None] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:14:12.883338 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6bb345e-9f10-45a6-bb07-2e2eee35c7f3 None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:14:12.884137 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6bb345e-9f10-45a6-bb07-2e2eee35c7f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:14:12.925176 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f29329f0-cdfc-4137-8ec0-44cc6a47c5a7 None None] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:14:12.931792 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:14:12.954424 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.auth.core [None req-d6bb345e-9f10-45a6-bb07-2e2eee35c7f3 None None] MFA Rules not processed for user `4c2d20fb54a54ecc99b47f237be4917c`. Rule list: `[]` (Enabled: `True`). {{(pid=81399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 03:14:12.955662 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-d6bb345e-9f10-45a6-bb07-2e2eee35c7f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:14:13.039427 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b02226f0-fe21-4a0f-a523-bb85db7cba54 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7377f5bee4a49959182e1bd34bc97a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b9d94f6a68f7470d9494f112175620dc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:14:13.040080 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b02226f0-fe21-4a0f-a523-bb85db7cba54 service nova] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:14:13.040350 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b02226f0-fe21-4a0f-a523-bb85db7cba54 service nova] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:14:13.040616 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b02226f0-fe21-4a0f-a523-bb85db7cba54 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:14:13.042573 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-b02226f0-fe21-4a0f-a523-bb85db7cba54 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:14:13.045132 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 695/1396] 10.222.0.18 () {58 vars in 1009 bytes} [Tue Sep 13 03:14:12 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 13 03:14:13.092122 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50df8940-ad05-4ad5-b984-45299476bc37 None None] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:14:13.092404 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50df8940-ad05-4ad5-b984-45299476bc37 None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:14:13.092693 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50df8940-ad05-4ad5-b984-45299476bc37 None None] PATH_INFO: `/` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:14:13.094069 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 696/1397] 10.222.0.18 () {54 vars in 943 bytes} [Tue Sep 13 03:14:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 13 03:14:13.100193 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f6a124-a9dc-45ab-a9c2-1cfb3b93b592 None None] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:14:13.100533 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f6a124-a9dc-45ab-a9c2-1cfb3b93b592 None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:14:13.100813 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f6a124-a9dc-45ab-a9c2-1cfb3b93b592 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:14:13.121100 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:14:13.142460 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b02226f0-fe21-4a0f-a523-bb85db7cba54 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:14:13.144279 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b02226f0-fe21-4a0f-a523-bb85db7cba54 service nova] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:14:13.162197 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 702/1398] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Sep 13 03:14:12 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 401 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 03:14:13.178840 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.auth.core [None req-97f6a124-a9dc-45ab-a9c2-1cfb3b93b592 None None] MFA Rules not processed for user `893b0c0d85964b7dbc9705c0a091682c`. Rule list: `[]` (Enabled: `True`). {{(pid=81399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 03:14:13.180096 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-97f6a124-a9dc-45ab-a9c2-1cfb3b93b592 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:14:13.244419 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:14:13.296623 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 697/1399] 10.222.0.18 () {58 vars in 1040 bytes} [Tue Sep 13 03:14:13 2022] POST /identity/v3/auth/tokens => generated 3304 bytes in 197 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 13 03:14:13.308050 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b02226f0-fe21-4a0f-a523-bb85db7cba54 service nova] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:14:13.309305 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-b02226f0-fe21-4a0f-a523-bb85db7cba54 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:14:13.400561 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b02226f0-fe21-4a0f-a523-bb85db7cba54 service nova] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:14:13.505416 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:14:13.507003 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5362eb4-0207-4314-82c8-679cfd4493a9 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '893b0c0d85964b7dbc9705c0a091682c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b9d94f6a68f7470d9494f112175620dc', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:14:13.507566 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5362eb4-0207-4314-82c8-679cfd4493a9 service swift] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:14:13.507845 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5362eb4-0207-4314-82c8-679cfd4493a9 service swift] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:14:13.508494 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5362eb4-0207-4314-82c8-679cfd4493a9 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:14:13.510732 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-a5362eb4-0207-4314-82c8-679cfd4493a9 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:14:13.603814 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5362eb4-0207-4314-82c8-679cfd4493a9 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:14:13.605813 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5362eb4-0207-4314-82c8-679cfd4493a9 service swift] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:14:13.614633 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 703/1400] 10.222.0.18 () {60 vars in 1384 bytes} [Tue Sep 13 03:14:13 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 307 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 13 03:14:15.366709 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f9c2971-ada6-4783-8e91-b484ea3c11d7 None None] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:14:15.367217 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f9c2971-ada6-4783-8e91-b484ea3c11d7 None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:14:15.367217 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f9c2971-ada6-4783-8e91-b484ea3c11d7 None None] PATH_INFO: `/` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:14:15.369061 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 698/1401] 10.222.0.18 () {54 vars in 927 bytes} [Tue Sep 13 03:14:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 13 03:14:15.374494 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ff30e9-8cdd-49ee-a1ce-2b6af7210ba0 None None] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:14:15.374936 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ff30e9-8cdd-49ee-a1ce-2b6af7210ba0 None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:14:15.375119 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ff30e9-8cdd-49ee-a1ce-2b6af7210ba0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:14:15.455419 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.auth.core [None req-72ff30e9-8cdd-49ee-a1ce-2b6af7210ba0 None None] MFA Rules not processed for user `49f280ab086b4610855c4ffa378517d9`. Rule list: `[]` (Enabled: `True`). {{(pid=81400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 03:14:15.457066 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-72ff30e9-8cdd-49ee-a1ce-2b6af7210ba0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:14:15.560780 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:14:15.570168 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 704/1402] 10.222.0.18 () {58 vars in 1024 bytes} [Tue Sep 13 03:14:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 03:14:15.861170 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f9c2971-ada6-4783-8e91-b484ea3c11d7 None None] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:14:15.862772 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-8f9c2971-ada6-4783-8e91-b484ea3c11d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:14:16.034751 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f9c2971-ada6-4783-8e91-b484ea3c11d7 None None] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:14:16.084286 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:14:16.166764 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c53a27e-1791-4463-a88c-a35016ce2bbc service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7377f5bee4a49959182e1bd34bc97a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b9d94f6a68f7470d9494f112175620dc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:14:16.167443 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c53a27e-1791-4463-a88c-a35016ce2bbc service nova] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:14:16.167725 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c53a27e-1791-4463-a88c-a35016ce2bbc service nova] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:14:16.168001 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c53a27e-1791-4463-a88c-a35016ce2bbc service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:14:16.169645 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-4c53a27e-1791-4463-a88c-a35016ce2bbc service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:14:16.267705 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c53a27e-1791-4463-a88c-a35016ce2bbc service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:14:16.269300 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c53a27e-1791-4463-a88c-a35016ce2bbc service nova] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:14:16.291349 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 699/1403] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Sep 13 03:14:15 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 431 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 03:14:18.548408 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb3da608-2479-40ab-8f31-ad93223d7990 None None] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:14:18.548965 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb3da608-2479-40ab-8f31-ad93223d7990 None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:14:18.548965 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb3da608-2479-40ab-8f31-ad93223d7990 None None] PATH_INFO: `/` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:14:18.550267 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 705/1404] 10.222.0.18 () {54 vars in 927 bytes} [Tue Sep 13 03:14:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 03:14:18.556286 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c85cdc68-08db-4d57-9057-571a5e88a37e None None] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:14:18.556737 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c85cdc68-08db-4d57-9057-571a5e88a37e None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:14:18.556839 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c85cdc68-08db-4d57-9057-571a5e88a37e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:14:18.623269 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:14:18.634356 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.auth.core [None req-c85cdc68-08db-4d57-9057-571a5e88a37e None None] MFA Rules not processed for user `49f280ab086b4610855c4ffa378517d9`. Rule list: `[]` (Enabled: `True`). {{(pid=81399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 03:14:18.635611 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-c85cdc68-08db-4d57-9057-571a5e88a37e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:14:18.738661 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 700/1405] 10.222.0.18 () {58 vars in 1024 bytes} [Tue Sep 13 03:14:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 03:14:19.031166 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb3da608-2479-40ab-8f31-ad93223d7990 None None] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:14:19.032713 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-fb3da608-2479-40ab-8f31-ad93223d7990 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:14:19.208322 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb3da608-2479-40ab-8f31-ad93223d7990 None None] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:14:19.283792 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:14:19.335990 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d9966c6-749d-4ead-9ceb-2cdf863a930c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7377f5bee4a49959182e1bd34bc97a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b9d94f6a68f7470d9494f112175620dc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:14:19.337152 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d9966c6-749d-4ead-9ceb-2cdf863a930c service nova] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:14:19.337424 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d9966c6-749d-4ead-9ceb-2cdf863a930c service nova] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:14:19.337687 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d9966c6-749d-4ead-9ceb-2cdf863a930c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:14:19.339305 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-3d9966c6-749d-4ead-9ceb-2cdf863a930c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:14:19.436609 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d9966c6-749d-4ead-9ceb-2cdf863a930c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:14:19.438747 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d9966c6-749d-4ead-9ceb-2cdf863a930c service nova] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:14:19.458980 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 706/1406] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Sep 13 03:14:19 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 428 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 03:14:21.662901 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2939b52-6bf6-4bea-b16f-754ee229d611 None None] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:14:21.662901 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2939b52-6bf6-4bea-b16f-754ee229d611 None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:14:21.662901 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2939b52-6bf6-4bea-b16f-754ee229d611 None None] PATH_INFO: `/` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:14:21.665865 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 701/1407] 10.222.0.18 () {54 vars in 927 bytes} [Tue Sep 13 03:14:21 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 03:14:21.667432 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95258a9b-a3f8-42a7-a8ac-cae3885dc1bc None None] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:14:21.667878 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95258a9b-a3f8-42a7-a8ac-cae3885dc1bc None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:14:21.667978 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95258a9b-a3f8-42a7-a8ac-cae3885dc1bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:14:21.736224 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.auth.core [None req-95258a9b-a3f8-42a7-a8ac-cae3885dc1bc None None] MFA Rules not processed for user `49f280ab086b4610855c4ffa378517d9`. Rule list: `[]` (Enabled: `True`). {{(pid=81400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 03:14:21.737451 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-95258a9b-a3f8-42a7-a8ac-cae3885dc1bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:14:21.770407 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:14:21.862724 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 707/1408] 10.222.0.18 () {58 vars in 1024 bytes} [Tue Sep 13 03:14:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 13 03:14:22.139465 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2939b52-6bf6-4bea-b16f-754ee229d611 None None] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:14:22.140989 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-f2939b52-6bf6-4bea-b16f-754ee229d611 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:14:22.267504 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:14:22.297748 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2939b52-6bf6-4bea-b16f-754ee229d611 None None] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:14:22.421071 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9101cf8-5674-4db1-a907-adaa1df09c64 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7377f5bee4a49959182e1bd34bc97a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b9d94f6a68f7470d9494f112175620dc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:14:22.422666 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9101cf8-5674-4db1-a907-adaa1df09c64 service nova] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:14:22.422666 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9101cf8-5674-4db1-a907-adaa1df09c64 service nova] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:14:22.422666 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9101cf8-5674-4db1-a907-adaa1df09c64 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:14:22.426284 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-d9101cf8-5674-4db1-a907-adaa1df09c64 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:14:22.525670 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:14:22.530558 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9101cf8-5674-4db1-a907-adaa1df09c64 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:14:22.532226 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9101cf8-5674-4db1-a907-adaa1df09c64 service nova] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:14:22.556041 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 702/1409] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Sep 13 03:14:22 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 417 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 03:14:31.848848 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=30 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:14:32.552930 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=7 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:14:36.311455 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4581e0b-8b7b-4155-ad4a-e58b0f926163 None None] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:14:36.312017 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4581e0b-8b7b-4155-ad4a-e58b0f926163 None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:14:36.312017 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4581e0b-8b7b-4155-ad4a-e58b0f926163 None None] PATH_INFO: `/` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:14:36.313392 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 708/1410] 10.222.0.18 () {54 vars in 918 bytes} [Tue Sep 13 03:14:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 03:14:36.318641 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26561b15-f269-41b2-8d1a-57ed34d6b3c5 None None] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:14:36.319015 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26561b15-f269-41b2-8d1a-57ed34d6b3c5 None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:14:36.319179 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26561b15-f269-41b2-8d1a-57ed34d6b3c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:14:36.511438 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.auth.core [None req-26561b15-f269-41b2-8d1a-57ed34d6b3c5 None None] MFA Rules not processed for user `4b99f1b2374d4d2f8db78306316180d9`. Rule list: `[]` (Enabled: `True`). {{(pid=81399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 03:14:36.512690 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-26561b15-f269-41b2-8d1a-57ed34d6b3c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:14:36.617495 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 703/1411] 10.222.0.18 () {58 vars in 1015 bytes} [Tue Sep 13 03:14:36 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 300 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 03:14:36.631463 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbad1eaf-6031-4cb8-96fe-7606f4cd8323 None None] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:14:36.631746 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbad1eaf-6031-4cb8-96fe-7606f4cd8323 None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:14:36.632019 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbad1eaf-6031-4cb8-96fe-7606f4cd8323 None None] PATH_INFO: `/` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:14:36.633343 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 709/1412] 10.222.0.18 () {54 vars in 918 bytes} [Tue Sep 13 03:14:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 03:14:36.638630 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30f48fe5-9781-4242-b563-c10e896dd673 None None] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:14:36.638911 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30f48fe5-9781-4242-b563-c10e896dd673 None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:14:36.639181 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30f48fe5-9781-4242-b563-c10e896dd673 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:14:36.709642 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:14:36.713148 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.auth.core [None req-30f48fe5-9781-4242-b563-c10e896dd673 None None] MFA Rules not processed for user `c7377f5bee4a49959182e1bd34bc97a4`. Rule list: `[]` (Enabled: `True`). {{(pid=81399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 03:14:36.714824 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-30f48fe5-9781-4242-b563-c10e896dd673 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:14:36.826467 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 704/1413] 10.222.0.18 () {58 vars in 1015 bytes} [Tue Sep 13 03:14:36 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 03:14:39.397005 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f741cc5-65e5-4649-936c-bf1b421259e5 None None] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:14:39.397511 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f741cc5-65e5-4649-936c-bf1b421259e5 None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:14:39.397511 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f741cc5-65e5-4649-936c-bf1b421259e5 None None] PATH_INFO: `/` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:14:39.398861 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 710/1414] 127.0.0.1 () {54 vars in 917 bytes} [Tue Sep 13 03:14:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 13 03:14:39.404370 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0324be11-aa31-4924-b0f5-413cfab77b57 None None] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:14:39.404793 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0324be11-aa31-4924-b0f5-413cfab77b57 None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:14:39.404897 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0324be11-aa31-4924-b0f5-413cfab77b57 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:14:39.469926 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.auth.core [None req-0324be11-aa31-4924-b0f5-413cfab77b57 None None] MFA Rules not processed for user `49f280ab086b4610855c4ffa378517d9`. Rule list: `[]` (Enabled: `True`). {{(pid=81399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 03:14:39.471220 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-0324be11-aa31-4924-b0f5-413cfab77b57 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:14:39.525410 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:14:39.567458 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 705/1415] 127.0.0.1 () {58 vars in 1014 bytes} [Tue Sep 13 03:14:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 13 03:14:39.857416 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f741cc5-65e5-4649-936c-bf1b421259e5 None None] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:14:39.858955 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-8f741cc5-65e5-4649-936c-bf1b421259e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:14:40.003074 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f741cc5-65e5-4649-936c-bf1b421259e5 None None] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:14:40.075691 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:14:40.116963 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fc8c6bc-1f8f-4547-b927-b4fe9636aaa4 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7377f5bee4a49959182e1bd34bc97a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b9d94f6a68f7470d9494f112175620dc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:14:40.117491 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc8c6bc-1f8f-4547-b927-b4fe9636aaa4 service nova] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:14:40.117760 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc8c6bc-1f8f-4547-b927-b4fe9636aaa4 service nova] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:14:40.118020 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc8c6bc-1f8f-4547-b927-b4fe9636aaa4 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:14:40.119616 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-7fc8c6bc-1f8f-4547-b927-b4fe9636aaa4 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:14:40.207366 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fc8c6bc-1f8f-4547-b927-b4fe9636aaa4 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:14:40.208952 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fc8c6bc-1f8f-4547-b927-b4fe9636aaa4 service nova] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:14:40.227076 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 711/1416] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Sep 13 03:14:39 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 370 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 03:14:41.356790 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67ba4afb-06b9-4147-a6bd-418143b12cef None None] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:14:41.357300 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67ba4afb-06b9-4147-a6bd-418143b12cef None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:14:41.357300 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67ba4afb-06b9-4147-a6bd-418143b12cef None None] PATH_INFO: `/` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:14:41.358635 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 706/1417] 127.0.0.1 () {54 vars in 901 bytes} [Tue Sep 13 03:14:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 03:14:41.364562 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b44cfad-e136-4130-903e-b433954bc154 None None] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:14:41.364959 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b44cfad-e136-4130-903e-b433954bc154 None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:14:41.365094 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b44cfad-e136-4130-903e-b433954bc154 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:14:41.439409 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.auth.core [None req-3b44cfad-e136-4130-903e-b433954bc154 None None] MFA Rules not processed for user `49f280ab086b4610855c4ffa378517d9`. Rule list: `[]` (Enabled: `True`). {{(pid=81400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 03:14:41.440582 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-3b44cfad-e136-4130-903e-b433954bc154 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:14:41.466806 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:14:41.535229 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 712/1418] 127.0.0.1 () {58 vars in 998 bytes} [Tue Sep 13 03:14:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 03:14:41.685441 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67ba4afb-06b9-4147-a6bd-418143b12cef None None] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:14:41.687040 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-67ba4afb-06b9-4147-a6bd-418143b12cef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:14:41.831853 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67ba4afb-06b9-4147-a6bd-418143b12cef None None] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:14:41.871023 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:14:41.940895 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-234f6912-1069-4e6c-a499-a6a0c6f91a9c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7377f5bee4a49959182e1bd34bc97a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b9d94f6a68f7470d9494f112175620dc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:14:41.941424 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-234f6912-1069-4e6c-a499-a6a0c6f91a9c service nova] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:14:41.941696 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-234f6912-1069-4e6c-a499-a6a0c6f91a9c service nova] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:14:41.941985 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-234f6912-1069-4e6c-a499-a6a0c6f91a9c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:14:41.943614 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-234f6912-1069-4e6c-a499-a6a0c6f91a9c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:14:42.038035 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-234f6912-1069-4e6c-a499-a6a0c6f91a9c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:14:42.039658 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-234f6912-1069-4e6c-a499-a6a0c6f91a9c service nova] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:14:42.057863 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 707/1419] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Sep 13 03:14:41 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 373 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 03:14:43.179060 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63aa4c63-bbe9-405e-8aff-c7f55d69ddcf None None] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:14:43.179592 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63aa4c63-bbe9-405e-8aff-c7f55d69ddcf None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:14:43.179592 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63aa4c63-bbe9-405e-8aff-c7f55d69ddcf None None] PATH_INFO: `/` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:14:43.180925 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 713/1420] 127.0.0.1 () {54 vars in 901 bytes} [Tue Sep 13 03:14:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 03:14:43.187137 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a011fc3-773c-495b-ae55-b31b011cb128 None None] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:14:43.187565 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a011fc3-773c-495b-ae55-b31b011cb128 None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:14:43.187666 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a011fc3-773c-495b-ae55-b31b011cb128 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:14:43.240547 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:14:43.263933 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.auth.core [None req-3a011fc3-773c-495b-ae55-b31b011cb128 None None] MFA Rules not processed for user `49f280ab086b4610855c4ffa378517d9`. Rule list: `[]` (Enabled: `True`). {{(pid=81399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 03:14:43.265191 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-3a011fc3-773c-495b-ae55-b31b011cb128 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:14:43.358662 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 708/1421] 127.0.0.1 () {58 vars in 998 bytes} [Tue Sep 13 03:14:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 03:14:43.510266 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63aa4c63-bbe9-405e-8aff-c7f55d69ddcf None None] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:14:43.511863 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-63aa4c63-bbe9-405e-8aff-c7f55d69ddcf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:14:43.655222 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63aa4c63-bbe9-405e-8aff-c7f55d69ddcf None None] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:14:43.666852 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:14:43.768043 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d3e8e26-598d-423b-9f53-32e66cb83f2d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7377f5bee4a49959182e1bd34bc97a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b9d94f6a68f7470d9494f112175620dc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:14:43.768634 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d3e8e26-598d-423b-9f53-32e66cb83f2d service nova] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:14:43.768904 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d3e8e26-598d-423b-9f53-32e66cb83f2d service nova] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:14:43.769166 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d3e8e26-598d-423b-9f53-32e66cb83f2d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:14:43.770764 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-5d3e8e26-598d-423b-9f53-32e66cb83f2d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:14:43.856798 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d3e8e26-598d-423b-9f53-32e66cb83f2d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:14:43.858300 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d3e8e26-598d-423b-9f53-32e66cb83f2d service nova] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:14:43.876067 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 714/1422] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Sep 13 03:14:43 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 366 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 03:14:44.991878 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-822feb5d-517c-4c0a-9c37-8e9c138b1ce6 None None] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:14:44.992480 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-822feb5d-517c-4c0a-9c37-8e9c138b1ce6 None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:14:44.992480 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-822feb5d-517c-4c0a-9c37-8e9c138b1ce6 None None] PATH_INFO: `/` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:14:44.993763 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 709/1423] 127.0.0.1 () {54 vars in 901 bytes} [Tue Sep 13 03:14:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 03:14:44.999822 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36178c76-f3b1-4897-bb95-da9780757b3b None None] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:14:45.000240 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36178c76-f3b1-4897-bb95-da9780757b3b None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:14:45.000343 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36178c76-f3b1-4897-bb95-da9780757b3b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:14:45.020026 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:14:45.077314 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.auth.core [None req-36178c76-f3b1-4897-bb95-da9780757b3b None None] MFA Rules not processed for user `49f280ab086b4610855c4ffa378517d9`. Rule list: `[]` (Enabled: `True`). {{(pid=81400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 03:14:45.078494 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-36178c76-f3b1-4897-bb95-da9780757b3b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:14:45.171933 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 715/1424] 127.0.0.1 () {58 vars in 998 bytes} [Tue Sep 13 03:14:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 03:14:45.323343 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-822feb5d-517c-4c0a-9c37-8e9c138b1ce6 None None] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:14:45.324747 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-822feb5d-517c-4c0a-9c37-8e9c138b1ce6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:14:45.431417 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:14:45.465949 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-822feb5d-517c-4c0a-9c37-8e9c138b1ce6 None None] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:14:45.576151 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d18a7b9-1ff3-4cc6-9a75-5b740e17b039 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7377f5bee4a49959182e1bd34bc97a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b9d94f6a68f7470d9494f112175620dc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:14:45.576696 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d18a7b9-1ff3-4cc6-9a75-5b740e17b039 service nova] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:14:45.576970 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d18a7b9-1ff3-4cc6-9a75-5b740e17b039 service nova] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:14:45.577235 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d18a7b9-1ff3-4cc6-9a75-5b740e17b039 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:14:45.578870 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-8d18a7b9-1ff3-4cc6-9a75-5b740e17b039 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:14:45.653141 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:14:45.666647 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d18a7b9-1ff3-4cc6-9a75-5b740e17b039 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:14:45.668234 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d18a7b9-1ff3-4cc6-9a75-5b740e17b039 service nova] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:14:45.686533 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 710/1425] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Sep 13 03:14:45 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 364 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 13 03:14:46.799645 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79118dc1-edbd-4186-9a1a-eec92ae6a1a1 None None] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:14:46.800484 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79118dc1-edbd-4186-9a1a-eec92ae6a1a1 None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:14:46.800484 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79118dc1-edbd-4186-9a1a-eec92ae6a1a1 None None] PATH_INFO: `/` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:14:46.801800 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 716/1426] 127.0.0.1 () {54 vars in 901 bytes} [Tue Sep 13 03:14:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 03:14:46.808776 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ea9950-255f-4e74-8418-977b13b8a880 None None] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:14:46.809231 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ea9950-255f-4e74-8418-977b13b8a880 None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:14:46.809341 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ea9950-255f-4e74-8418-977b13b8a880 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:14:46.885745 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.auth.core [None req-e1ea9950-255f-4e74-8418-977b13b8a880 None None] MFA Rules not processed for user `49f280ab086b4610855c4ffa378517d9`. Rule list: `[]` (Enabled: `True`). {{(pid=81399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 03:14:46.886972 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-e1ea9950-255f-4e74-8418-977b13b8a880 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:14:46.982774 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 711/1427] 127.0.0.1 () {58 vars in 998 bytes} [Tue Sep 13 03:14:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 03:14:47.132095 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79118dc1-edbd-4186-9a1a-eec92ae6a1a1 None None] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:14:47.133666 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-79118dc1-edbd-4186-9a1a-eec92ae6a1a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:14:47.215816 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:14:47.276473 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79118dc1-edbd-4186-9a1a-eec92ae6a1a1 None None] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:14:47.387689 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e100ee3-87d1-459e-a0e3-defd25165fdb service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7377f5bee4a49959182e1bd34bc97a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b9d94f6a68f7470d9494f112175620dc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:14:47.388215 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e100ee3-87d1-459e-a0e3-defd25165fdb service nova] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:14:47.388490 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e100ee3-87d1-459e-a0e3-defd25165fdb service nova] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:14:47.388753 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e100ee3-87d1-459e-a0e3-defd25165fdb service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:14:47.390316 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-4e100ee3-87d1-459e-a0e3-defd25165fdb service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:14:47.438089 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:14:47.479326 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e100ee3-87d1-459e-a0e3-defd25165fdb service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:14:47.480909 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e100ee3-87d1-459e-a0e3-defd25165fdb service nova] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:14:47.498432 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 717/1428] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Sep 13 03:14:47 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 367 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 03:14:48.559034 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-008bb7c2-253f-44f6-a280-948e8e65ad30 None None] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:14:48.559563 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-008bb7c2-253f-44f6-a280-948e8e65ad30 None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:14:48.559563 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-008bb7c2-253f-44f6-a280-948e8e65ad30 None None] PATH_INFO: `/` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:14:48.560851 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 712/1429] 127.0.0.1 () {54 vars in 901 bytes} [Tue Sep 13 03:14:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 03:14:48.567145 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae245bda-b9f5-4a48-b0e1-fe02eb56b852 None None] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:14:48.567577 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae245bda-b9f5-4a48-b0e1-fe02eb56b852 None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:14:48.567689 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae245bda-b9f5-4a48-b0e1-fe02eb56b852 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:14:48.641716 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.auth.core [None req-ae245bda-b9f5-4a48-b0e1-fe02eb56b852 None None] MFA Rules not processed for user `49f280ab086b4610855c4ffa378517d9`. Rule list: `[]` (Enabled: `True`). {{(pid=81400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 03:14:48.642936 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-ae245bda-b9f5-4a48-b0e1-fe02eb56b852 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:14:48.736985 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 718/1430] 127.0.0.1 () {58 vars in 998 bytes} [Tue Sep 13 03:14:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 03:14:48.885607 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-008bb7c2-253f-44f6-a280-948e8e65ad30 None None] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:14:48.887081 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-008bb7c2-253f-44f6-a280-948e8e65ad30 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:14:48.935756 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:14:49.029200 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-008bb7c2-253f-44f6-a280-948e8e65ad30 None None] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:14:49.139011 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52bde7ff-ab95-48c7-b741-4b53e4f0abe3 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7377f5bee4a49959182e1bd34bc97a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b9d94f6a68f7470d9494f112175620dc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:14:49.139545 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52bde7ff-ab95-48c7-b741-4b53e4f0abe3 service nova] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:14:49.139827 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52bde7ff-ab95-48c7-b741-4b53e4f0abe3 service nova] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:14:49.140079 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52bde7ff-ab95-48c7-b741-4b53e4f0abe3 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:14:49.141803 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-52bde7ff-ab95-48c7-b741-4b53e4f0abe3 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:14:49.149109 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:14:49.231332 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52bde7ff-ab95-48c7-b741-4b53e4f0abe3 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:14:49.232912 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52bde7ff-ab95-48c7-b741-4b53e4f0abe3 service nova] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:14:49.250320 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 713/1431] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Sep 13 03:14:48 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 365 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 03:14:50.295367 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80cbb873-0763-4963-8cca-9352c8b94212 None None] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:14:50.295933 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80cbb873-0763-4963-8cca-9352c8b94212 None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:14:50.295933 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80cbb873-0763-4963-8cca-9352c8b94212 None None] PATH_INFO: `/` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:14:50.297252 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 719/1432] 127.0.0.1 () {54 vars in 901 bytes} [Tue Sep 13 03:14:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 03:14:50.303653 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d9d221-c6c3-43cb-bdab-812409754612 None None] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:14:50.304081 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d9d221-c6c3-43cb-bdab-812409754612 None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:14:50.304190 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d9d221-c6c3-43cb-bdab-812409754612 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:14:50.378360 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.auth.core [None req-c9d9d221-c6c3-43cb-bdab-812409754612 None None] MFA Rules not processed for user `49f280ab086b4610855c4ffa378517d9`. Rule list: `[]` (Enabled: `True`). {{(pid=81399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 03:14:50.379574 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-c9d9d221-c6c3-43cb-bdab-812409754612 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:14:50.445279 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:14:50.473575 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 714/1433] 127.0.0.1 () {58 vars in 998 bytes} [Tue Sep 13 03:14:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 03:14:50.624184 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80cbb873-0763-4963-8cca-9352c8b94212 None None] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:14:50.625619 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-80cbb873-0763-4963-8cca-9352c8b94212 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:14:50.643665 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:14:50.770289 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80cbb873-0763-4963-8cca-9352c8b94212 None None] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:14:50.856594 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:14:50.882900 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff5bdba6-7d7d-42f2-abc4-74eefe4aa0a7 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7377f5bee4a49959182e1bd34bc97a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b9d94f6a68f7470d9494f112175620dc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:14:50.883403 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff5bdba6-7d7d-42f2-abc4-74eefe4aa0a7 service nova] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:14:50.883671 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff5bdba6-7d7d-42f2-abc4-74eefe4aa0a7 service nova] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:14:50.883932 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff5bdba6-7d7d-42f2-abc4-74eefe4aa0a7 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:14:50.885494 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-ff5bdba6-7d7d-42f2-abc4-74eefe4aa0a7 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:14:50.970987 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff5bdba6-7d7d-42f2-abc4-74eefe4aa0a7 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:14:50.972554 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff5bdba6-7d7d-42f2-abc4-74eefe4aa0a7 service nova] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:14:50.990256 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 720/1434] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Sep 13 03:14:50 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 367 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 03:14:52.045153 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9394c565-8ae8-4df5-b630-df220d44d123 None None] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:14:52.045618 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9394c565-8ae8-4df5-b630-df220d44d123 None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:14:52.045713 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9394c565-8ae8-4df5-b630-df220d44d123 None None] PATH_INFO: `/` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:14:52.047077 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 715/1435] 127.0.0.1 () {54 vars in 901 bytes} [Tue Sep 13 03:14:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 03:14:52.053547 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e53eb12e-3379-44ad-a6ca-856d862bb571 None None] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:14:52.054013 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e53eb12e-3379-44ad-a6ca-856d862bb571 None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:14:52.054112 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e53eb12e-3379-44ad-a6ca-856d862bb571 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:14:52.129598 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.auth.core [None req-e53eb12e-3379-44ad-a6ca-856d862bb571 None None] MFA Rules not processed for user `49f280ab086b4610855c4ffa378517d9`. Rule list: `[]` (Enabled: `True`). {{(pid=81400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 03:14:52.130827 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-e53eb12e-3379-44ad-a6ca-856d862bb571 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:14:52.171987 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:14:52.225926 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 721/1436] 127.0.0.1 () {58 vars in 998 bytes} [Tue Sep 13 03:14:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 03:14:52.377410 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9394c565-8ae8-4df5-b630-df220d44d123 None None] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:14:52.378889 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-9394c565-8ae8-4df5-b630-df220d44d123 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:14:52.519707 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9394c565-8ae8-4df5-b630-df220d44d123 None None] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:14:52.572952 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:14:52.633982 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0298ed5-a229-4453-beff-fb9ab5765905 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7377f5bee4a49959182e1bd34bc97a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b9d94f6a68f7470d9494f112175620dc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:14:52.634523 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0298ed5-a229-4453-beff-fb9ab5765905 service nova] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:14:52.634826 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0298ed5-a229-4453-beff-fb9ab5765905 service nova] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:14:52.635094 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0298ed5-a229-4453-beff-fb9ab5765905 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:14:52.636694 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-a0298ed5-a229-4453-beff-fb9ab5765905 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:14:52.725988 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0298ed5-a229-4453-beff-fb9ab5765905 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:14:52.728139 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0298ed5-a229-4453-beff-fb9ab5765905 service nova] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:14:52.745535 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 716/1437] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Sep 13 03:14:52 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 369 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 03:14:53.798876 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-696113b4-91b5-466c-a1f3-220af570b1a7 None None] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:14:53.799434 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-696113b4-91b5-466c-a1f3-220af570b1a7 None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:14:53.799434 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-696113b4-91b5-466c-a1f3-220af570b1a7 None None] PATH_INFO: `/` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:14:53.801045 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 722/1438] 127.0.0.1 () {54 vars in 901 bytes} [Tue Sep 13 03:14:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 03:14:53.807364 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7def1e91-91ec-439d-ac11-b57da1cea9d7 None None] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:14:53.807803 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7def1e91-91ec-439d-ac11-b57da1cea9d7 None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:14:53.807917 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7def1e91-91ec-439d-ac11-b57da1cea9d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:14:53.881976 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.auth.core [None req-7def1e91-91ec-439d-ac11-b57da1cea9d7 None None] MFA Rules not processed for user `49f280ab086b4610855c4ffa378517d9`. Rule list: `[]` (Enabled: `True`). {{(pid=81399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 03:14:53.883716 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-7def1e91-91ec-439d-ac11-b57da1cea9d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:14:53.885706 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:14:53.979931 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 717/1439] 127.0.0.1 () {58 vars in 998 bytes} [Tue Sep 13 03:14:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 03:14:54.131385 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-696113b4-91b5-466c-a1f3-220af570b1a7 None None] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:14:54.132885 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-696113b4-91b5-466c-a1f3-220af570b1a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:14:54.274956 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-696113b4-91b5-466c-a1f3-220af570b1a7 None None] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:14:54.302672 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:14:54.387326 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b0aad12-e6d6-4115-b4a6-af1985818f90 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7377f5bee4a49959182e1bd34bc97a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b9d94f6a68f7470d9494f112175620dc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:14:54.387855 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b0aad12-e6d6-4115-b4a6-af1985818f90 service nova] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:14:54.388131 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b0aad12-e6d6-4115-b4a6-af1985818f90 service nova] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:14:54.388401 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b0aad12-e6d6-4115-b4a6-af1985818f90 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:14:54.389985 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-8b0aad12-e6d6-4115-b4a6-af1985818f90 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:14:54.477116 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b0aad12-e6d6-4115-b4a6-af1985818f90 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:14:54.478717 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b0aad12-e6d6-4115-b4a6-af1985818f90 service nova] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:14:54.496076 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 723/1440] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Sep 13 03:14:54 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 365 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 03:14:56.556318 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0458db77-278f-4b3c-8f6f-6ab8874138ce None None] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:14:56.556876 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0458db77-278f-4b3c-8f6f-6ab8874138ce None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:14:56.556876 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0458db77-278f-4b3c-8f6f-6ab8874138ce None None] PATH_INFO: `/` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:14:56.558242 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 718/1441] 127.0.0.1 () {54 vars in 917 bytes} [Tue Sep 13 03:14:56 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 03:14:56.563630 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35adb2fe-908c-453d-a577-5412eab6ea70 None None] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:14:56.564069 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35adb2fe-908c-453d-a577-5412eab6ea70 None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:14:56.564168 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35adb2fe-908c-453d-a577-5412eab6ea70 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:14:56.604871 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:14:56.630088 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.auth.core [None req-35adb2fe-908c-453d-a577-5412eab6ea70 None None] MFA Rules not processed for user `49f280ab086b4610855c4ffa378517d9`. Rule list: `[]` (Enabled: `True`). {{(pid=81400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 03:14:56.631313 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-35adb2fe-908c-453d-a577-5412eab6ea70 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:14:56.725982 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 724/1442] 127.0.0.1 () {58 vars in 1014 bytes} [Tue Sep 13 03:14:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 03:14:57.012756 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0458db77-278f-4b3c-8f6f-6ab8874138ce None None] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:14:57.014340 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-0458db77-278f-4b3c-8f6f-6ab8874138ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:14:57.123551 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0458db77-278f-4b3c-8f6f-6ab8874138ce None None] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:14:57.155063 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:14:57.197025 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b13db87-d7c5-4b5f-a27e-c1f3865b634b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdffbc851086441ebd01022d1e4c2609', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b9d94f6a68f7470d9494f112175620dc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:14:57.197630 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b13db87-d7c5-4b5f-a27e-c1f3865b634b service neutron] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:14:57.197911 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b13db87-d7c5-4b5f-a27e-c1f3865b634b service neutron] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:14:57.198184 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b13db87-d7c5-4b5f-a27e-c1f3865b634b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:14:57.199848 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-2b13db87-d7c5-4b5f-a27e-c1f3865b634b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:14:57.308424 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b13db87-d7c5-4b5f-a27e-c1f3865b634b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:14:57.309760 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b13db87-d7c5-4b5f-a27e-c1f3865b634b service neutron] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:14:57.327741 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 719/1443] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Sep 13 03:14:57 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 315 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 03:14:59.627190 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d35843e9-3b39-4e82-beed-93b97470c72c None None] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:14:59.627736 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d35843e9-3b39-4e82-beed-93b97470c72c None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:14:59.627736 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d35843e9-3b39-4e82-beed-93b97470c72c None None] PATH_INFO: `/` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:14:59.629098 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 725/1444] 127.0.0.1 () {54 vars in 917 bytes} [Tue Sep 13 03:14:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 03:14:59.634431 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc24f84-f0a0-48a3-8ce3-fdea211e9ddc None None] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:14:59.634858 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc24f84-f0a0-48a3-8ce3-fdea211e9ddc None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:14:59.635020 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc24f84-f0a0-48a3-8ce3-fdea211e9ddc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:14:59.700631 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.auth.core [None req-2cc24f84-f0a0-48a3-8ce3-fdea211e9ddc None None] MFA Rules not processed for user `49f280ab086b4610855c4ffa378517d9`. Rule list: `[]` (Enabled: `True`). {{(pid=81399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 03:14:59.701827 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-2cc24f84-f0a0-48a3-8ce3-fdea211e9ddc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:14:59.714898 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:14:59.797829 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 720/1445] 127.0.0.1 () {58 vars in 1014 bytes} [Tue Sep 13 03:14:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 03:15:00.082220 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d35843e9-3b39-4e82-beed-93b97470c72c None None] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:15:00.083812 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-d35843e9-3b39-4e82-beed-93b97470c72c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:15:00.184754 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d35843e9-3b39-4e82-beed-93b97470c72c None None] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:15:00.197809 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:15:00.257281 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b3a8c46-eaf0-485e-8944-b1437d6adf9c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdffbc851086441ebd01022d1e4c2609', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b9d94f6a68f7470d9494f112175620dc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:15:00.258467 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b3a8c46-eaf0-485e-8944-b1437d6adf9c service neutron] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:15:00.258778 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b3a8c46-eaf0-485e-8944-b1437d6adf9c service neutron] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:15:00.259049 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b3a8c46-eaf0-485e-8944-b1437d6adf9c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:15:00.260692 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-0b3a8c46-eaf0-485e-8944-b1437d6adf9c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:15:00.368016 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b3a8c46-eaf0-485e-8944-b1437d6adf9c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:15:00.369446 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b3a8c46-eaf0-485e-8944-b1437d6adf9c service neutron] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:15:00.387471 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 726/1446] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Sep 13 03:15:00 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 306 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 03:15:02.846931 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05690808-2bf3-4be9-88e2-11bc919e239b None None] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:15:02.847508 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05690808-2bf3-4be9-88e2-11bc919e239b None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:15:02.847508 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05690808-2bf3-4be9-88e2-11bc919e239b None None] PATH_INFO: `/` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:15:02.848820 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 721/1447] 127.0.0.1 () {54 vars in 917 bytes} [Tue Sep 13 03:15:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 03:15:02.854263 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4831265-0dec-48c0-bf5f-5bc56f8c0b3f None None] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:15:02.854705 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4831265-0dec-48c0-bf5f-5bc56f8c0b3f None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:15:02.854829 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4831265-0dec-48c0-bf5f-5bc56f8c0b3f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:15:02.904076 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:15:02.921820 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.auth.core [None req-e4831265-0dec-48c0-bf5f-5bc56f8c0b3f None None] MFA Rules not processed for user `49f280ab086b4610855c4ffa378517d9`. Rule list: `[]` (Enabled: `True`). {{(pid=81400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 03:15:02.923070 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-e4831265-0dec-48c0-bf5f-5bc56f8c0b3f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:15:03.017980 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 727/1448] 127.0.0.1 () {58 vars in 1014 bytes} [Tue Sep 13 03:15:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 03:15:03.301706 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05690808-2bf3-4be9-88e2-11bc919e239b None None] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:15:03.303268 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-05690808-2bf3-4be9-88e2-11bc919e239b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:15:03.405328 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05690808-2bf3-4be9-88e2-11bc919e239b None None] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:15:03.444700 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:15:03.478843 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec73853e-d631-4cd3-8e60-9bbebd33b726 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdffbc851086441ebd01022d1e4c2609', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b9d94f6a68f7470d9494f112175620dc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:15:03.479448 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec73853e-d631-4cd3-8e60-9bbebd33b726 service neutron] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:15:03.479731 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec73853e-d631-4cd3-8e60-9bbebd33b726 service neutron] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:15:03.480000 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec73853e-d631-4cd3-8e60-9bbebd33b726 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:15:03.481602 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-ec73853e-d631-4cd3-8e60-9bbebd33b726 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:15:03.590976 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec73853e-d631-4cd3-8e60-9bbebd33b726 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:15:03.592417 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec73853e-d631-4cd3-8e60-9bbebd33b726 service neutron] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:15:03.610061 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 722/1449] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Sep 13 03:15:03 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 03:15:05.878664 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d29b7a8-0713-4ed5-a0e4-3fd1b4af70f8 None None] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:15:05.879191 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d29b7a8-0713-4ed5-a0e4-3fd1b4af70f8 None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:15:05.879191 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d29b7a8-0713-4ed5-a0e4-3fd1b4af70f8 None None] PATH_INFO: `/` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:15:05.880501 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 728/1450] 127.0.0.1 () {54 vars in 917 bytes} [Tue Sep 13 03:15:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 03:15:05.886220 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-670b2318-d01c-4e02-b3b3-89162bcb1988 None None] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:15:05.886658 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-670b2318-d01c-4e02-b3b3-89162bcb1988 None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:15:05.886810 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-670b2318-d01c-4e02-b3b3-89162bcb1988 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:15:05.951946 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.auth.core [None req-670b2318-d01c-4e02-b3b3-89162bcb1988 None None] MFA Rules not processed for user `49f280ab086b4610855c4ffa378517d9`. Rule list: `[]` (Enabled: `True`). {{(pid=81399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 03:15:05.953156 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-670b2318-d01c-4e02-b3b3-89162bcb1988 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:15:05.973954 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:15:06.048309 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 723/1451] 127.0.0.1 () {58 vars in 1014 bytes} [Tue Sep 13 03:15:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 03:15:06.331718 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d29b7a8-0713-4ed5-a0e4-3fd1b4af70f8 None None] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:15:06.333241 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-9d29b7a8-0713-4ed5-a0e4-3fd1b4af70f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:15:06.433690 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d29b7a8-0713-4ed5-a0e4-3fd1b4af70f8 None None] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:15:06.453497 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:15:06.505918 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abf1366b-1aee-41a9-8e29-6f4a3c4a7ec3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdffbc851086441ebd01022d1e4c2609', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b9d94f6a68f7470d9494f112175620dc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:15:06.506501 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abf1366b-1aee-41a9-8e29-6f4a3c4a7ec3 service neutron] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:15:06.506809 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abf1366b-1aee-41a9-8e29-6f4a3c4a7ec3 service neutron] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:15:06.507081 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abf1366b-1aee-41a9-8e29-6f4a3c4a7ec3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:15:06.508679 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-abf1366b-1aee-41a9-8e29-6f4a3c4a7ec3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:15:06.615593 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abf1366b-1aee-41a9-8e29-6f4a3c4a7ec3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:15:06.617574 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abf1366b-1aee-41a9-8e29-6f4a3c4a7ec3 service neutron] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:15:06.635060 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 729/1452] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Sep 13 03:15:06 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 304 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 03:15:08.830364 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-160d1e0a-3d0b-4ce4-b09d-0151c6432415 None None] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:15:08.831001 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-160d1e0a-3d0b-4ce4-b09d-0151c6432415 None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:15:08.831001 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-160d1e0a-3d0b-4ce4-b09d-0151c6432415 None None] PATH_INFO: `/` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:15:08.832327 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 724/1453] 127.0.0.1 () {54 vars in 917 bytes} [Tue Sep 13 03:15:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 03:15:08.837632 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b768678-1508-48b5-8190-e4246e99f41a None None] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:15:08.838034 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b768678-1508-48b5-8190-e4246e99f41a None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:15:08.838180 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b768678-1508-48b5-8190-e4246e99f41a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:15:08.900548 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:15:08.903286 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.auth.core [None req-5b768678-1508-48b5-8190-e4246e99f41a None None] MFA Rules not processed for user `49f280ab086b4610855c4ffa378517d9`. Rule list: `[]` (Enabled: `True`). {{(pid=81400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 03:15:08.905159 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-5b768678-1508-48b5-8190-e4246e99f41a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:15:08.999112 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 730/1454] 127.0.0.1 () {58 vars in 1014 bytes} [Tue Sep 13 03:15:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 13 03:15:09.281099 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-160d1e0a-3d0b-4ce4-b09d-0151c6432415 None None] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:15:09.282695 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-160d1e0a-3d0b-4ce4-b09d-0151c6432415 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:15:09.384690 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-160d1e0a-3d0b-4ce4-b09d-0151c6432415 None None] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:15:09.432844 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:15:09.456696 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-421f1d8e-51ae-4bdc-a824-0aa56d1726a9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdffbc851086441ebd01022d1e4c2609', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b9d94f6a68f7470d9494f112175620dc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:15:09.457287 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-421f1d8e-51ae-4bdc-a824-0aa56d1726a9 service neutron] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:15:09.457560 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-421f1d8e-51ae-4bdc-a824-0aa56d1726a9 service neutron] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:15:09.457827 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-421f1d8e-51ae-4bdc-a824-0aa56d1726a9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:15:09.459418 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-421f1d8e-51ae-4bdc-a824-0aa56d1726a9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:15:09.565008 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-421f1d8e-51ae-4bdc-a824-0aa56d1726a9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:15:09.566443 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-421f1d8e-51ae-4bdc-a824-0aa56d1726a9 service neutron] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:15:09.584480 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 725/1455] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Sep 13 03:15:09 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 304 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 03:15:11.779973 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f8c5cd3-720b-4f7a-8938-ecef959533cb None None] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:15:11.780527 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f8c5cd3-720b-4f7a-8938-ecef959533cb None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:15:11.780527 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f8c5cd3-720b-4f7a-8938-ecef959533cb None None] PATH_INFO: `/` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:15:11.781844 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 731/1456] 127.0.0.1 () {54 vars in 917 bytes} [Tue Sep 13 03:15:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 03:15:11.787100 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-311b8160-584b-4a52-ab01-76331552f5ed None None] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:15:11.787499 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-311b8160-584b-4a52-ab01-76331552f5ed None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:15:11.787630 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-311b8160-584b-4a52-ab01-76331552f5ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:15:11.854003 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.auth.core [None req-311b8160-584b-4a52-ab01-76331552f5ed None None] MFA Rules not processed for user `49f280ab086b4610855c4ffa378517d9`. Rule list: `[]` (Enabled: `True`). {{(pid=81399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 03:15:11.855166 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-311b8160-584b-4a52-ab01-76331552f5ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:15:11.883673 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:15:11.950436 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 726/1457] 127.0.0.1 () {58 vars in 1014 bytes} [Tue Sep 13 03:15:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 03:15:12.237303 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f8c5cd3-720b-4f7a-8938-ecef959533cb None None] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:15:12.238893 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-1f8c5cd3-720b-4f7a-8938-ecef959533cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:15:12.340359 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f8c5cd3-720b-4f7a-8938-ecef959533cb None None] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:15:12.368325 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:15:12.412422 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f42701c-4927-483a-8e6c-3fcd3e4005a4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdffbc851086441ebd01022d1e4c2609', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b9d94f6a68f7470d9494f112175620dc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:15:12.413009 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f42701c-4927-483a-8e6c-3fcd3e4005a4 service neutron] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:15:12.413289 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f42701c-4927-483a-8e6c-3fcd3e4005a4 service neutron] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:15:12.414061 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f42701c-4927-483a-8e6c-3fcd3e4005a4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:15:12.415651 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-3f42701c-4927-483a-8e6c-3fcd3e4005a4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:15:12.523090 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f42701c-4927-483a-8e6c-3fcd3e4005a4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:15:12.524440 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f42701c-4927-483a-8e6c-3fcd3e4005a4 service neutron] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:15:12.542719 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 732/1458] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Sep 13 03:15:12 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 306 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 03:15:14.835892 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-322fd891-d830-4799-88ed-d25c5adfa2d6 None None] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:15:14.836451 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-322fd891-d830-4799-88ed-d25c5adfa2d6 None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:15:14.836451 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-322fd891-d830-4799-88ed-d25c5adfa2d6 None None] PATH_INFO: `/` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:15:14.837672 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 727/1459] 127.0.0.1 () {54 vars in 917 bytes} [Tue Sep 13 03:15:14 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 03:15:14.842892 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9295099b-74df-41b9-971e-f0641b110c22 None None] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:15:14.843311 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9295099b-74df-41b9-971e-f0641b110c22 None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:15:14.843435 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9295099b-74df-41b9-971e-f0641b110c22 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:15:14.914535 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.auth.core [None req-9295099b-74df-41b9-971e-f0641b110c22 None None] MFA Rules not processed for user `49f280ab086b4610855c4ffa378517d9`. Rule list: `[]` (Enabled: `True`). {{(pid=81400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 03:15:14.915752 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-9295099b-74df-41b9-971e-f0641b110c22 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:15:14.923838 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:15:15.011246 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 733/1460] 127.0.0.1 () {58 vars in 1014 bytes} [Tue Sep 13 03:15:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 03:15:15.290810 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-322fd891-d830-4799-88ed-d25c5adfa2d6 None None] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:15:15.292348 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-322fd891-d830-4799-88ed-d25c5adfa2d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:15:15.394010 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-322fd891-d830-4799-88ed-d25c5adfa2d6 None None] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:15:15.449184 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:15:15.466813 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c8e8e4e-135c-4c06-b6f8-82c954cb4be9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdffbc851086441ebd01022d1e4c2609', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b9d94f6a68f7470d9494f112175620dc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:15:15.467412 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c8e8e4e-135c-4c06-b6f8-82c954cb4be9 service neutron] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:15:15.467687 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c8e8e4e-135c-4c06-b6f8-82c954cb4be9 service neutron] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:15:15.467956 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c8e8e4e-135c-4c06-b6f8-82c954cb4be9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:15:15.469530 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-6c8e8e4e-135c-4c06-b6f8-82c954cb4be9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:15:15.577015 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c8e8e4e-135c-4c06-b6f8-82c954cb4be9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:15:15.578474 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c8e8e4e-135c-4c06-b6f8-82c954cb4be9 service neutron] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:15:15.595812 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 728/1461] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Sep 13 03:15:15 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 306 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 03:15:17.785505 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f73475-90a1-4d0b-9473-eaa18a3852bc None None] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:15:17.786060 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f73475-90a1-4d0b-9473-eaa18a3852bc None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:15:17.786060 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f73475-90a1-4d0b-9473-eaa18a3852bc None None] PATH_INFO: `/` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:15:17.787422 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 734/1462] 127.0.0.1 () {54 vars in 917 bytes} [Tue Sep 13 03:15:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 03:15:17.792597 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f8cee4-404d-4967-a80e-5609ea693109 None None] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:15:17.793021 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f8cee4-404d-4967-a80e-5609ea693109 None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:15:17.793126 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f8cee4-404d-4967-a80e-5609ea693109 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:15:17.858787 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.auth.core [None req-33f8cee4-404d-4967-a80e-5609ea693109 None None] MFA Rules not processed for user `49f280ab086b4610855c4ffa378517d9`. Rule list: `[]` (Enabled: `True`). {{(pid=81399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 03:15:17.859889 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-33f8cee4-404d-4967-a80e-5609ea693109 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:15:17.896921 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:15:17.954320 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 729/1463] 127.0.0.1 () {58 vars in 1014 bytes} [Tue Sep 13 03:15:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 03:15:18.240682 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66f73475-90a1-4d0b-9473-eaa18a3852bc None None] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:15:18.242806 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-66f73475-90a1-4d0b-9473-eaa18a3852bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:15:18.346128 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66f73475-90a1-4d0b-9473-eaa18a3852bc None None] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:15:18.383238 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:15:18.418854 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb1e1b3b-0144-4b9c-9cac-9ab756be42d6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdffbc851086441ebd01022d1e4c2609', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b9d94f6a68f7470d9494f112175620dc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:15:18.419758 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb1e1b3b-0144-4b9c-9cac-9ab756be42d6 service neutron] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:15:18.420031 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb1e1b3b-0144-4b9c-9cac-9ab756be42d6 service neutron] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:15:18.420295 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb1e1b3b-0144-4b9c-9cac-9ab756be42d6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:15:18.421915 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-fb1e1b3b-0144-4b9c-9cac-9ab756be42d6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:15:18.535875 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb1e1b3b-0144-4b9c-9cac-9ab756be42d6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:15:18.537322 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb1e1b3b-0144-4b9c-9cac-9ab756be42d6 service neutron] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:15:18.557739 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 735/1464] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Sep 13 03:15:18 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 318 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 03:15:20.724448 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a99729-b5ee-4f05-ac26-bcc3f4d38e4e None None] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:15:20.725030 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a99729-b5ee-4f05-ac26-bcc3f4d38e4e None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:15:20.725030 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a99729-b5ee-4f05-ac26-bcc3f4d38e4e None None] PATH_INFO: `/` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:15:20.726385 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 730/1465] 127.0.0.1 () {54 vars in 917 bytes} [Tue Sep 13 03:15:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 03:15:20.731604 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7121c049-f0e8-4518-aed4-d051cba569c7 None None] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:15:20.732029 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7121c049-f0e8-4518-aed4-d051cba569c7 None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:15:20.732141 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7121c049-f0e8-4518-aed4-d051cba569c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:15:20.797345 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.auth.core [None req-7121c049-f0e8-4518-aed4-d051cba569c7 None None] MFA Rules not processed for user `49f280ab086b4610855c4ffa378517d9`. Rule list: `[]` (Enabled: `True`). {{(pid=81400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 03:15:20.798539 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-7121c049-f0e8-4518-aed4-d051cba569c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:15:20.816146 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:15:20.895351 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 736/1466] 127.0.0.1 () {58 vars in 1014 bytes} [Tue Sep 13 03:15:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 03:15:21.175656 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8a99729-b5ee-4f05-ac26-bcc3f4d38e4e None None] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:15:21.177196 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-b8a99729-b5ee-4f05-ac26-bcc3f4d38e4e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:15:21.277988 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8a99729-b5ee-4f05-ac26-bcc3f4d38e4e None None] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:15:21.343927 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:15:21.350970 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee85e870-b39b-435f-91c2-8ef36ec0d79b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdffbc851086441ebd01022d1e4c2609', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b9d94f6a68f7470d9494f112175620dc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:15:21.351576 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee85e870-b39b-435f-91c2-8ef36ec0d79b service neutron] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:15:21.351854 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee85e870-b39b-435f-91c2-8ef36ec0d79b service neutron] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:15:21.352131 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee85e870-b39b-435f-91c2-8ef36ec0d79b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:15:21.353714 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-ee85e870-b39b-435f-91c2-8ef36ec0d79b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:15:21.464872 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee85e870-b39b-435f-91c2-8ef36ec0d79b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:15:21.466301 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee85e870-b39b-435f-91c2-8ef36ec0d79b service neutron] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:15:21.484398 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 731/1467] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Sep 13 03:15:21 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 03:15:23.623317 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1e8fffa-9333-45b6-85cc-da9131e7cd58 None None] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:15:23.623804 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1e8fffa-9333-45b6-85cc-da9131e7cd58 None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:15:23.623914 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1e8fffa-9333-45b6-85cc-da9131e7cd58 None None] PATH_INFO: `/` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:15:23.625196 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 737/1468] 127.0.0.1 () {54 vars in 917 bytes} [Tue Sep 13 03:15:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 03:15:23.630393 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be9d6e0-f890-41b4-a814-be4015576d77 None None] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:15:23.630889 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be9d6e0-f890-41b4-a814-be4015576d77 None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:15:23.630994 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be9d6e0-f890-41b4-a814-be4015576d77 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:15:23.696643 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.auth.core [None req-1be9d6e0-f890-41b4-a814-be4015576d77 None None] MFA Rules not processed for user `49f280ab086b4610855c4ffa378517d9`. Rule list: `[]` (Enabled: `True`). {{(pid=81399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 03:15:23.697820 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-1be9d6e0-f890-41b4-a814-be4015576d77 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:15:23.745585 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:15:23.793810 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 732/1469] 127.0.0.1 () {58 vars in 1014 bytes} [Tue Sep 13 03:15:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 03:15:24.081798 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1e8fffa-9333-45b6-85cc-da9131e7cd58 None None] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:15:24.083386 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-e1e8fffa-9333-45b6-85cc-da9131e7cd58 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:15:24.185707 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1e8fffa-9333-45b6-85cc-da9131e7cd58 None None] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:15:24.233437 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:15:24.260243 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2029c95b-6bdf-479c-b24c-f092688f9aa0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdffbc851086441ebd01022d1e4c2609', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b9d94f6a68f7470d9494f112175620dc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:15:24.260845 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2029c95b-6bdf-479c-b24c-f092688f9aa0 service neutron] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:15:24.261120 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2029c95b-6bdf-479c-b24c-f092688f9aa0 service neutron] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:15:24.261383 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2029c95b-6bdf-479c-b24c-f092688f9aa0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:15:24.262979 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-2029c95b-6bdf-479c-b24c-f092688f9aa0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:15:24.370772 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2029c95b-6bdf-479c-b24c-f092688f9aa0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:15:24.372108 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2029c95b-6bdf-479c-b24c-f092688f9aa0 service neutron] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:15:24.389937 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 738/1470] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Sep 13 03:15:24 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 03:15:33.792406 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=22 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:15:34.387769 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=66 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:15:44.192159 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c51e34a2-0793-4cf9-bcd3-c9f7029193dd None None] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:15:44.192709 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c51e34a2-0793-4cf9-bcd3-c9f7029193dd None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:15:44.192709 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c51e34a2-0793-4cf9-bcd3-c9f7029193dd None None] PATH_INFO: `/` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:15:44.194072 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 733/1471] 10.222.0.18 () {54 vars in 927 bytes} [Tue Sep 13 03:15:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 03:15:44.199506 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ccc51a-213d-44ef-a100-3610172bcb72 None None] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:15:44.199918 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ccc51a-213d-44ef-a100-3610172bcb72 None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:15:44.200027 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ccc51a-213d-44ef-a100-3610172bcb72 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:15:44.265011 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.auth.core [None req-e7ccc51a-213d-44ef-a100-3610172bcb72 None None] MFA Rules not processed for user `49f280ab086b4610855c4ffa378517d9`. Rule list: `[]` (Enabled: `True`). {{(pid=81400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 03:15:44.266197 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-e7ccc51a-213d-44ef-a100-3610172bcb72 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:15:44.364209 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 739/1472] 10.222.0.18 () {58 vars in 1024 bytes} [Tue Sep 13 03:15:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 03:15:44.661533 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb572871-0514-4127-b739-2318f4fcbb2f None None] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:15:44.661817 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb572871-0514-4127-b739-2318f4fcbb2f None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:15:44.662089 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb572871-0514-4127-b739-2318f4fcbb2f None None] PATH_INFO: `/` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:15:44.664123 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 734/1473] 10.222.0.18 () {54 vars in 966 bytes} [Tue Sep 13 03:15:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 03:15:44.669025 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc004688-2bcc-4323-938d-7b2e59d56a9c None None] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:15:44.669304 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc004688-2bcc-4323-938d-7b2e59d56a9c None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:15:44.669573 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc004688-2bcc-4323-938d-7b2e59d56a9c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:15:44.744269 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.auth.core [None req-fc004688-2bcc-4323-938d-7b2e59d56a9c None None] MFA Rules not processed for user `b5fdc2ff46a14638b409a13e73ba5500`. Rule list: `[]` (Enabled: `True`). {{(pid=81400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 03:15:44.745459 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-fc004688-2bcc-4323-938d-7b2e59d56a9c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:15:44.766116 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:15:44.822510 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 740/1474] 10.222.0.18 () {58 vars in 1063 bytes} [Tue Sep 13 03:15:44 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 03:15:44.830281 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb572871-0514-4127-b739-2318f4fcbb2f None None] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:15:44.831821 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-cb572871-0514-4127-b739-2318f4fcbb2f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:15:44.936229 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb572871-0514-4127-b739-2318f4fcbb2f None None] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:15:45.007386 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1c1e6f2-31e7-47fd-97e8-cc51a2a873e2 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b5fdc2ff46a14638b409a13e73ba5500', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b9d94f6a68f7470d9494f112175620dc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:15:45.007985 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1c1e6f2-31e7-47fd-97e8-cc51a2a873e2 service glance] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:15:45.008259 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1c1e6f2-31e7-47fd-97e8-cc51a2a873e2 service glance] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:15:45.008525 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1c1e6f2-31e7-47fd-97e8-cc51a2a873e2 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:15:45.010100 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-e1c1e6f2-31e7-47fd-97e8-cc51a2a873e2 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:15:45.063505 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:15:45.120741 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1c1e6f2-31e7-47fd-97e8-cc51a2a873e2 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:15:45.122184 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1c1e6f2-31e7-47fd-97e8-cc51a2a873e2 service glance] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:15:45.140775 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 735/1475] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Sep 13 03:15:44 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 13 03:15:47.372017 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb8fc8b-f165-4af2-aed9-22cd882b7608 None None] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:15:47.372617 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb8fc8b-f165-4af2-aed9-22cd882b7608 None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:15:47.372617 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb8fc8b-f165-4af2-aed9-22cd882b7608 None None] PATH_INFO: `/` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:15:47.373927 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 741/1476] 10.222.0.18 () {54 vars in 927 bytes} [Tue Sep 13 03:15:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 03:15:47.379328 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20e40986-11fc-431a-a575-6ff48c9de030 None None] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:15:47.379768 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20e40986-11fc-431a-a575-6ff48c9de030 None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:15:47.379872 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20e40986-11fc-431a-a575-6ff48c9de030 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:15:47.444896 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.auth.core [None req-20e40986-11fc-431a-a575-6ff48c9de030 None None] MFA Rules not processed for user `49f280ab086b4610855c4ffa378517d9`. Rule list: `[]` (Enabled: `True`). {{(pid=81399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 03:15:47.446102 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-20e40986-11fc-431a-a575-6ff48c9de030 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:15:47.541688 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 736/1477] 10.222.0.18 () {58 vars in 1024 bytes} [Tue Sep 13 03:15:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 03:15:47.815316 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeb8fc8b-f165-4af2-aed9-22cd882b7608 None None] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:15:47.816889 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-eeb8fc8b-f165-4af2-aed9-22cd882b7608 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:15:47.941324 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeb8fc8b-f165-4af2-aed9-22cd882b7608 None None] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:15:47.989765 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:15:48.032953 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2086997-60ab-4bf4-8b72-6d2b94b875dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76c5181fc7be45279ac00017bb4640b3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:15:48.033573 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2086997-60ab-4bf4-8b72-6d2b94b875dc admin admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:15:48.033843 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2086997-60ab-4bf4-8b72-6d2b94b875dc admin admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:15:48.034104 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2086997-60ab-4bf4-8b72-6d2b94b875dc admin admin] PATH_INFO: `/v3/projects` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:15:48.034752 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2086997-60ab-4bf4-8b72-6d2b94b875dc admin admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:15:48.036523 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: WARNING py.warnings [None req-c2086997-60ab-4bf4-8b72-6d2b94b875dc admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_projects": "(role:reader and system_scope:all) or (role:reader and domain_id:%(target.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 13 03:15:48.036523 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: warnings.warn(msg) Sep 13 03:15:48.036523 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: Sep 13 03:15:48.037695 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2086997-60ab-4bf4-8b72-6d2b94b875dc admin admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:15:48.057094 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 742/1478] 10.222.0.18 () {56 vars in 1108 bytes} [Tue Sep 13 03:15:47 2022] GET /identity/v3/projects => generated 2553 bytes in 242 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 03:15:50.228326 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0744c02-9428-445a-bc7d-b7eaac9a1cb3 None None] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:15:50.228882 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0744c02-9428-445a-bc7d-b7eaac9a1cb3 None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:15:50.228882 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0744c02-9428-445a-bc7d-b7eaac9a1cb3 None None] PATH_INFO: `/` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:15:50.230207 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 737/1479] 10.222.0.18 () {54 vars in 927 bytes} [Tue Sep 13 03:15:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 03:15:50.235630 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e6c7fe-9005-4801-b4ff-27d32c3156f4 None None] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:15:50.236037 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e6c7fe-9005-4801-b4ff-27d32c3156f4 None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:15:50.236166 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e6c7fe-9005-4801-b4ff-27d32c3156f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:15:50.301753 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.auth.core [None req-f6e6c7fe-9005-4801-b4ff-27d32c3156f4 None None] MFA Rules not processed for user `49f280ab086b4610855c4ffa378517d9`. Rule list: `[]` (Enabled: `True`). {{(pid=81400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 03:15:50.303021 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-f6e6c7fe-9005-4801-b4ff-27d32c3156f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:15:50.397903 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 743/1480] 10.222.0.18 () {58 vars in 1024 bytes} [Tue Sep 13 03:15:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 03:15:50.693364 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0744c02-9428-445a-bc7d-b7eaac9a1cb3 None None] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:15:50.695439 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-c0744c02-9428-445a-bc7d-b7eaac9a1cb3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:15:50.705214 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:15:50.839867 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0744c02-9428-445a-bc7d-b7eaac9a1cb3 None None] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:15:50.918465 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:15:50.950819 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dfdcb5f-ef52-4725-8801-e7ca836afc93 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7377f5bee4a49959182e1bd34bc97a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b9d94f6a68f7470d9494f112175620dc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:15:50.951399 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dfdcb5f-ef52-4725-8801-e7ca836afc93 service nova] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:15:50.951654 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dfdcb5f-ef52-4725-8801-e7ca836afc93 service nova] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:15:50.951919 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dfdcb5f-ef52-4725-8801-e7ca836afc93 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:15:50.953510 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-6dfdcb5f-ef52-4725-8801-e7ca836afc93 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:15:51.041831 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dfdcb5f-ef52-4725-8801-e7ca836afc93 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:15:51.043480 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dfdcb5f-ef52-4725-8801-e7ca836afc93 service nova] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:15:51.062308 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 738/1481] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Sep 13 03:15:50 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 369 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 03:15:53.385720 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f6e6e4-b42f-4478-bed3-9a3e552c7724 None None] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:15:53.386248 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f6e6e4-b42f-4478-bed3-9a3e552c7724 None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:15:53.386248 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f6e6e4-b42f-4478-bed3-9a3e552c7724 None None] PATH_INFO: `/` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:15:53.387550 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 744/1482] 10.222.0.18 () {54 vars in 927 bytes} [Tue Sep 13 03:15:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 03:15:53.392859 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53d737fb-b277-489d-9da9-ed6a535eee89 None None] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:15:53.393299 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53d737fb-b277-489d-9da9-ed6a535eee89 None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:15:53.393404 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53d737fb-b277-489d-9da9-ed6a535eee89 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:15:53.457779 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.auth.core [None req-53d737fb-b277-489d-9da9-ed6a535eee89 None None] MFA Rules not processed for user `49f280ab086b4610855c4ffa378517d9`. Rule list: `[]` (Enabled: `True`). {{(pid=81399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 03:15:53.459013 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-53d737fb-b277-489d-9da9-ed6a535eee89 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:15:53.505595 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:15:53.555798 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 739/1483] 10.222.0.18 () {58 vars in 1024 bytes} [Tue Sep 13 03:15:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 03:15:53.846090 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44f6e6e4-b42f-4478-bed3-9a3e552c7724 None None] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:15:53.847686 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-44f6e6e4-b42f-4478-bed3-9a3e552c7724 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:15:53.870801 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:15:53.949313 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44f6e6e4-b42f-4478-bed3-9a3e552c7724 None None] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:15:54.021304 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e268b615-4ada-4e75-9c85-817313e28807 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdffbc851086441ebd01022d1e4c2609', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b9d94f6a68f7470d9494f112175620dc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:15:54.022486 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e268b615-4ada-4e75-9c85-817313e28807 service neutron] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:15:54.022794 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e268b615-4ada-4e75-9c85-817313e28807 service neutron] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:15:54.023068 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e268b615-4ada-4e75-9c85-817313e28807 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:15:54.024654 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-e268b615-4ada-4e75-9c85-817313e28807 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:15:54.099626 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:15:54.134619 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e268b615-4ada-4e75-9c85-817313e28807 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:15:54.136062 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e268b615-4ada-4e75-9c85-817313e28807 service neutron] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:15:54.153508 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 745/1484] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Sep 13 03:15:53 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 03:15:56.341646 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3d47376-2eb8-4396-89ec-bb0a502f703d None None] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:15:56.342182 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3d47376-2eb8-4396-89ec-bb0a502f703d None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:15:56.342182 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3d47376-2eb8-4396-89ec-bb0a502f703d None None] PATH_INFO: `/` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:15:56.343514 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 740/1485] 10.222.0.18 () {54 vars in 927 bytes} [Tue Sep 13 03:15:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 03:15:56.348632 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d55a05-5a01-4af4-921e-98a9572eecca None None] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:15:56.349081 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d55a05-5a01-4af4-921e-98a9572eecca None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:15:56.349186 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d55a05-5a01-4af4-921e-98a9572eecca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:15:56.414938 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.auth.core [None req-c2d55a05-5a01-4af4-921e-98a9572eecca None None] MFA Rules not processed for user `49f280ab086b4610855c4ffa378517d9`. Rule list: `[]` (Enabled: `True`). {{(pid=81400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 03:15:56.416139 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-c2d55a05-5a01-4af4-921e-98a9572eecca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:15:56.509340 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 746/1486] 10.222.0.18 () {58 vars in 1024 bytes} [Tue Sep 13 03:15:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 03:15:56.800838 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3d47376-2eb8-4396-89ec-bb0a502f703d None None] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:15:56.802355 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-d3d47376-2eb8-4396-89ec-bb0a502f703d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:15:56.907199 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3d47376-2eb8-4396-89ec-bb0a502f703d None None] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:15:56.978162 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:15:56.979768 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22ff1692-65d5-49ea-9b6e-00585f776782 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdffbc851086441ebd01022d1e4c2609', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b9d94f6a68f7470d9494f112175620dc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:15:56.980431 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ff1692-65d5-49ea-9b6e-00585f776782 service neutron] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:15:56.980892 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ff1692-65d5-49ea-9b6e-00585f776782 service neutron] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:15:56.981364 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ff1692-65d5-49ea-9b6e-00585f776782 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:15:56.983190 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-22ff1692-65d5-49ea-9b6e-00585f776782 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:15:57.103623 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22ff1692-65d5-49ea-9b6e-00585f776782 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:15:57.105053 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22ff1692-65d5-49ea-9b6e-00585f776782 service neutron] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:15:57.122961 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 741/1487] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Sep 13 03:15:56 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 323 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 03:15:59.341462 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-295adfb2-c678-4bac-88af-11051d29238c None None] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:15:59.341981 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-295adfb2-c678-4bac-88af-11051d29238c None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:15:59.341981 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-295adfb2-c678-4bac-88af-11051d29238c None None] PATH_INFO: `/` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:15:59.343316 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 747/1488] 10.222.0.18 () {54 vars in 927 bytes} [Tue Sep 13 03:15:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 03:15:59.348396 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ed2225-1da7-47d8-923c-98ec15dcfdf3 None None] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:15:59.348798 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ed2225-1da7-47d8-923c-98ec15dcfdf3 None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:15:59.348935 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ed2225-1da7-47d8-923c-98ec15dcfdf3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:15:59.413435 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.auth.core [None req-b1ed2225-1da7-47d8-923c-98ec15dcfdf3 None None] MFA Rules not processed for user `49f280ab086b4610855c4ffa378517d9`. Rule list: `[]` (Enabled: `True`). {{(pid=81399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 03:15:59.414635 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-b1ed2225-1da7-47d8-923c-98ec15dcfdf3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:15:59.465376 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:15:59.509316 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 742/1489] 10.222.0.18 () {58 vars in 1024 bytes} [Tue Sep 13 03:15:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 03:15:59.796153 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-295adfb2-c678-4bac-88af-11051d29238c None None] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:15:59.797691 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-295adfb2-c678-4bac-88af-11051d29238c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:15:59.835058 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:15:59.901881 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-295adfb2-c678-4bac-88af-11051d29238c None None] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:15:59.973820 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8202cc8-fb56-4508-890d-24cdeb11d799 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdffbc851086441ebd01022d1e4c2609', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b9d94f6a68f7470d9494f112175620dc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:15:59.974480 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8202cc8-fb56-4508-890d-24cdeb11d799 service neutron] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:15:59.974844 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8202cc8-fb56-4508-890d-24cdeb11d799 service neutron] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:15:59.975114 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8202cc8-fb56-4508-890d-24cdeb11d799 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:15:59.976696 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-b8202cc8-fb56-4508-890d-24cdeb11d799 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:16:00.056053 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:16:00.087350 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8202cc8-fb56-4508-890d-24cdeb11d799 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:16:00.088790 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8202cc8-fb56-4508-890d-24cdeb11d799 service neutron] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:16:00.106432 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 748/1490] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Sep 13 03:15:59 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 03:16:02.401579 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f09590-3af6-429a-9b6e-5ab1d376cc26 None None] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:16:02.402107 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f09590-3af6-429a-9b6e-5ab1d376cc26 None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:16:02.402107 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f09590-3af6-429a-9b6e-5ab1d376cc26 None None] PATH_INFO: `/` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:16:02.403395 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 743/1491] 10.222.0.18 () {54 vars in 927 bytes} [Tue Sep 13 03:16:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 03:16:02.408571 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad404533-f3f1-4e5c-984c-b95099d10cf1 None None] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:16:02.408997 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad404533-f3f1-4e5c-984c-b95099d10cf1 None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:16:02.409104 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad404533-f3f1-4e5c-984c-b95099d10cf1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:16:02.473956 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.auth.core [None req-ad404533-f3f1-4e5c-984c-b95099d10cf1 None None] MFA Rules not processed for user `49f280ab086b4610855c4ffa378517d9`. Rule list: `[]` (Enabled: `True`). {{(pid=81400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 03:16:02.475918 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-ad404533-f3f1-4e5c-984c-b95099d10cf1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:16:02.568890 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 749/1492] 10.222.0.18 () {58 vars in 1024 bytes} [Tue Sep 13 03:16:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 13 03:16:02.856955 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05f09590-3af6-429a-9b6e-5ab1d376cc26 None None] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:16:02.858478 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-05f09590-3af6-429a-9b6e-5ab1d376cc26 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:16:02.959455 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05f09590-3af6-429a-9b6e-5ab1d376cc26 None None] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:16:03.031972 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0433d556-cfe4-4bab-b651-5e028b7528b6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdffbc851086441ebd01022d1e4c2609', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b9d94f6a68f7470d9494f112175620dc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:16:03.032593 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0433d556-cfe4-4bab-b651-5e028b7528b6 service neutron] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:16:03.032866 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0433d556-cfe4-4bab-b651-5e028b7528b6 service neutron] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:16:03.033133 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0433d556-cfe4-4bab-b651-5e028b7528b6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:16:03.034727 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-0433d556-cfe4-4bab-b651-5e028b7528b6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:16:03.043803 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:16:03.143257 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0433d556-cfe4-4bab-b651-5e028b7528b6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:16:03.144611 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0433d556-cfe4-4bab-b651-5e028b7528b6 service neutron] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:16:03.162924 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 744/1493] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Sep 13 03:16:02 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 306 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 03:16:12.567356 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=82 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:16:13.160371 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=50 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:17:16.182033 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf26909f-ead0-4a79-8cf1-6dc72a71bff2 None None] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:17:16.182492 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf26909f-ead0-4a79-8cf1-6dc72a71bff2 None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:17:16.182492 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf26909f-ead0-4a79-8cf1-6dc72a71bff2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:17:16.257322 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.auth.core [None req-bf26909f-ead0-4a79-8cf1-6dc72a71bff2 None None] MFA Rules not processed for user `49f280ab086b4610855c4ffa378517d9`. Rule list: `[]` (Enabled: `True`). {{(pid=81400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 03:17:16.258508 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-bf26909f-ead0-4a79-8cf1-6dc72a71bff2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:17:16.353043 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 750/1494] 10.222.0.18 () {58 vars in 958 bytes} [Tue Sep 13 03:17:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 03:17:16.361017 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aa595f4-86f2-4f8f-918a-1132cc689b65 None None] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:17:16.361497 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aa595f4-86f2-4f8f-918a-1132cc689b65 None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:17:16.361603 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aa595f4-86f2-4f8f-918a-1132cc689b65 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:17:16.437134 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.auth.core [None req-9aa595f4-86f2-4f8f-918a-1132cc689b65 None None] MFA Rules not processed for user `49f280ab086b4610855c4ffa378517d9`. Rule list: `[]` (Enabled: `True`). {{(pid=81399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 03:17:16.438325 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-9aa595f4-86f2-4f8f-918a-1132cc689b65 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:17:16.531338 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 745/1495] 10.222.0.18 () {58 vars in 958 bytes} [Tue Sep 13 03:17:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 03:17:16.535958 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf26909f-ead0-4a79-8cf1-6dc72a71bff2 None None] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:17:16.537160 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-bf26909f-ead0-4a79-8cf1-6dc72a71bff2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:17:16.549103 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf26909f-ead0-4a79-8cf1-6dc72a71bff2 None None] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:17:16.608839 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:17:16.642224 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2916b44-ab88-46d5-a7b1-7adf8c4f385e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76c5181fc7be45279ac00017bb4640b3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:17:16.642859 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2916b44-ab88-46d5-a7b1-7adf8c4f385e admin admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:17:16.643138 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2916b44-ab88-46d5-a7b1-7adf8c4f385e admin admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:17:16.643410 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2916b44-ab88-46d5-a7b1-7adf8c4f385e admin admin] PATH_INFO: `/v3/domains` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:17:16.644046 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2916b44-ab88-46d5-a7b1-7adf8c4f385e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 03:17:16.644345 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2916b44-ab88-46d5-a7b1-7adf8c4f385e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:17:16.645349 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: WARNING py.warnings [None req-d2916b44-ab88-46d5-a7b1-7adf8c4f385e admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 13 03:17:16.645349 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: warnings.warn(msg) Sep 13 03:17:16.645349 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: Sep 13 03:17:16.646259 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2916b44-ab88-46d5-a7b1-7adf8c4f385e admin admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:17:16.657750 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 751/1496] 10.222.0.18 () {58 vars in 1153 bytes} [Tue Sep 13 03:17:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 122 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 03:17:16.666444 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aa595f4-86f2-4f8f-918a-1132cc689b65 None None] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:17:16.675861 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aa595f4-86f2-4f8f-918a-1132cc689b65 None None] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:17:16.736789 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:17:16.770837 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86a28c01-88f7-4ffb-84d7-029f80b9cb29 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76c5181fc7be45279ac00017bb4640b3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:17:16.771581 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a28c01-88f7-4ffb-84d7-029f80b9cb29 admin admin] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:17:16.772857 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a28c01-88f7-4ffb-84d7-029f80b9cb29 admin admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:17:16.773133 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a28c01-88f7-4ffb-84d7-029f80b9cb29 admin admin] PATH_INFO: `/v3/projects` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:17:16.773764 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86a28c01-88f7-4ffb-84d7-029f80b9cb29 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:17:16.775507 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: WARNING py.warnings [None req-86a28c01-88f7-4ffb-84d7-029f80b9cb29 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 13 03:17:16.775507 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: warnings.warn(msg) Sep 13 03:17:16.775507 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: Sep 13 03:17:16.776701 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86a28c01-88f7-4ffb-84d7-029f80b9cb29 admin admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:17:16.831168 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 746/1497] 10.222.0.18 () {60 vars in 1153 bytes} [Tue Sep 13 03:17:16 2022] POST /identity/v3/projects => generated 377 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 13 03:17:16.836092 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2916b44-ab88-46d5-a7b1-7adf8c4f385e admin admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:17:16.837647 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-d2916b44-ab88-46d5-a7b1-7adf8c4f385e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:17:16.871371 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2916b44-ab88-46d5-a7b1-7adf8c4f385e admin admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:17:16.967169 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf79961b-63a7-455a-b14a-b652143ee90b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76c5181fc7be45279ac00017bb4640b3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:17:16.967768 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf79961b-63a7-455a-b14a-b652143ee90b admin admin] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:17:16.968036 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf79961b-63a7-455a-b14a-b652143ee90b admin admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:17:16.968300 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf79961b-63a7-455a-b14a-b652143ee90b admin admin] PATH_INFO: `/v3/users` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:17:16.968914 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf79961b-63a7-455a-b14a-b652143ee90b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:17:16.969906 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: WARNING py.warnings [None req-bf79961b-63a7-455a-b14a-b652143ee90b admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 13 03:17:16.969906 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: warnings.warn(msg) Sep 13 03:17:16.969906 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: Sep 13 03:17:16.971136 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf79961b-63a7-455a-b14a-b652143ee90b admin admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:17:17.017487 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 752/1498] 10.222.0.18 () {60 vars in 1144 bytes} [Tue Sep 13 03:17:16 2022] POST /identity/v3/users => generated 289 bytes in 181 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 03:17:17.020996 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86a28c01-88f7-4ffb-84d7-029f80b9cb29 admin admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:17:17.023134 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-86a28c01-88f7-4ffb-84d7-029f80b9cb29 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:17:17.085521 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86a28c01-88f7-4ffb-84d7-029f80b9cb29 admin admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:17:17.168535 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:17:17.179970 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3b71885-eace-4326-9ae0-20643ea3b0d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76c5181fc7be45279ac00017bb4640b3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:17:17.180603 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b71885-eace-4326-9ae0-20643ea3b0d7 admin admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:17:17.180879 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b71885-eace-4326-9ae0-20643ea3b0d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:17:17.181153 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b71885-eace-4326-9ae0-20643ea3b0d7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:17:17.181785 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3b71885-eace-4326-9ae0-20643ea3b0d7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:17:17.182848 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: WARNING py.warnings [None req-c3b71885-eace-4326-9ae0-20643ea3b0d7 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 13 03:17:17.182848 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: warnings.warn(msg) Sep 13 03:17:17.182848 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: Sep 13 03:17:17.183741 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3b71885-eace-4326-9ae0-20643ea3b0d7 admin admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:17:17.196497 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 747/1499] 10.222.0.18 () {58 vars in 1122 bytes} [Tue Sep 13 03:17:17 2022] GET /identity/v3/roles => generated 1420 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 03:17:17.200530 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf79961b-63a7-455a-b14a-b652143ee90b admin admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:17:17.202052 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-bf79961b-63a7-455a-b14a-b652143ee90b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:17:17.214017 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf79961b-63a7-455a-b14a-b652143ee90b admin admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:17:17.238437 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:17:17.307727 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaebb624-9eb6-414e-8ef7-368967c8427b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76c5181fc7be45279ac00017bb4640b3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:17:17.308317 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaebb624-9eb6-414e-8ef7-368967c8427b admin admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:17:17.308589 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaebb624-9eb6-414e-8ef7-368967c8427b admin admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:17:17.308854 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaebb624-9eb6-414e-8ef7-368967c8427b admin admin] PATH_INFO: `/v3/roles` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:17:17.309475 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaebb624-9eb6-414e-8ef7-368967c8427b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:17:17.310471 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: WARNING py.warnings [None req-eaebb624-9eb6-414e-8ef7-368967c8427b admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 13 03:17:17.310471 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: warnings.warn(msg) Sep 13 03:17:17.310471 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: Sep 13 03:17:17.311371 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaebb624-9eb6-414e-8ef7-368967c8427b admin admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:17:17.323799 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 753/1500] 10.222.0.18 () {58 vars in 1122 bytes} [Tue Sep 13 03:17:17 2022] GET /identity/v3/roles => generated 1420 bytes in 124 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 03:17:17.327945 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3b71885-eace-4326-9ae0-20643ea3b0d7 admin admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:17:17.329449 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-c3b71885-eace-4326-9ae0-20643ea3b0d7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:17:17.361995 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3b71885-eace-4326-9ae0-20643ea3b0d7 admin admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:17:17.453956 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8e26801-afff-4258-939a-32b4935a6b3b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76c5181fc7be45279ac00017bb4640b3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:17:17.454606 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8e26801-afff-4258-939a-32b4935a6b3b admin admin] REQUEST_METHOD: `PUT` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:17:17.454921 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8e26801-afff-4258-939a-32b4935a6b3b admin admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:17:17.455195 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8e26801-afff-4258-939a-32b4935a6b3b admin admin] PATH_INFO: `/v3/projects/4e3482b3940944dca077695475bef0b0/users/29f928f2b0c6429f97d7d427547ccdb3/roles/1eb7a5b5203945c29e6d22d653f47847` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:17:17.476865 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8e26801-afff-4258-939a-32b4935a6b3b admin admin] RBAC: Authorizing `identity:create_grant(project_id=4e3482b3940944dca077695475bef0b0, user_id=29f928f2b0c6429f97d7d427547ccdb3, role_id=1eb7a5b5203945c29e6d22d653f47847)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:17:17.477870 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: WARNING py.warnings [None req-b8e26801-afff-4258-939a-32b4935a6b3b admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 13 03:17:17.477870 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: warnings.warn(msg) Sep 13 03:17:17.477870 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: Sep 13 03:17:17.479854 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8e26801-afff-4258-939a-32b4935a6b3b admin admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:17:17.503606 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 748/1501] 10.222.0.18 () {60 vars in 1483 bytes} [Tue Sep 13 03:17:17 2022] PUT /identity/v3/projects/4e3482b3940944dca077695475bef0b0/users/29f928f2b0c6429f97d7d427547ccdb3/roles/1eb7a5b5203945c29e6d22d653f47847 => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 13 03:17:17.516270 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da07739-ef0c-47a7-be9e-821b3f43fc80 None None] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:17:17.516554 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da07739-ef0c-47a7-be9e-821b3f43fc80 None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:17:17.516827 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da07739-ef0c-47a7-be9e-821b3f43fc80 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:17:17.573432 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.auth.core [None req-3da07739-ef0c-47a7-be9e-821b3f43fc80 None None] MFA Rules not processed for user `29f928f2b0c6429f97d7d427547ccdb3`. Rule list: `[]` (Enabled: `True`). {{(pid=81400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 03:17:17.574648 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-3da07739-ef0c-47a7-be9e-821b3f43fc80 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:17:17.651040 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 754/1502] 10.222.0.18 () {58 vars in 958 bytes} [Tue Sep 13 03:17:17 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 03:17:17.664567 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c1689d1-c7b3-42c7-be33-edfee4db6b95 None None] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:17:17.664853 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c1689d1-c7b3-42c7-be33-edfee4db6b95 None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:17:17.665123 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c1689d1-c7b3-42c7-be33-edfee4db6b95 None None] PATH_INFO: `/` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:17:17.666463 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 749/1503] 10.222.0.18 () {54 vars in 964 bytes} [Tue Sep 13 03:17:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 03:17:17.671144 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc06e86-a1b7-4210-88c9-5abd3d8d33f0 None None] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:17:17.671454 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc06e86-a1b7-4210-88c9-5abd3d8d33f0 None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:17:17.671724 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc06e86-a1b7-4210-88c9-5abd3d8d33f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:17:17.703588 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:17:17.749455 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.auth.core [None req-ccc06e86-a1b7-4210-88c9-5abd3d8d33f0 None None] MFA Rules not processed for user `924ee3f608d34a18b0ddc4695f102a56`. Rule list: `[]` (Enabled: `True`). {{(pid=81400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 03:17:17.750642 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-ccc06e86-a1b7-4210-88c9-5abd3d8d33f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:17:17.825338 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 755/1504] 10.222.0.18 () {58 vars in 1061 bytes} [Tue Sep 13 03:17:17 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 03:17:17.833036 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c1689d1-c7b3-42c7-be33-edfee4db6b95 None None] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:17:17.834587 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-7c1689d1-c7b3-42c7-be33-edfee4db6b95 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:17:17.885273 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:17:17.938930 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c1689d1-c7b3-42c7-be33-edfee4db6b95 None None] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:17:18.010501 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7025452-0426-4718-88c9-9d687faa8328 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '924ee3f608d34a18b0ddc4695f102a56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b9d94f6a68f7470d9494f112175620dc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:17:18.011065 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7025452-0426-4718-88c9-9d687faa8328 service cinder] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:17:18.011344 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7025452-0426-4718-88c9-9d687faa8328 service cinder] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:17:18.011605 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7025452-0426-4718-88c9-9d687faa8328 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:17:18.013161 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-c7025452-0426-4718-88c9-9d687faa8328 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:17:18.111605 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7025452-0426-4718-88c9-9d687faa8328 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:17:18.113060 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7025452-0426-4718-88c9-9d687faa8328 service cinder] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:17:18.117805 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:17:18.132017 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 750/1505] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Sep 13 03:17:17 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 299 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 13 03:17:18.178031 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad50118-afe0-480b-9b98-e8712ccc2995 None None] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:17:18.178312 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad50118-afe0-480b-9b98-e8712ccc2995 None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:17:18.178620 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad50118-afe0-480b-9b98-e8712ccc2995 None None] PATH_INFO: `/` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:17:18.179928 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 756/1506] 10.222.0.18 () {52 vars in 830 bytes} [Tue Sep 13 03:17:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 03:17:18.183466 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7025452-0426-4718-88c9-9d687faa8328 service cinder] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:17:18.184665 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-c7025452-0426-4718-88c9-9d687faa8328 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:17:18.282204 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7025452-0426-4718-88c9-9d687faa8328 service cinder] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:17:18.377046 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-165348e3-13dc-4bee-ab29-fde30f64514d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76c5181fc7be45279ac00017bb4640b3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:17:18.377703 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-165348e3-13dc-4bee-ab29-fde30f64514d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:17:18.377928 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-165348e3-13dc-4bee-ab29-fde30f64514d admin admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:17:18.378190 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-165348e3-13dc-4bee-ab29-fde30f64514d admin admin] PATH_INFO: `/v3/users/29f928f2b0c6429f97d7d427547ccdb3` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:17:18.392285 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-165348e3-13dc-4bee-ab29-fde30f64514d admin admin] RBAC: Authorizing `identity:delete_user(user_id=29f928f2b0c6429f97d7d427547ccdb3)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:17:18.393291 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: WARNING py.warnings [None req-165348e3-13dc-4bee-ab29-fde30f64514d admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 13 03:17:18.393291 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: warnings.warn(msg) Sep 13 03:17:18.393291 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: Sep 13 03:17:18.394498 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-165348e3-13dc-4bee-ab29-fde30f64514d admin admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:17:18.405018 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:17:18.461420 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-165348e3-13dc-4bee-ab29-fde30f64514d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '29f928f2b0c6429f97d7d427547ccdb3'} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:17:18.471913 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-165348e3-13dc-4bee-ab29-fde30f64514d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '29f928f2b0c6429f97d7d427547ccdb3'} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:17:18.495313 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-165348e3-13dc-4bee-ab29-fde30f64514d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '29f928f2b0c6429f97d7d427547ccdb3'} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:17:18.495782 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-165348e3-13dc-4bee-ab29-fde30f64514d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '29f928f2b0c6429f97d7d427547ccdb3'} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:17:18.509881 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-165348e3-13dc-4bee-ab29-fde30f64514d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '29f928f2b0c6429f97d7d427547ccdb3'} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:17:18.515403 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 751/1507] 10.222.0.18 () {58 vars in 1224 bytes} [Tue Sep 13 03:17:18 2022] DELETE /identity/v3/users/29f928f2b0c6429f97d7d427547ccdb3 => generated 0 bytes in 332 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 03:17:18.519934 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3066e18d-2186-4694-b607-b6162b55b6ea None None] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:17:18.520407 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3066e18d-2186-4694-b607-b6162b55b6ea None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:17:18.520515 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3066e18d-2186-4694-b607-b6162b55b6ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:17:18.579586 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.auth.core [None req-3066e18d-2186-4694-b607-b6162b55b6ea None None] MFA Rules not processed for user `49f280ab086b4610855c4ffa378517d9`. Rule list: `[]` (Enabled: `True`). {{(pid=81400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 03:17:18.580685 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-3066e18d-2186-4694-b607-b6162b55b6ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:17:18.654695 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:17:18.667072 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 757/1508] 10.222.0.18 () {58 vars in 958 bytes} [Tue Sep 13 03:17:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 03:17:18.677486 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-165348e3-13dc-4bee-ab29-fde30f64514d admin admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:17:18.678751 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-165348e3-13dc-4bee-ab29-fde30f64514d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:17:18.783493 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-165348e3-13dc-4bee-ab29-fde30f64514d admin admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:17:18.838123 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:DELETE=10,keystone:SELECT=89,keystone:INSERT=1 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:17:18.857487 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38067d3b-b580-476d-bd13-97b79cd33916 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdffbc851086441ebd01022d1e4c2609', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b9d94f6a68f7470d9494f112175620dc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:17:18.858090 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38067d3b-b580-476d-bd13-97b79cd33916 service neutron] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:17:18.858363 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38067d3b-b580-476d-bd13-97b79cd33916 service neutron] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:17:18.858647 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38067d3b-b580-476d-bd13-97b79cd33916 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:17:18.860221 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-38067d3b-b580-476d-bd13-97b79cd33916 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:17:18.973542 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38067d3b-b580-476d-bd13-97b79cd33916 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:17:18.975346 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38067d3b-b580-476d-bd13-97b79cd33916 service neutron] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:17:18.995924 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 752/1509] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Sep 13 03:17:18 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 319 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 03:17:19.335525 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3066e18d-2186-4694-b607-b6162b55b6ea None None] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:17:19.336884 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-3066e18d-2186-4694-b607-b6162b55b6ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:17:19.348824 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3066e18d-2186-4694-b607-b6162b55b6ea None None] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:17:19.445876 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e800ff6e-3132-4108-a993-191d1729d06e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76c5181fc7be45279ac00017bb4640b3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:17:19.446498 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e800ff6e-3132-4108-a993-191d1729d06e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:17:19.446807 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e800ff6e-3132-4108-a993-191d1729d06e admin admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:17:19.447077 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e800ff6e-3132-4108-a993-191d1729d06e admin admin] PATH_INFO: `/v3/projects/4e3482b3940944dca077695475bef0b0` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:17:19.456828 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e800ff6e-3132-4108-a993-191d1729d06e admin admin] RBAC: Authorizing `identity:delete_project(project_id=4e3482b3940944dca077695475bef0b0)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:17:19.457854 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: WARNING py.warnings [None req-e800ff6e-3132-4108-a993-191d1729d06e admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 13 03:17:19.457854 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: warnings.warn(msg) Sep 13 03:17:19.457854 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: Sep 13 03:17:19.459107 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e800ff6e-3132-4108-a993-191d1729d06e admin admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:17:19.520514 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-e800ff6e-3132-4108-a993-191d1729d06e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4e3482b3940944dca077695475bef0b0', 'request_id': 'req-e800ff6e-3132-4108-a993-191d1729d06e'} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:17:19.527314 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-e800ff6e-3132-4108-a993-191d1729d06e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4e3482b3940944dca077695475bef0b0', 'request_id': 'req-e800ff6e-3132-4108-a993-191d1729d06e'} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:17:19.542845 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-e800ff6e-3132-4108-a993-191d1729d06e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4e3482b3940944dca077695475bef0b0', 'request_id': 'req-e800ff6e-3132-4108-a993-191d1729d06e'} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:17:19.684265 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-e800ff6e-3132-4108-a993-191d1729d06e admin admin] The token cache is being invalidate because project 4e3482b3940944dca077695475bef0b0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81400) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 03:17:19.684552 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-e800ff6e-3132-4108-a993-191d1729d06e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:17:19.684978 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-e800ff6e-3132-4108-a993-191d1729d06e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:17:19.686368 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 758/1510] 10.222.0.18 () {58 vars in 1233 bytes} [Tue Sep 13 03:17:19 2022] DELETE /identity/v3/projects/4e3482b3940944dca077695475bef0b0 => generated 0 bytes in 351 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 03:17:28.993399 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=62 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:17:29.682039 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=84,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:18:49.539052 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b24fdf-2226-4a42-90ab-5c39f6ad4cbe None None] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:18:49.539527 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b24fdf-2226-4a42-90ab-5c39f6ad4cbe None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:18:49.539633 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b24fdf-2226-4a42-90ab-5c39f6ad4cbe None None] PATH_INFO: `/` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:18:49.540951 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 753/1511] 127.0.0.1 () {54 vars in 901 bytes} [Tue Sep 13 03:18:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 03:18:49.545776 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e99a399-97bb-4d5a-9a63-73022da594bf None None] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:18:49.546225 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e99a399-97bb-4d5a-9a63-73022da594bf None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:18:49.546330 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e99a399-97bb-4d5a-9a63-73022da594bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:18:49.626288 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.auth.core [None req-2e99a399-97bb-4d5a-9a63-73022da594bf None None] MFA Rules not processed for user `49f280ab086b4610855c4ffa378517d9`. Rule list: `[]` (Enabled: `True`). {{(pid=81400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 03:18:49.627515 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-2e99a399-97bb-4d5a-9a63-73022da594bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:18:49.729565 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 759/1512] 127.0.0.1 () {58 vars in 998 bytes} [Tue Sep 13 03:18:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 13 03:18:49.883648 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84b24fdf-2226-4a42-90ab-5c39f6ad4cbe None None] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:18:49.886004 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-84b24fdf-2226-4a42-90ab-5c39f6ad4cbe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:18:50.007794 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84b24fdf-2226-4a42-90ab-5c39f6ad4cbe None None] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:18:50.013524 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39648796-f624-4952-a042-cef36a8e7835 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7377f5bee4a49959182e1bd34bc97a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b9d94f6a68f7470d9494f112175620dc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:18:50.014154 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39648796-f624-4952-a042-cef36a8e7835 service nova] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:18:50.014430 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39648796-f624-4952-a042-cef36a8e7835 service nova] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:18:50.014736 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39648796-f624-4952-a042-cef36a8e7835 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:18:50.022891 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39648796-f624-4952-a042-cef36a8e7835 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:18:50.024493 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39648796-f624-4952-a042-cef36a8e7835 service nova] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:18:50.033809 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 754/1513] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Sep 13 03:18:49 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 151 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 03:18:51.109725 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb6571de-f5fd-49aa-ba74-9ba953c18811 None None] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:18:51.110210 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb6571de-f5fd-49aa-ba74-9ba953c18811 None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:18:51.110308 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb6571de-f5fd-49aa-ba74-9ba953c18811 None None] PATH_INFO: `/` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:18:51.111679 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 760/1514] 127.0.0.1 () {54 vars in 901 bytes} [Tue Sep 13 03:18:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 03:18:51.118511 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a935aec-f461-49af-b8e1-157e7f1ca510 None None] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:18:51.118996 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a935aec-f461-49af-b8e1-157e7f1ca510 None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:18:51.119113 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a935aec-f461-49af-b8e1-157e7f1ca510 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:18:51.139132 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.auth.core [None req-0a935aec-f461-49af-b8e1-157e7f1ca510 None None] MFA Rules not processed for user `49f280ab086b4610855c4ffa378517d9`. Rule list: `[]` (Enabled: `True`). {{(pid=81399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 03:18:51.140335 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-0a935aec-f461-49af-b8e1-157e7f1ca510 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:18:51.148224 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 755/1515] 127.0.0.1 () {58 vars in 998 bytes} [Tue Sep 13 03:18:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 03:18:51.305169 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb6571de-f5fd-49aa-ba74-9ba953c18811 None None] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:18:51.316644 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bc74819-0846-4916-95f4-d8885845ff8f None None] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:18:51.316957 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bc74819-0846-4916-95f4-d8885845ff8f None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:18:51.317230 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bc74819-0846-4916-95f4-d8885845ff8f None None] PATH_INFO: `/v3` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:18:51.318543 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 756/1516] 10.222.0.57 () {54 vars in 925 bytes} [Tue Sep 13 03:18:51 2022] GET /identity/v3 => generated 255 bytes in 3 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 03:18:51.319404 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb6571de-f5fd-49aa-ba74-9ba953c18811 None None] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:18:51.324846 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1f0e249-10b6-4c46-a016-32fc390f8e9e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7377f5bee4a49959182e1bd34bc97a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b9d94f6a68f7470d9494f112175620dc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:18:51.325081 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c38966f5-73fa-463f-bd39-83a5cb586895 None None] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:18:51.325381 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c38966f5-73fa-463f-bd39-83a5cb586895 None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:18:51.325488 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f0e249-10b6-4c46-a016-32fc390f8e9e service nova] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:18:51.325640 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f0e249-10b6-4c46-a016-32fc390f8e9e service nova] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:18:51.325787 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c38966f5-73fa-463f-bd39-83a5cb586895 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:18:51.325886 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f0e249-10b6-4c46-a016-32fc390f8e9e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:18:51.334070 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1f0e249-10b6-4c46-a016-32fc390f8e9e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:18:51.335679 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1f0e249-10b6-4c46-a016-32fc390f8e9e service nova] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:18:51.346089 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 761/1517] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Sep 13 03:18:51 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 03:18:51.389052 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.auth.core [None req-c38966f5-73fa-463f-bd39-83a5cb586895 None None] MFA Rules not processed for user `4b99f1b2374d4d2f8db78306316180d9`. Rule list: `[]` (Enabled: `True`). {{(pid=81399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 03:18:51.390312 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-c38966f5-73fa-463f-bd39-83a5cb586895 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:18:51.450991 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:18:51.470754 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 757/1518] 10.222.0.57 () {58 vars in 1015 bytes} [Tue Sep 13 03:18:51 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 13 03:19:00.160193 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53729027-bf46-4340-9fa4-1ed36b840050 None None] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:19:00.160623 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53729027-bf46-4340-9fa4-1ed36b840050 None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:19:00.160737 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53729027-bf46-4340-9fa4-1ed36b840050 None None] PATH_INFO: `/` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:19:00.162078 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 762/1519] 10.222.0.18 () {54 vars in 961 bytes} [Tue Sep 13 03:19:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 03:19:00.167368 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c51adc8-d945-48a2-adea-c6bdf93aa1db None None] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:19:00.167809 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c51adc8-d945-48a2-adea-c6bdf93aa1db None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:19:00.167980 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c51adc8-d945-48a2-adea-c6bdf93aa1db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:19:00.188802 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.auth.core [None req-1c51adc8-d945-48a2-adea-c6bdf93aa1db None None] MFA Rules not processed for user `4b99f1b2374d4d2f8db78306316180d9`. Rule list: `[]` (Enabled: `True`). {{(pid=81399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 03:19:00.189980 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-1c51adc8-d945-48a2-adea-c6bdf93aa1db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:19:00.199181 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 758/1520] 10.222.0.18 () {58 vars in 1058 bytes} [Tue Sep 13 03:19:00 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 03:19:00.206838 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53729027-bf46-4340-9fa4-1ed36b840050 None None] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:19:00.219830 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53729027-bf46-4340-9fa4-1ed36b840050 None None] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:19:00.224677 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd00ac05-883d-495e-8def-c90d2c1ba3b3 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b99f1b2374d4d2f8db78306316180d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b9d94f6a68f7470d9494f112175620dc', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:19:00.225260 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd00ac05-883d-495e-8def-c90d2c1ba3b3 service placement] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:19:00.225530 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd00ac05-883d-495e-8def-c90d2c1ba3b3 service placement] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:19:00.225793 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd00ac05-883d-495e-8def-c90d2c1ba3b3 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:19:00.234184 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd00ac05-883d-495e-8def-c90d2c1ba3b3 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:19:00.235768 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd00ac05-883d-495e-8def-c90d2c1ba3b3 service placement] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:19:00.245598 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 763/1521] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Sep 13 03:19:00 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 03:19:02.827522 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eac54a4f-cfa7-46a1-8fc1-7ba806c95580 None None] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:19:02.828111 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eac54a4f-cfa7-46a1-8fc1-7ba806c95580 None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:19:02.828111 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eac54a4f-cfa7-46a1-8fc1-7ba806c95580 None None] PATH_INFO: `/` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:19:02.829435 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 759/1522] 127.0.0.1 () {54 vars in 901 bytes} [Tue Sep 13 03:19:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 03:19:02.836452 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45cc4b7c-2cd3-4a26-be2e-b217afa5f6e1 None None] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:19:02.836939 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45cc4b7c-2cd3-4a26-be2e-b217afa5f6e1 None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:19:02.837039 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45cc4b7c-2cd3-4a26-be2e-b217afa5f6e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:19:02.857943 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.auth.core [None req-45cc4b7c-2cd3-4a26-be2e-b217afa5f6e1 None None] MFA Rules not processed for user `49f280ab086b4610855c4ffa378517d9`. Rule list: `[]` (Enabled: `True`). {{(pid=81400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 03:19:02.859168 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-45cc4b7c-2cd3-4a26-be2e-b217afa5f6e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:19:02.867577 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 764/1523] 127.0.0.1 () {58 vars in 998 bytes} [Tue Sep 13 03:19:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 03:19:03.026048 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eac54a4f-cfa7-46a1-8fc1-7ba806c95580 None None] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:19:03.041074 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eac54a4f-cfa7-46a1-8fc1-7ba806c95580 None None] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:19:03.046582 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d180f23-4c10-455f-915b-7f64c88c3f8a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7377f5bee4a49959182e1bd34bc97a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b9d94f6a68f7470d9494f112175620dc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:19:03.047212 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d180f23-4c10-455f-915b-7f64c88c3f8a service nova] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:19:03.047488 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d180f23-4c10-455f-915b-7f64c88c3f8a service nova] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:19:03.047759 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d180f23-4c10-455f-915b-7f64c88c3f8a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:19:03.055808 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d180f23-4c10-455f-915b-7f64c88c3f8a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:19:03.057417 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d180f23-4c10-455f-915b-7f64c88c3f8a service nova] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:19:03.067063 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 760/1524] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Sep 13 03:19:03 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 03:19:04.134340 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5736ab4b-fedd-4b16-bc6f-113e063790b1 None None] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:19:04.134899 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5736ab4b-fedd-4b16-bc6f-113e063790b1 None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:19:04.134899 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5736ab4b-fedd-4b16-bc6f-113e063790b1 None None] PATH_INFO: `/` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:19:04.136263 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 765/1525] 127.0.0.1 () {54 vars in 901 bytes} [Tue Sep 13 03:19:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 03:19:04.142385 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d75ba328-ad2a-40fe-9409-9fe59a97581e None None] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:19:04.142869 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d75ba328-ad2a-40fe-9409-9fe59a97581e None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:19:04.143001 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d75ba328-ad2a-40fe-9409-9fe59a97581e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:19:04.161788 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.auth.core [None req-d75ba328-ad2a-40fe-9409-9fe59a97581e None None] MFA Rules not processed for user `49f280ab086b4610855c4ffa378517d9`. Rule list: `[]` (Enabled: `True`). {{(pid=81399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 03:19:04.163506 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-d75ba328-ad2a-40fe-9409-9fe59a97581e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:19:04.171172 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 761/1526] 127.0.0.1 () {58 vars in 998 bytes} [Tue Sep 13 03:19:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 03:19:04.322046 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5736ab4b-fedd-4b16-bc6f-113e063790b1 None None] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:19:04.336260 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5736ab4b-fedd-4b16-bc6f-113e063790b1 None None] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:19:04.341647 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-407a195d-f915-48f7-a589-5b7ed9a9ac63 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7377f5bee4a49959182e1bd34bc97a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b9d94f6a68f7470d9494f112175620dc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:19:04.342231 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-407a195d-f915-48f7-a589-5b7ed9a9ac63 service nova] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:19:04.342512 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-407a195d-f915-48f7-a589-5b7ed9a9ac63 service nova] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:19:04.342812 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-407a195d-f915-48f7-a589-5b7ed9a9ac63 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:19:04.351242 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-407a195d-f915-48f7-a589-5b7ed9a9ac63 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:19:04.352813 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-407a195d-f915-48f7-a589-5b7ed9a9ac63 service nova] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:19:04.362049 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 766/1527] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Sep 13 03:19:04 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 03:19:08.424122 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05a68c2d-b720-4465-ae2b-16700447fc0b None None] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:19:08.424644 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05a68c2d-b720-4465-ae2b-16700447fc0b None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:19:08.424754 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05a68c2d-b720-4465-ae2b-16700447fc0b None None] PATH_INFO: `/` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:19:08.426042 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 762/1528] 127.0.0.1 () {54 vars in 917 bytes} [Tue Sep 13 03:19:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 03:19:08.431518 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25472516-0ca8-421a-a7dc-a68ea3052c97 None None] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:19:08.431904 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25472516-0ca8-421a-a7dc-a68ea3052c97 None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:19:08.432671 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25472516-0ca8-421a-a7dc-a68ea3052c97 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:19:08.454447 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.auth.core [None req-25472516-0ca8-421a-a7dc-a68ea3052c97 None None] MFA Rules not processed for user `49f280ab086b4610855c4ffa378517d9`. Rule list: `[]` (Enabled: `True`). {{(pid=81400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 03:19:08.455675 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-25472516-0ca8-421a-a7dc-a68ea3052c97 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:19:08.463964 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 767/1529] 127.0.0.1 () {58 vars in 1014 bytes} [Tue Sep 13 03:19:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 03:19:08.756893 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05a68c2d-b720-4465-ae2b-16700447fc0b None None] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:19:08.758508 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-05a68c2d-b720-4465-ae2b-16700447fc0b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:19:08.836208 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05a68c2d-b720-4465-ae2b-16700447fc0b None None] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:19:08.839843 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e7fabd5-53ea-4843-b009-1ddb2db34557 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdffbc851086441ebd01022d1e4c2609', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b9d94f6a68f7470d9494f112175620dc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:19:08.840449 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e7fabd5-53ea-4843-b009-1ddb2db34557 service neutron] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:19:08.840729 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e7fabd5-53ea-4843-b009-1ddb2db34557 service neutron] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:19:08.840998 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e7fabd5-53ea-4843-b009-1ddb2db34557 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:19:08.851320 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e7fabd5-53ea-4843-b009-1ddb2db34557 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:19:08.852815 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e7fabd5-53ea-4843-b009-1ddb2db34557 service neutron] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:19:08.862225 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 763/1530] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Sep 13 03:19:08 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 03:19:10.992229 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18134330-58ed-4f1e-b240-f526c0fd9675 None None] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:19:10.992780 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18134330-58ed-4f1e-b240-f526c0fd9675 None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:19:10.992780 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18134330-58ed-4f1e-b240-f526c0fd9675 None None] PATH_INFO: `/` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:19:10.994046 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 768/1531] 127.0.0.1 () {54 vars in 917 bytes} [Tue Sep 13 03:19:10 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 03:19:10.999380 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc29fc7c-9777-44d1-9620-22c40bf2cbe2 None None] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:19:10.999836 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc29fc7c-9777-44d1-9620-22c40bf2cbe2 None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:19:10.999946 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc29fc7c-9777-44d1-9620-22c40bf2cbe2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:19:11.019042 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.auth.core [None req-bc29fc7c-9777-44d1-9620-22c40bf2cbe2 None None] MFA Rules not processed for user `49f280ab086b4610855c4ffa378517d9`. Rule list: `[]` (Enabled: `True`). {{(pid=81399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 03:19:11.020246 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-bc29fc7c-9777-44d1-9620-22c40bf2cbe2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:19:11.028034 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 764/1532] 127.0.0.1 () {58 vars in 1014 bytes} [Tue Sep 13 03:19:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 03:19:11.315879 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18134330-58ed-4f1e-b240-f526c0fd9675 None None] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:19:11.321197 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=94 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:19:11.327383 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18134330-58ed-4f1e-b240-f526c0fd9675 None None] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:19:11.330846 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adbe579b-f397-42b4-98e6-e6f9085a9044 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdffbc851086441ebd01022d1e4c2609', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b9d94f6a68f7470d9494f112175620dc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:19:11.331429 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adbe579b-f397-42b4-98e6-e6f9085a9044 service neutron] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:19:11.331697 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adbe579b-f397-42b4-98e6-e6f9085a9044 service neutron] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:19:11.331956 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adbe579b-f397-42b4-98e6-e6f9085a9044 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:19:11.341129 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adbe579b-f397-42b4-98e6-e6f9085a9044 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:19:11.342542 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adbe579b-f397-42b4-98e6-e6f9085a9044 service neutron] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:19:11.352372 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 769/1533] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Sep 13 03:19:11 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 03:19:14.943134 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d04cbd57-ce31-4160-a28c-dbf0749d01c0 None None] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:19:14.943705 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d04cbd57-ce31-4160-a28c-dbf0749d01c0 None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:19:14.943705 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d04cbd57-ce31-4160-a28c-dbf0749d01c0 None None] PATH_INFO: `/` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:19:14.945031 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 765/1534] 127.0.0.1 () {54 vars in 917 bytes} [Tue Sep 13 03:19:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 03:19:14.950110 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6cca466-291b-4921-803e-0509e1023cdf None None] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:19:14.950602 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6cca466-291b-4921-803e-0509e1023cdf None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:19:14.950739 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6cca466-291b-4921-803e-0509e1023cdf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:19:14.970204 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.auth.core [None req-d6cca466-291b-4921-803e-0509e1023cdf None None] MFA Rules not processed for user `49f280ab086b4610855c4ffa378517d9`. Rule list: `[]` (Enabled: `True`). {{(pid=81400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 03:19:14.971425 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-d6cca466-291b-4921-803e-0509e1023cdf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:19:14.979733 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 770/1535] 127.0.0.1 () {58 vars in 1014 bytes} [Tue Sep 13 03:19:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 03:19:15.287590 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d04cbd57-ce31-4160-a28c-dbf0749d01c0 None None] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:19:15.289265 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-d04cbd57-ce31-4160-a28c-dbf0749d01c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:19:15.366598 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d04cbd57-ce31-4160-a28c-dbf0749d01c0 None None] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:19:15.370140 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fee1181-b91f-46c9-9902-6aa625248da2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '924ee3f608d34a18b0ddc4695f102a56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b9d94f6a68f7470d9494f112175620dc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:19:15.370768 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fee1181-b91f-46c9-9902-6aa625248da2 service cinder] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:19:15.371045 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fee1181-b91f-46c9-9902-6aa625248da2 service cinder] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:19:15.371310 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fee1181-b91f-46c9-9902-6aa625248da2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:19:15.380057 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fee1181-b91f-46c9-9902-6aa625248da2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:19:15.381962 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fee1181-b91f-46c9-9902-6aa625248da2 service cinder] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:19:15.391123 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 766/1536] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Sep 13 03:19:15 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 03:19:17.532891 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3ece5c3-e5c1-442a-a5f1-1e93dde4eb6b None None] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:19:17.533455 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3ece5c3-e5c1-442a-a5f1-1e93dde4eb6b None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:19:17.533455 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3ece5c3-e5c1-442a-a5f1-1e93dde4eb6b None None] PATH_INFO: `/` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:19:17.534806 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 771/1537] 127.0.0.1 () {54 vars in 917 bytes} [Tue Sep 13 03:19:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 03:19:17.540034 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ab98ee-959e-4503-a524-b95dd477ff99 None None] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:19:17.540446 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ab98ee-959e-4503-a524-b95dd477ff99 None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:19:17.540581 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ab98ee-959e-4503-a524-b95dd477ff99 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:19:17.559893 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.auth.core [None req-33ab98ee-959e-4503-a524-b95dd477ff99 None None] MFA Rules not processed for user `49f280ab086b4610855c4ffa378517d9`. Rule list: `[]` (Enabled: `True`). {{(pid=81399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 03:19:17.561064 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-33ab98ee-959e-4503-a524-b95dd477ff99 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:19:17.569211 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 767/1538] 127.0.0.1 () {58 vars in 1014 bytes} [Tue Sep 13 03:19:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 03:19:17.867493 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3ece5c3-e5c1-442a-a5f1-1e93dde4eb6b None None] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:19:17.869150 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-e3ece5c3-e5c1-442a-a5f1-1e93dde4eb6b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:19:17.882664 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3ece5c3-e5c1-442a-a5f1-1e93dde4eb6b None None] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:19:17.886173 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f05b9536-1ac0-4113-82c1-21480306c2df service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '924ee3f608d34a18b0ddc4695f102a56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b9d94f6a68f7470d9494f112175620dc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:19:17.886815 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f05b9536-1ac0-4113-82c1-21480306c2df service cinder] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:19:17.887091 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f05b9536-1ac0-4113-82c1-21480306c2df service cinder] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:19:17.887359 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f05b9536-1ac0-4113-82c1-21480306c2df service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:19:17.896819 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f05b9536-1ac0-4113-82c1-21480306c2df service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:19:17.898233 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f05b9536-1ac0-4113-82c1-21480306c2df service cinder] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:19:17.907425 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 772/1539] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Sep 13 03:19:17 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 03:19:27.555013 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=92 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:19:27.903365 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=14 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:20:41.453223 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b67cf783-0f98-4e75-9103-08ac3a1a2634 None None] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:20:41.453877 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b67cf783-0f98-4e75-9103-08ac3a1a2634 None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:20:41.453877 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b67cf783-0f98-4e75-9103-08ac3a1a2634 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:20:41.475221 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.auth.core [None req-b67cf783-0f98-4e75-9103-08ac3a1a2634 None None] MFA Rules not processed for user `49f280ab086b4610855c4ffa378517d9`. Rule list: `[]` (Enabled: `True`). {{(pid=81399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 03:20:41.476762 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-b67cf783-0f98-4e75-9103-08ac3a1a2634 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:20:41.495360 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 768/1540] 10.222.0.18 () {58 vars in 958 bytes} [Tue Sep 13 03:20:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 03:20:41.504221 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f23d218e-f821-4c90-975c-c3365f95d6d6 None None] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:20:41.505894 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f23d218e-f821-4c90-975c-c3365f95d6d6 None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:20:41.505894 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f23d218e-f821-4c90-975c-c3365f95d6d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:20:41.533000 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.auth.core [None req-f23d218e-f821-4c90-975c-c3365f95d6d6 None None] MFA Rules not processed for user `49f280ab086b4610855c4ffa378517d9`. Rule list: `[]` (Enabled: `True`). {{(pid=81400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 03:20:41.534598 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-f23d218e-f821-4c90-975c-c3365f95d6d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:20:41.548136 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 773/1541] 10.222.0.18 () {58 vars in 958 bytes} [Tue Sep 13 03:20:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 03:20:41.554322 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b67cf783-0f98-4e75-9103-08ac3a1a2634 None None] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:20:41.568116 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b67cf783-0f98-4e75-9103-08ac3a1a2634 None None] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:20:41.578376 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-056b98e6-aafd-4398-a3f8-0aae11076b7f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76c5181fc7be45279ac00017bb4640b3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:20:41.579127 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-056b98e6-aafd-4398-a3f8-0aae11076b7f admin admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:20:41.579404 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-056b98e6-aafd-4398-a3f8-0aae11076b7f admin admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:20:41.579673 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-056b98e6-aafd-4398-a3f8-0aae11076b7f admin admin] PATH_INFO: `/v3/domains` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:20:41.580317 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-056b98e6-aafd-4398-a3f8-0aae11076b7f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 03:20:41.580615 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-056b98e6-aafd-4398-a3f8-0aae11076b7f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:20:41.581615 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: WARNING py.warnings [None req-056b98e6-aafd-4398-a3f8-0aae11076b7f admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 13 03:20:41.581615 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: warnings.warn(msg) Sep 13 03:20:41.581615 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: Sep 13 03:20:41.582502 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-056b98e6-aafd-4398-a3f8-0aae11076b7f admin admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:20:41.600687 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 769/1542] 10.222.0.18 () {58 vars in 1153 bytes} [Tue Sep 13 03:20:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 03:20:41.609833 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f23d218e-f821-4c90-975c-c3365f95d6d6 None None] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:20:41.624157 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f23d218e-f821-4c90-975c-c3365f95d6d6 None None] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:20:41.635373 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f9a202a-1af1-4740-a706-b835d9269e89 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76c5181fc7be45279ac00017bb4640b3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:20:41.636348 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f9a202a-1af1-4740-a706-b835d9269e89 admin admin] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:20:41.636838 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f9a202a-1af1-4740-a706-b835d9269e89 admin admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:20:41.637326 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f9a202a-1af1-4740-a706-b835d9269e89 admin admin] PATH_INFO: `/v3/projects` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:20:41.638195 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f9a202a-1af1-4740-a706-b835d9269e89 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:20:41.639482 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: WARNING py.warnings [None req-7f9a202a-1af1-4740-a706-b835d9269e89 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 13 03:20:41.639482 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: warnings.warn(msg) Sep 13 03:20:41.639482 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: Sep 13 03:20:41.641088 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f9a202a-1af1-4740-a706-b835d9269e89 admin admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:20:41.696072 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 774/1543] 10.222.0.18 () {60 vars in 1153 bytes} [Tue Sep 13 03:20:41 2022] POST /identity/v3/projects => generated 371 bytes in 86 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 03:20:41.700398 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-056b98e6-aafd-4398-a3f8-0aae11076b7f admin admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:20:41.717014 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-056b98e6-aafd-4398-a3f8-0aae11076b7f admin admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:20:41.817641 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d023565e-0d13-4d74-883b-56ae0818eac5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76c5181fc7be45279ac00017bb4640b3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:20:41.818395 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d023565e-0d13-4d74-883b-56ae0818eac5 admin admin] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:20:41.818738 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d023565e-0d13-4d74-883b-56ae0818eac5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:20:41.819321 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d023565e-0d13-4d74-883b-56ae0818eac5 admin admin] PATH_INFO: `/v3/users` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:20:41.819956 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d023565e-0d13-4d74-883b-56ae0818eac5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:20:41.824665 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: WARNING py.warnings [None req-d023565e-0d13-4d74-883b-56ae0818eac5 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 13 03:20:41.824665 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: warnings.warn(msg) Sep 13 03:20:41.824665 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: Sep 13 03:20:41.825904 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d023565e-0d13-4d74-883b-56ae0818eac5 admin admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:20:41.865190 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 770/1544] 10.222.0.18 () {60 vars in 1144 bytes} [Tue Sep 13 03:20:41 2022] POST /identity/v3/users => generated 286 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 03:20:41.869770 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f9a202a-1af1-4740-a706-b835d9269e89 admin admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:20:41.888176 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f9a202a-1af1-4740-a706-b835d9269e89 admin admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:20:41.893812 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abc3ebec-1939-45e4-9e6a-9d8d7a2b9375 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76c5181fc7be45279ac00017bb4640b3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:20:41.896867 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc3ebec-1939-45e4-9e6a-9d8d7a2b9375 admin admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:20:41.896867 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc3ebec-1939-45e4-9e6a-9d8d7a2b9375 admin admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:20:41.896867 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc3ebec-1939-45e4-9e6a-9d8d7a2b9375 admin admin] PATH_INFO: `/v3/roles` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:20:41.896867 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abc3ebec-1939-45e4-9e6a-9d8d7a2b9375 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:20:41.898753 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abc3ebec-1939-45e4-9e6a-9d8d7a2b9375 admin admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:20:41.916855 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 775/1545] 10.222.0.18 () {58 vars in 1122 bytes} [Tue Sep 13 03:20:41 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 03:20:41.921129 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d023565e-0d13-4d74-883b-56ae0818eac5 admin admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:20:41.934001 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d023565e-0d13-4d74-883b-56ae0818eac5 admin admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:20:41.938112 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0beee21-d38c-4495-8c79-be46002bf2c3 None None] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:20:41.938787 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0beee21-d38c-4495-8c79-be46002bf2c3 None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:20:41.939279 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0beee21-d38c-4495-8c79-be46002bf2c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:20:41.946177 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f9a5867-2537-4f2a-a06b-ccef8cf1d7c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76c5181fc7be45279ac00017bb4640b3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:20:41.946858 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f9a5867-2537-4f2a-a06b-ccef8cf1d7c4 admin admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:20:41.947175 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f9a5867-2537-4f2a-a06b-ccef8cf1d7c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:20:41.947406 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f9a5867-2537-4f2a-a06b-ccef8cf1d7c4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:20:41.948048 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f9a5867-2537-4f2a-a06b-ccef8cf1d7c4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:20:41.953295 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f9a5867-2537-4f2a-a06b-ccef8cf1d7c4 admin admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:20:41.969283 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.auth.core [None req-c0beee21-d38c-4495-8c79-be46002bf2c3 None None] MFA Rules not processed for user `49f280ab086b4610855c4ffa378517d9`. Rule list: `[]` (Enabled: `True`). {{(pid=81400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 03:20:41.978646 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 771/1546] 10.222.0.18 () {58 vars in 1122 bytes} [Tue Sep 13 03:20:41 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 13 03:20:41.978646 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f9a5867-2537-4f2a-a06b-ccef8cf1d7c4 admin admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:20:41.980037 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-c0beee21-d38c-4495-8c79-be46002bf2c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:20:41.993702 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 776/1547] 10.222.0.18 () {58 vars in 958 bytes} [Tue Sep 13 03:20:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 56 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 03:20:41.995377 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f9a5867-2537-4f2a-a06b-ccef8cf1d7c4 admin admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:20:42.002984 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbbf8381-f553-4060-8290-a04f4199a5ff None None] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:20:42.003649 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbbf8381-f553-4060-8290-a04f4199a5ff None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:20:42.004168 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbbf8381-f553-4060-8290-a04f4199a5ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:20:42.013899 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55fcefb6-dfd0-42ac-93cd-83d8a951c716 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76c5181fc7be45279ac00017bb4640b3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:20:42.013899 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55fcefb6-dfd0-42ac-93cd-83d8a951c716 admin admin] REQUEST_METHOD: `PUT` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:20:42.013899 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55fcefb6-dfd0-42ac-93cd-83d8a951c716 admin admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:20:42.013899 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55fcefb6-dfd0-42ac-93cd-83d8a951c716 admin admin] PATH_INFO: `/v3/projects/133f7e95cccf44e98b48b9ac551098d7/users/bdfaadaa24804313a6274259ffb2a18c/roles/1eb7a5b5203945c29e6d22d653f47847` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:20:42.041333 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.auth.core [None req-fbbf8381-f553-4060-8290-a04f4199a5ff None None] MFA Rules not processed for user `49f280ab086b4610855c4ffa378517d9`. Rule list: `[]` (Enabled: `True`). {{(pid=81400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 03:20:42.043156 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-fbbf8381-f553-4060-8290-a04f4199a5ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:20:42.057700 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55fcefb6-dfd0-42ac-93cd-83d8a951c716 admin admin] RBAC: Authorizing `identity:create_grant(project_id=133f7e95cccf44e98b48b9ac551098d7, user_id=bdfaadaa24804313a6274259ffb2a18c, role_id=1eb7a5b5203945c29e6d22d653f47847)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:20:42.060521 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55fcefb6-dfd0-42ac-93cd-83d8a951c716 admin admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:20:42.066577 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 777/1548] 10.222.0.18 () {58 vars in 958 bytes} [Tue Sep 13 03:20:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 59 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 13 03:20:42.068703 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14214707-78aa-4f61-a1e0-ef9613cfeb10 None None] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:20:42.069227 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14214707-78aa-4f61-a1e0-ef9613cfeb10 None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:20:42.069799 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14214707-78aa-4f61-a1e0-ef9613cfeb10 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:20:42.092590 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.auth.core [None req-14214707-78aa-4f61-a1e0-ef9613cfeb10 None None] MFA Rules not processed for user `49f280ab086b4610855c4ffa378517d9`. Rule list: `[]` (Enabled: `True`). {{(pid=81400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 03:20:42.094130 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-14214707-78aa-4f61-a1e0-ef9613cfeb10 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:20:42.104753 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 772/1549] 10.222.0.18 () {60 vars in 1483 bytes} [Tue Sep 13 03:20:41 2022] PUT /identity/v3/projects/133f7e95cccf44e98b48b9ac551098d7/users/bdfaadaa24804313a6274259ffb2a18c/roles/1eb7a5b5203945c29e6d22d653f47847 => generated 0 bytes in 130 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 13 03:20:42.106033 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55fcefb6-dfd0-42ac-93cd-83d8a951c716 admin admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:20:42.197097 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 778/1550] 10.222.0.18 () {58 vars in 958 bytes} [Tue Sep 13 03:20:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 03:20:42.198469 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55fcefb6-dfd0-42ac-93cd-83d8a951c716 admin admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:20:42.199050 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b99583f-e570-42ec-990d-e683e60ff404 None None] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:20:42.199523 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b99583f-e570-42ec-990d-e683e60ff404 None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:20:42.199989 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b99583f-e570-42ec-990d-e683e60ff404 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:20:42.203807 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11c6f739-a298-4989-8524-9b9ada8ac44e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76c5181fc7be45279ac00017bb4640b3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:20:42.204638 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c6f739-a298-4989-8524-9b9ada8ac44e admin admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:20:42.205084 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c6f739-a298-4989-8524-9b9ada8ac44e admin admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:20:42.205523 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c6f739-a298-4989-8524-9b9ada8ac44e admin admin] PATH_INFO: `/v3/domains` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:20:42.206372 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11c6f739-a298-4989-8524-9b9ada8ac44e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 03:20:42.207024 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11c6f739-a298-4989-8524-9b9ada8ac44e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:20:42.209712 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11c6f739-a298-4989-8524-9b9ada8ac44e admin admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:20:42.226067 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.auth.core [None req-8b99583f-e570-42ec-990d-e683e60ff404 None None] MFA Rules not processed for user `49f280ab086b4610855c4ffa378517d9`. Rule list: `[]` (Enabled: `True`). {{(pid=81400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 03:20:42.226656 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 773/1551] 10.222.0.18 () {58 vars in 1153 bytes} [Tue Sep 13 03:20:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 121 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 03:20:42.226834 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-8b99583f-e570-42ec-990d-e683e60ff404 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:20:42.228460 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f3999e-8c8a-4fbf-9c13-c4299c7ff522 None None] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:20:42.229082 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f3999e-8c8a-4fbf-9c13-c4299c7ff522 None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:20:42.229562 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f3999e-8c8a-4fbf-9c13-c4299c7ff522 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:20:42.235764 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 779/1552] 10.222.0.18 () {58 vars in 958 bytes} [Tue Sep 13 03:20:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 03:20:42.236968 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41d3c688-31dc-40ed-b230-d2d3cd9acef6 None None] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:20:42.237243 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41d3c688-31dc-40ed-b230-d2d3cd9acef6 None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:20:42.237508 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41d3c688-31dc-40ed-b230-d2d3cd9acef6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:20:42.252200 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.auth.core [None req-63f3999e-8c8a-4fbf-9c13-c4299c7ff522 None None] MFA Rules not processed for user `49f280ab086b4610855c4ffa378517d9`. Rule list: `[]` (Enabled: `True`). {{(pid=81399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 03:20:42.253415 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-63f3999e-8c8a-4fbf-9c13-c4299c7ff522 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:20:42.266674 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 774/1553] 10.222.0.18 () {58 vars in 958 bytes} [Tue Sep 13 03:20:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 03:20:42.268118 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6136e643-8dd6-47dd-a086-b9ac5fdd63bf None None] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:20:42.268407 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6136e643-8dd6-47dd-a086-b9ac5fdd63bf None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:20:42.268812 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6136e643-8dd6-47dd-a086-b9ac5fdd63bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:20:42.268925 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.auth.core [None req-41d3c688-31dc-40ed-b230-d2d3cd9acef6 None None] MFA Rules not processed for user `bdfaadaa24804313a6274259ffb2a18c`. Rule list: `[]` (Enabled: `True`). {{(pid=81400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 03:20:42.270006 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-41d3c688-31dc-40ed-b230-d2d3cd9acef6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:20:42.291372 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.auth.core [None req-6136e643-8dd6-47dd-a086-b9ac5fdd63bf None None] MFA Rules not processed for user `49f280ab086b4610855c4ffa378517d9`. Rule list: `[]` (Enabled: `True`). {{(pid=81399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 03:20:42.292577 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-6136e643-8dd6-47dd-a086-b9ac5fdd63bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:20:42.300661 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 775/1554] 10.222.0.18 () {58 vars in 958 bytes} [Tue Sep 13 03:20:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 03:20:42.301801 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3882346-593a-4aa4-859b-b132ccd5ea6c None None] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:20:42.302078 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3882346-593a-4aa4-859b-b132ccd5ea6c None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:20:42.302367 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3882346-593a-4aa4-859b-b132ccd5ea6c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:20:42.321596 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.auth.core [None req-c3882346-593a-4aa4-859b-b132ccd5ea6c None None] MFA Rules not processed for user `49f280ab086b4610855c4ffa378517d9`. Rule list: `[]` (Enabled: `True`). {{(pid=81399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 03:20:42.322814 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-c3882346-593a-4aa4-859b-b132ccd5ea6c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:20:42.330692 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 776/1555] 10.222.0.18 () {58 vars in 958 bytes} [Tue Sep 13 03:20:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 03:20:42.334461 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3882346-593a-4aa4-859b-b132ccd5ea6c None None] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:20:42.341007 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3882346-593a-4aa4-859b-b132ccd5ea6c None None] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:20:42.345589 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b63d7c8d-c93c-41e6-bb92-a1c989919ab9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76c5181fc7be45279ac00017bb4640b3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:20:42.346168 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b63d7c8d-c93c-41e6-bb92-a1c989919ab9 admin admin] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:20:42.346343 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 780/1556] 10.222.0.18 () {58 vars in 958 bytes} [Tue Sep 13 03:20:42 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 03:20:42.346459 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b63d7c8d-c93c-41e6-bb92-a1c989919ab9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:20:42.346730 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b63d7c8d-c93c-41e6-bb92-a1c989919ab9 admin admin] PATH_INFO: `/v3/projects` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:20:42.347339 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b63d7c8d-c93c-41e6-bb92-a1c989919ab9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:20:42.347581 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0464ac8-4627-4264-bb6e-df1c69983159 None None] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:20:42.347861 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0464ac8-4627-4264-bb6e-df1c69983159 None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:20:42.348126 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0464ac8-4627-4264-bb6e-df1c69983159 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:20:42.349280 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b63d7c8d-c93c-41e6-bb92-a1c989919ab9 admin admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:20:42.354697 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:20:42.368056 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.auth.core [None req-f0464ac8-4627-4264-bb6e-df1c69983159 None None] MFA Rules not processed for user `49f280ab086b4610855c4ffa378517d9`. Rule list: `[]` (Enabled: `True`). {{(pid=81400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 03:20:42.369392 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-f0464ac8-4627-4264-bb6e-df1c69983159 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:20:42.377938 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 781/1557] 10.222.0.18 () {58 vars in 958 bytes} [Tue Sep 13 03:20:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 03:20:42.382154 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 777/1558] 10.222.0.18 () {60 vars in 1153 bytes} [Tue Sep 13 03:20:42 2022] POST /identity/v3/projects => generated 389 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 03:20:42.382319 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f59a30fd-7e79-4ad3-9ca6-45403e4be708 None None] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:20:42.382319 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f59a30fd-7e79-4ad3-9ca6-45403e4be708 None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:20:42.382482 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0f91220-bd61-40a3-939f-0fd2cd6ad75d None None] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:20:42.382566 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f59a30fd-7e79-4ad3-9ca6-45403e4be708 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:20:42.382697 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0f91220-bd61-40a3-939f-0fd2cd6ad75d None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:20:42.382697 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0f91220-bd61-40a3-939f-0fd2cd6ad75d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:20:42.401015 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.auth.core [None req-b0f91220-bd61-40a3-939f-0fd2cd6ad75d None None] MFA Rules not processed for user `49f280ab086b4610855c4ffa378517d9`. Rule list: `[]` (Enabled: `True`). {{(pid=81399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 03:20:42.401285 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.auth.core [None req-f59a30fd-7e79-4ad3-9ca6-45403e4be708 None None] MFA Rules not processed for user `49f280ab086b4610855c4ffa378517d9`. Rule list: `[]` (Enabled: `True`). {{(pid=81400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 03:20:42.402219 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-b0f91220-bd61-40a3-939f-0fd2cd6ad75d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:20:42.402450 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-f59a30fd-7e79-4ad3-9ca6-45403e4be708 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:20:42.411892 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:20:42.463085 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 778/1559] 10.222.0.18 () {58 vars in 958 bytes} [Tue Sep 13 03:20:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 03:20:42.464452 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 782/1560] 10.222.0.18 () {58 vars in 958 bytes} [Tue Sep 13 03:20:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 03:20:42.464589 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc830bda-166c-4d91-a811-81b09563740d None None] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:20:42.465576 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9033455-cdf4-4d90-a783-5a620b8b79e0 None None] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:20:42.465862 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9033455-cdf4-4d90-a783-5a620b8b79e0 None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:20:42.466129 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9033455-cdf4-4d90-a783-5a620b8b79e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:20:42.466989 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc830bda-166c-4d91-a811-81b09563740d None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:20:42.467497 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc830bda-166c-4d91-a811-81b09563740d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:20:42.485316 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.auth.core [None req-b9033455-cdf4-4d90-a783-5a620b8b79e0 None None] MFA Rules not processed for user `49f280ab086b4610855c4ffa378517d9`. Rule list: `[]` (Enabled: `True`). {{(pid=81400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 03:20:42.486419 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-b9033455-cdf4-4d90-a783-5a620b8b79e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:20:42.487722 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.auth.core [None req-dc830bda-166c-4d91-a811-81b09563740d None None] MFA Rules not processed for user `49f280ab086b4610855c4ffa378517d9`. Rule list: `[]` (Enabled: `True`). {{(pid=81399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 03:20:42.488881 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-dc830bda-166c-4d91-a811-81b09563740d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:20:42.494340 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 783/1561] 10.222.0.18 () {58 vars in 958 bytes} [Tue Sep 13 03:20:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 03:20:42.495157 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9033455-cdf4-4d90-a783-5a620b8b79e0 None None] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:20:42.500956 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 779/1562] 10.222.0.18 () {58 vars in 958 bytes} [Tue Sep 13 03:20:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 03:20:42.501532 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc830bda-166c-4d91-a811-81b09563740d None None] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:20:42.504586 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9033455-cdf4-4d90-a783-5a620b8b79e0 None None] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:20:42.510018 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac50d661-5e9b-4695-b8a0-1f90123cf8c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76c5181fc7be45279ac00017bb4640b3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:20:42.510636 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac50d661-5e9b-4695-b8a0-1f90123cf8c4 admin admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:20:42.510943 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac50d661-5e9b-4695-b8a0-1f90123cf8c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:20:42.511217 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac50d661-5e9b-4695-b8a0-1f90123cf8c4 admin admin] PATH_INFO: `/v3/domains` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:20:42.511316 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc830bda-166c-4d91-a811-81b09563740d None None] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:20:42.511884 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac50d661-5e9b-4695-b8a0-1f90123cf8c4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 03:20:42.512331 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac50d661-5e9b-4695-b8a0-1f90123cf8c4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:20:42.513979 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac50d661-5e9b-4695-b8a0-1f90123cf8c4 admin admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:20:42.515737 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4241c8c2-299c-4ad6-84dd-ff93984fda1a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76c5181fc7be45279ac00017bb4640b3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:20:42.516313 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4241c8c2-299c-4ad6-84dd-ff93984fda1a admin admin] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:20:42.516588 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4241c8c2-299c-4ad6-84dd-ff93984fda1a admin admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:20:42.516856 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4241c8c2-299c-4ad6-84dd-ff93984fda1a admin admin] PATH_INFO: `/v3/projects` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:20:42.517468 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4241c8c2-299c-4ad6-84dd-ff93984fda1a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:20:42.519383 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4241c8c2-299c-4ad6-84dd-ff93984fda1a admin admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:20:42.526258 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 784/1563] 10.222.0.18 () {58 vars in 1153 bytes} [Tue Sep 13 03:20:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 03:20:42.527076 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac50d661-5e9b-4695-b8a0-1f90123cf8c4 admin admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:20:42.538887 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac50d661-5e9b-4695-b8a0-1f90123cf8c4 admin admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:20:42.544087 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17757cfd-211d-4fa8-8a6f-c5f821420ebc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76c5181fc7be45279ac00017bb4640b3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:20:42.544678 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17757cfd-211d-4fa8-8a6f-c5f821420ebc admin admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:20:42.544953 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17757cfd-211d-4fa8-8a6f-c5f821420ebc admin admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:20:42.545502 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17757cfd-211d-4fa8-8a6f-c5f821420ebc admin admin] PATH_INFO: `/v3/domains` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:20:42.546133 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17757cfd-211d-4fa8-8a6f-c5f821420ebc admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 03:20:42.546431 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17757cfd-211d-4fa8-8a6f-c5f821420ebc admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:20:42.548071 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17757cfd-211d-4fa8-8a6f-c5f821420ebc admin admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:20:42.552988 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 780/1564] 10.222.0.18 () {60 vars in 1153 bytes} [Tue Sep 13 03:20:42 2022] POST /identity/v3/projects => generated 371 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 03:20:42.552988 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4241c8c2-299c-4ad6-84dd-ff93984fda1a admin admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:20:42.567682 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 785/1565] 10.222.0.18 () {58 vars in 1153 bytes} [Tue Sep 13 03:20:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 03:20:42.567682 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a24e5694-7419-40b3-9bc6-5c8cce2e2511 None None] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:20:42.567682 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a24e5694-7419-40b3-9bc6-5c8cce2e2511 None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:20:42.567682 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a24e5694-7419-40b3-9bc6-5c8cce2e2511 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:20:42.581759 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.auth.core [None req-a24e5694-7419-40b3-9bc6-5c8cce2e2511 None None] MFA Rules not processed for user `49f280ab086b4610855c4ffa378517d9`. Rule list: `[]` (Enabled: `True`). {{(pid=81400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 03:20:42.582965 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-a24e5694-7419-40b3-9bc6-5c8cce2e2511 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:20:42.620627 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4241c8c2-299c-4ad6-84dd-ff93984fda1a admin admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:20:42.626152 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47151750-e1f2-48e9-a520-0709da5873d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76c5181fc7be45279ac00017bb4640b3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:20:42.626897 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47151750-e1f2-48e9-a520-0709da5873d9 admin admin] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:20:42.627244 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47151750-e1f2-48e9-a520-0709da5873d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:20:42.627581 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47151750-e1f2-48e9-a520-0709da5873d9 admin admin] PATH_INFO: `/v3/users` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:20:42.628271 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47151750-e1f2-48e9-a520-0709da5873d9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:20:42.630213 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47151750-e1f2-48e9-a520-0709da5873d9 admin admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:20:42.643290 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 786/1566] 10.222.0.18 () {58 vars in 958 bytes} [Tue Sep 13 03:20:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 03:20:42.644559 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13aa0bf8-7ede-4e95-950c-62849792631c None None] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:20:42.644887 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13aa0bf8-7ede-4e95-950c-62849792631c None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:20:42.645143 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13aa0bf8-7ede-4e95-950c-62849792631c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:20:42.660441 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 781/1567] 10.222.0.18 () {60 vars in 1144 bytes} [Tue Sep 13 03:20:42 2022] POST /identity/v3/users => generated 295 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 03:20:42.660441 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47151750-e1f2-48e9-a520-0709da5873d9 admin admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:20:42.665666 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.auth.core [None req-13aa0bf8-7ede-4e95-950c-62849792631c None None] MFA Rules not processed for user `49f280ab086b4610855c4ffa378517d9`. Rule list: `[]` (Enabled: `True`). {{(pid=81400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 03:20:42.666919 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-13aa0bf8-7ede-4e95-950c-62849792631c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:20:42.669291 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47151750-e1f2-48e9-a520-0709da5873d9 admin admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:20:42.674398 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e465ff71-9a20-4e0e-a999-0f5db8ca63d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76c5181fc7be45279ac00017bb4640b3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:20:42.675025 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e465ff71-9a20-4e0e-a999-0f5db8ca63d2 admin admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:20:42.675297 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e465ff71-9a20-4e0e-a999-0f5db8ca63d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:20:42.675567 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e465ff71-9a20-4e0e-a999-0f5db8ca63d2 admin admin] PATH_INFO: `/v3/domains` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:20:42.679841 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 787/1568] 10.222.0.18 () {58 vars in 958 bytes} [Tue Sep 13 03:20:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 03:20:42.679970 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e465ff71-9a20-4e0e-a999-0f5db8ca63d2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 03:20:42.680067 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13aa0bf8-7ede-4e95-950c-62849792631c None None] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:20:42.680166 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e465ff71-9a20-4e0e-a999-0f5db8ca63d2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:20:42.680166 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e465ff71-9a20-4e0e-a999-0f5db8ca63d2 admin admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:20:42.686097 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13aa0bf8-7ede-4e95-950c-62849792631c None None] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:20:42.690689 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-869d3954-e6cc-4281-9528-0d184531a7ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76c5181fc7be45279ac00017bb4640b3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:20:42.691278 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-869d3954-e6cc-4281-9528-0d184531a7ad admin admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:20:42.691491 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 782/1569] 10.222.0.18 () {58 vars in 1153 bytes} [Tue Sep 13 03:20:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 03:20:42.699117 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-869d3954-e6cc-4281-9528-0d184531a7ad admin admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:20:42.699117 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-869d3954-e6cc-4281-9528-0d184531a7ad admin admin] PATH_INFO: `/v3/domains` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:20:42.699312 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e465ff71-9a20-4e0e-a999-0f5db8ca63d2 admin admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:20:42.699391 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-869d3954-e6cc-4281-9528-0d184531a7ad admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 03:20:42.699391 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-869d3954-e6cc-4281-9528-0d184531a7ad admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:20:42.699391 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-869d3954-e6cc-4281-9528-0d184531a7ad admin admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:20:42.703229 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e465ff71-9a20-4e0e-a999-0f5db8ca63d2 admin admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:20:42.706577 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 788/1570] 10.222.0.18 () {58 vars in 1153 bytes} [Tue Sep 13 03:20:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 03:20:42.707375 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-869d3954-e6cc-4281-9528-0d184531a7ad admin admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:20:42.708173 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7afeb418-31c2-412a-9117-cacf90898607 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76c5181fc7be45279ac00017bb4640b3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:20:42.708861 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7afeb418-31c2-412a-9117-cacf90898607 admin admin] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:20:42.709138 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7afeb418-31c2-412a-9117-cacf90898607 admin admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:20:42.709404 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7afeb418-31c2-412a-9117-cacf90898607 admin admin] PATH_INFO: `/v3/projects` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:20:42.710022 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7afeb418-31c2-412a-9117-cacf90898607 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:20:42.711942 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7afeb418-31c2-412a-9117-cacf90898607 admin admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:20:42.718741 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-869d3954-e6cc-4281-9528-0d184531a7ad admin admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:20:42.723763 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df5fa95f-3206-432d-b846-fb7895233e33 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76c5181fc7be45279ac00017bb4640b3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:20:42.724378 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df5fa95f-3206-432d-b846-fb7895233e33 admin admin] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:20:42.724655 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df5fa95f-3206-432d-b846-fb7895233e33 admin admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:20:42.724924 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df5fa95f-3206-432d-b846-fb7895233e33 admin admin] PATH_INFO: `/v3/users` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:20:42.725546 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df5fa95f-3206-432d-b846-fb7895233e33 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:20:42.727486 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df5fa95f-3206-432d-b846-fb7895233e33 admin admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:20:42.742232 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 783/1571] 10.222.0.18 () {60 vars in 1153 bytes} [Tue Sep 13 03:20:42 2022] POST /identity/v3/projects => generated 373 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 03:20:42.742232 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7afeb418-31c2-412a-9117-cacf90898607 admin admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:20:42.753270 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 789/1572] 10.222.0.18 () {60 vars in 1144 bytes} [Tue Sep 13 03:20:42 2022] POST /identity/v3/users => generated 292 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 03:20:42.754384 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df5fa95f-3206-432d-b846-fb7895233e33 admin admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:20:42.757359 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:20:42.765709 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df5fa95f-3206-432d-b846-fb7895233e33 admin admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:20:42.808677 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7afeb418-31c2-412a-9117-cacf90898607 admin admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:20:42.809814 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:20:42.814177 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe4e57a5-7541-4a6b-9893-b703753d7573 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76c5181fc7be45279ac00017bb4640b3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:20:42.814900 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe4e57a5-7541-4a6b-9893-b703753d7573 admin admin] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:20:42.815403 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe4e57a5-7541-4a6b-9893-b703753d7573 admin admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:20:42.815570 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe4e57a5-7541-4a6b-9893-b703753d7573 admin admin] PATH_INFO: `/v3/projects` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:20:42.816202 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe4e57a5-7541-4a6b-9893-b703753d7573 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:20:42.818705 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe4e57a5-7541-4a6b-9893-b703753d7573 admin admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:20:42.823863 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2800e796-8d00-4579-80c6-effd27b5eea1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76c5181fc7be45279ac00017bb4640b3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:20:42.824533 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2800e796-8d00-4579-80c6-effd27b5eea1 admin admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:20:42.824787 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2800e796-8d00-4579-80c6-effd27b5eea1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:20:42.825092 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2800e796-8d00-4579-80c6-effd27b5eea1 admin admin] PATH_INFO: `/v3/domains` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:20:42.825755 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2800e796-8d00-4579-80c6-effd27b5eea1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 03:20:42.826058 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2800e796-8d00-4579-80c6-effd27b5eea1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:20:42.827683 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2800e796-8d00-4579-80c6-effd27b5eea1 admin admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:20:42.840001 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 790/1573] 10.222.0.18 () {58 vars in 1153 bytes} [Tue Sep 13 03:20:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 03:20:42.840916 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2800e796-8d00-4579-80c6-effd27b5eea1 admin admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:20:42.846274 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 784/1574] 10.222.0.18 () {60 vars in 1153 bytes} [Tue Sep 13 03:20:42 2022] POST /identity/v3/projects => generated 375 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 03:20:42.850094 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe4e57a5-7541-4a6b-9893-b703753d7573 admin admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:20:42.850878 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2800e796-8d00-4579-80c6-effd27b5eea1 admin admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:20:42.907934 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23328d0f-8452-4f08-8d33-dcc498b7756f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76c5181fc7be45279ac00017bb4640b3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:20:42.908536 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23328d0f-8452-4f08-8d33-dcc498b7756f admin admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:20:42.908811 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23328d0f-8452-4f08-8d33-dcc498b7756f admin admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:20:42.909079 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23328d0f-8452-4f08-8d33-dcc498b7756f admin admin] PATH_INFO: `/v3/roles` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:20:42.909710 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23328d0f-8452-4f08-8d33-dcc498b7756f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:20:42.910518 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe4e57a5-7541-4a6b-9893-b703753d7573 admin admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:20:42.911347 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23328d0f-8452-4f08-8d33-dcc498b7756f admin admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:20:42.915190 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75295ed7-bed0-49a8-9e98-04fff866ef11 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76c5181fc7be45279ac00017bb4640b3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:20:42.915784 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75295ed7-bed0-49a8-9e98-04fff866ef11 admin admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:20:42.916060 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75295ed7-bed0-49a8-9e98-04fff866ef11 admin admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:20:42.916331 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75295ed7-bed0-49a8-9e98-04fff866ef11 admin admin] PATH_INFO: `/v3/domains` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:20:42.916968 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75295ed7-bed0-49a8-9e98-04fff866ef11 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 03:20:42.917264 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75295ed7-bed0-49a8-9e98-04fff866ef11 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:20:42.918867 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75295ed7-bed0-49a8-9e98-04fff866ef11 admin admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:20:42.921758 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 791/1575] 10.222.0.18 () {58 vars in 1122 bytes} [Tue Sep 13 03:20:42 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 03:20:42.922549 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23328d0f-8452-4f08-8d33-dcc498b7756f admin admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:20:42.938669 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 785/1576] 10.222.0.18 () {58 vars in 1153 bytes} [Tue Sep 13 03:20:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 03:20:42.938669 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75295ed7-bed0-49a8-9e98-04fff866ef11 admin admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:20:42.938955 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23328d0f-8452-4f08-8d33-dcc498b7756f admin admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:20:42.941896 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75295ed7-bed0-49a8-9e98-04fff866ef11 admin admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:20:42.942074 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb23b7fe-79c5-47d6-813f-bc8b71d82a45 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76c5181fc7be45279ac00017bb4640b3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:20:42.942702 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb23b7fe-79c5-47d6-813f-bc8b71d82a45 admin admin] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:20:42.943651 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb23b7fe-79c5-47d6-813f-bc8b71d82a45 admin admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:20:42.943931 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb23b7fe-79c5-47d6-813f-bc8b71d82a45 admin admin] PATH_INFO: `/v3/projects` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:20:42.944558 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb23b7fe-79c5-47d6-813f-bc8b71d82a45 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:20:42.946490 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb23b7fe-79c5-47d6-813f-bc8b71d82a45 admin admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:20:42.946997 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8db9a6d-cf92-4787-a98c-96c1259e06ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76c5181fc7be45279ac00017bb4640b3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:20:42.947576 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8db9a6d-cf92-4787-a98c-96c1259e06ce admin admin] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:20:42.947847 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8db9a6d-cf92-4787-a98c-96c1259e06ce admin admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:20:42.948114 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8db9a6d-cf92-4787-a98c-96c1259e06ce admin admin] PATH_INFO: `/v3/projects` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:20:42.948724 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8db9a6d-cf92-4787-a98c-96c1259e06ce admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:20:42.950596 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8db9a6d-cf92-4787-a98c-96c1259e06ce admin admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:20:42.975603 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 792/1577] 10.222.0.18 () {60 vars in 1153 bytes} [Tue Sep 13 03:20:42 2022] POST /identity/v3/projects => generated 373 bytes in 53 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 03:20:42.976554 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb23b7fe-79c5-47d6-813f-bc8b71d82a45 admin admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:20:42.979284 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 786/1578] 10.222.0.18 () {60 vars in 1153 bytes} [Tue Sep 13 03:20:42 2022] POST /identity/v3/projects => generated 377 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 03:20:42.980042 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8db9a6d-cf92-4787-a98c-96c1259e06ce admin admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:20:43.037929 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb23b7fe-79c5-47d6-813f-bc8b71d82a45 admin admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:20:43.042243 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8db9a6d-cf92-4787-a98c-96c1259e06ce admin admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:20:43.042463 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c050e95-723b-4f2c-b373-e609d4ed1db3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76c5181fc7be45279ac00017bb4640b3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:20:43.043097 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c050e95-723b-4f2c-b373-e609d4ed1db3 admin admin] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:20:43.043368 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c050e95-723b-4f2c-b373-e609d4ed1db3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:20:43.043634 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c050e95-723b-4f2c-b373-e609d4ed1db3 admin admin] PATH_INFO: `/v3/users` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:20:43.044253 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c050e95-723b-4f2c-b373-e609d4ed1db3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:20:43.046417 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c050e95-723b-4f2c-b373-e609d4ed1db3 admin admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:20:43.047187 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0815632-580c-4031-85d2-c05cb2feafbc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76c5181fc7be45279ac00017bb4640b3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:20:43.047785 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0815632-580c-4031-85d2-c05cb2feafbc admin admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:20:43.048062 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0815632-580c-4031-85d2-c05cb2feafbc admin admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:20:43.048327 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0815632-580c-4031-85d2-c05cb2feafbc admin admin] PATH_INFO: `/v3/roles` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:20:43.048952 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0815632-580c-4031-85d2-c05cb2feafbc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:20:43.050520 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0815632-580c-4031-85d2-c05cb2feafbc admin admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:20:43.062257 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 787/1579] 10.222.0.18 () {58 vars in 1122 bytes} [Tue Sep 13 03:20:42 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 03:20:43.063111 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0815632-580c-4031-85d2-c05cb2feafbc admin admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:20:43.070344 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 793/1580] 10.222.0.18 () {60 vars in 1144 bytes} [Tue Sep 13 03:20:42 2022] POST /identity/v3/users => generated 287 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 03:20:43.072142 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c050e95-723b-4f2c-b373-e609d4ed1db3 admin admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:20:43.074529 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0815632-580c-4031-85d2-c05cb2feafbc admin admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:20:43.080264 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e35be7e3-5bdf-4e01-bb0e-4db9701008a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76c5181fc7be45279ac00017bb4640b3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:20:43.080876 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e35be7e3-5bdf-4e01-bb0e-4db9701008a9 admin admin] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:20:43.081156 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e35be7e3-5bdf-4e01-bb0e-4db9701008a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:20:43.081438 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e35be7e3-5bdf-4e01-bb0e-4db9701008a9 admin admin] PATH_INFO: `/v3/projects` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:20:43.081880 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c050e95-723b-4f2c-b373-e609d4ed1db3 admin admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:20:43.082051 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e35be7e3-5bdf-4e01-bb0e-4db9701008a9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:20:43.084125 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e35be7e3-5bdf-4e01-bb0e-4db9701008a9 admin admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:20:43.086690 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5c6ee59-ae7b-4876-a6c5-025a6286615a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76c5181fc7be45279ac00017bb4640b3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:20:43.087280 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5c6ee59-ae7b-4876-a6c5-025a6286615a admin admin] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:20:43.087550 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5c6ee59-ae7b-4876-a6c5-025a6286615a admin admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:20:43.087813 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5c6ee59-ae7b-4876-a6c5-025a6286615a admin admin] PATH_INFO: `/v3/users` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:20:43.088428 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5c6ee59-ae7b-4876-a6c5-025a6286615a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:20:43.088844 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:20:43.090492 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5c6ee59-ae7b-4876-a6c5-025a6286615a admin admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:20:43.112303 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 794/1581] 10.222.0.18 () {60 vars in 1144 bytes} [Tue Sep 13 03:20:43 2022] POST /identity/v3/users => generated 288 bytes in 40 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 03:20:43.115569 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 788/1582] 10.222.0.18 () {60 vars in 1153 bytes} [Tue Sep 13 03:20:43 2022] POST /identity/v3/projects => generated 393 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 03:20:43.116569 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e35be7e3-5bdf-4e01-bb0e-4db9701008a9 admin admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:20:43.116783 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5c6ee59-ae7b-4876-a6c5-025a6286615a admin admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:20:43.127958 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5c6ee59-ae7b-4876-a6c5-025a6286615a admin admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:20:43.152773 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:20:43.178877 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e35be7e3-5bdf-4e01-bb0e-4db9701008a9 admin admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:20:43.183639 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b9bb360-2ad6-497f-bb36-2893310cc328 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76c5181fc7be45279ac00017bb4640b3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:20:43.184252 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b9bb360-2ad6-497f-bb36-2893310cc328 admin admin] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:20:43.184534 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b9bb360-2ad6-497f-bb36-2893310cc328 admin admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:20:43.184806 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b9bb360-2ad6-497f-bb36-2893310cc328 admin admin] PATH_INFO: `/v3/projects` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:20:43.185553 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b9bb360-2ad6-497f-bb36-2893310cc328 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:20:43.187065 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dc8251a-67c4-4692-bd2c-72694767d4a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76c5181fc7be45279ac00017bb4640b3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:20:43.187523 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b9bb360-2ad6-497f-bb36-2893310cc328 admin admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:20:43.187659 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc8251a-67c4-4692-bd2c-72694767d4a2 admin admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:20:43.187932 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc8251a-67c4-4692-bd2c-72694767d4a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:20:43.188198 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc8251a-67c4-4692-bd2c-72694767d4a2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:20:43.188829 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dc8251a-67c4-4692-bd2c-72694767d4a2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:20:43.190391 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dc8251a-67c4-4692-bd2c-72694767d4a2 admin admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:20:43.200041 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 795/1583] 10.222.0.18 () {58 vars in 1122 bytes} [Tue Sep 13 03:20:43 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 03:20:43.203548 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dc8251a-67c4-4692-bd2c-72694767d4a2 admin admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:20:43.213862 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dc8251a-67c4-4692-bd2c-72694767d4a2 admin admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:20:43.218352 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 789/1584] 10.222.0.18 () {60 vars in 1153 bytes} [Tue Sep 13 03:20:43 2022] POST /identity/v3/projects => generated 385 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 03:20:43.218352 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b9bb360-2ad6-497f-bb36-2893310cc328 admin admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:20:43.271019 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-636d044a-45fd-4e6b-9b2f-c1bda68d00a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76c5181fc7be45279ac00017bb4640b3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:20:43.271646 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-636d044a-45fd-4e6b-9b2f-c1bda68d00a8 admin admin] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:20:43.271929 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-636d044a-45fd-4e6b-9b2f-c1bda68d00a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:20:43.272200 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-636d044a-45fd-4e6b-9b2f-c1bda68d00a8 admin admin] PATH_INFO: `/v3/users` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:20:43.272824 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-636d044a-45fd-4e6b-9b2f-c1bda68d00a8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:20:43.274782 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-636d044a-45fd-4e6b-9b2f-c1bda68d00a8 admin admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:20:43.276562 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b9bb360-2ad6-497f-bb36-2893310cc328 admin admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:20:43.281284 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14f79252-19a5-431f-a3f1-b35a8e516af2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76c5181fc7be45279ac00017bb4640b3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:20:43.281895 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14f79252-19a5-431f-a3f1-b35a8e516af2 admin admin] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:20:43.282158 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14f79252-19a5-431f-a3f1-b35a8e516af2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:20:43.282427 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14f79252-19a5-431f-a3f1-b35a8e516af2 admin admin] PATH_INFO: `/v3/users` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:20:43.283067 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14f79252-19a5-431f-a3f1-b35a8e516af2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:20:43.285137 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14f79252-19a5-431f-a3f1-b35a8e516af2 admin admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:20:43.297007 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 796/1585] 10.222.0.18 () {60 vars in 1144 bytes} [Tue Sep 13 03:20:43 2022] POST /identity/v3/users => generated 287 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 03:20:43.297715 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-636d044a-45fd-4e6b-9b2f-c1bda68d00a8 admin admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:20:43.308379 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-636d044a-45fd-4e6b-9b2f-c1bda68d00a8 admin admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:20:43.310288 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 790/1586] 10.222.0.18 () {60 vars in 1144 bytes} [Tue Sep 13 03:20:43 2022] POST /identity/v3/users => generated 289 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 03:20:43.310997 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14f79252-19a5-431f-a3f1-b35a8e516af2 admin admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:20:43.313074 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1303903-e61d-4e8b-967f-bab947550e4e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76c5181fc7be45279ac00017bb4640b3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:20:43.313782 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1303903-e61d-4e8b-967f-bab947550e4e admin admin] REQUEST_METHOD: `PUT` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:20:43.314098 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1303903-e61d-4e8b-967f-bab947550e4e admin admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:20:43.314345 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1303903-e61d-4e8b-967f-bab947550e4e admin admin] PATH_INFO: `/v3/projects/6cc08951599b4506b42b24e9a8fd9eed/users/23038257634a4029856b0b8cb855235e/roles/425d4b481c7846e0bc07e7e805d5b15f` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:20:43.322492 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14f79252-19a5-431f-a3f1-b35a8e516af2 admin admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:20:43.327125 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d8db7fd-5197-4c6e-ad06-cbe34648ea4c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76c5181fc7be45279ac00017bb4640b3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:20:43.327712 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d8db7fd-5197-4c6e-ad06-cbe34648ea4c admin admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:20:43.327995 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d8db7fd-5197-4c6e-ad06-cbe34648ea4c admin admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:20:43.328268 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d8db7fd-5197-4c6e-ad06-cbe34648ea4c admin admin] PATH_INFO: `/v3/roles` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:20:43.328898 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d8db7fd-5197-4c6e-ad06-cbe34648ea4c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:20:43.329512 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1303903-e61d-4e8b-967f-bab947550e4e admin admin] RBAC: Authorizing `identity:create_grant(project_id=6cc08951599b4506b42b24e9a8fd9eed, user_id=23038257634a4029856b0b8cb855235e, role_id=425d4b481c7846e0bc07e7e805d5b15f)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:20:43.330513 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d8db7fd-5197-4c6e-ad06-cbe34648ea4c admin admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:20:43.330634 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: WARNING py.warnings [None req-d1303903-e61d-4e8b-967f-bab947550e4e admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 13 03:20:43.330634 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: warnings.warn(msg) Sep 13 03:20:43.330634 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: Sep 13 03:20:43.332503 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1303903-e61d-4e8b-967f-bab947550e4e admin admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:20:43.342032 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 791/1587] 10.222.0.18 () {58 vars in 1122 bytes} [Tue Sep 13 03:20:43 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 03:20:43.345782 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d8db7fd-5197-4c6e-ad06-cbe34648ea4c admin admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:20:43.347041 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 797/1588] 10.222.0.18 () {60 vars in 1483 bytes} [Tue Sep 13 03:20:43 2022] PUT /identity/v3/projects/6cc08951599b4506b42b24e9a8fd9eed/users/23038257634a4029856b0b8cb855235e/roles/425d4b481c7846e0bc07e7e805d5b15f => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 03:20:43.347939 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1303903-e61d-4e8b-967f-bab947550e4e admin admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:20:43.356626 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d8db7fd-5197-4c6e-ad06-cbe34648ea4c admin admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:20:43.401729 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:20:43.408275 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1303903-e61d-4e8b-967f-bab947550e4e admin admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:20:43.413356 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef6aee29-1279-4c61-a0c8-ba3cf9fe7510 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76c5181fc7be45279ac00017bb4640b3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:20:43.413949 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef6aee29-1279-4c61-a0c8-ba3cf9fe7510 admin admin] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:20:43.414227 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef6aee29-1279-4c61-a0c8-ba3cf9fe7510 admin admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:20:43.414485 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef6aee29-1279-4c61-a0c8-ba3cf9fe7510 admin admin] PATH_INFO: `/v3/users` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:20:43.414878 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b6c01a9-806c-4df3-a068-16a3a4cd26b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76c5181fc7be45279ac00017bb4640b3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:20:43.415127 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef6aee29-1279-4c61-a0c8-ba3cf9fe7510 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:20:43.415483 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b6c01a9-806c-4df3-a068-16a3a4cd26b5 admin admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:20:43.415758 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b6c01a9-806c-4df3-a068-16a3a4cd26b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:20:43.416051 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b6c01a9-806c-4df3-a068-16a3a4cd26b5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:20:43.416750 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b6c01a9-806c-4df3-a068-16a3a4cd26b5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:20:43.417034 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef6aee29-1279-4c61-a0c8-ba3cf9fe7510 admin admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:20:43.418652 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b6c01a9-806c-4df3-a068-16a3a4cd26b5 admin admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:20:43.429524 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 792/1589] 10.222.0.18 () {58 vars in 1122 bytes} [Tue Sep 13 03:20:43 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 03:20:43.430661 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b6c01a9-806c-4df3-a068-16a3a4cd26b5 admin admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:20:43.439053 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 798/1590] 10.222.0.18 () {60 vars in 1144 bytes} [Tue Sep 13 03:20:43 2022] POST /identity/v3/users => generated 297 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 03:20:43.439899 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef6aee29-1279-4c61-a0c8-ba3cf9fe7510 admin admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:20:43.440210 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b6c01a9-806c-4df3-a068-16a3a4cd26b5 admin admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:20:43.445029 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfaa1fc6-c27f-459a-ad9f-24693bee767a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76c5181fc7be45279ac00017bb4640b3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:20:43.445667 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfaa1fc6-c27f-459a-ad9f-24693bee767a admin admin] REQUEST_METHOD: `PUT` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:20:43.445951 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfaa1fc6-c27f-459a-ad9f-24693bee767a admin admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:20:43.446217 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfaa1fc6-c27f-459a-ad9f-24693bee767a admin admin] PATH_INFO: `/v3/projects/9b7c5a1fd8a24526b7859fdb030ccb95/users/83bd9ab163ba4229bcfd810345430564/roles/1eb7a5b5203945c29e6d22d653f47847` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:20:43.449557 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef6aee29-1279-4c61-a0c8-ba3cf9fe7510 admin admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:20:43.454129 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90ea5573-c727-4415-b75e-952c33e7c761 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76c5181fc7be45279ac00017bb4640b3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:20:43.454746 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ea5573-c727-4415-b75e-952c33e7c761 admin admin] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:20:43.455026 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ea5573-c727-4415-b75e-952c33e7c761 admin admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:20:43.455299 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ea5573-c727-4415-b75e-952c33e7c761 admin admin] PATH_INFO: `/v3/users` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:20:43.455910 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90ea5573-c727-4415-b75e-952c33e7c761 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:20:43.457862 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90ea5573-c727-4415-b75e-952c33e7c761 admin admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:20:43.460971 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfaa1fc6-c27f-459a-ad9f-24693bee767a admin admin] RBAC: Authorizing `identity:create_grant(project_id=9b7c5a1fd8a24526b7859fdb030ccb95, user_id=83bd9ab163ba4229bcfd810345430564, role_id=1eb7a5b5203945c29e6d22d653f47847)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:20:43.463631 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfaa1fc6-c27f-459a-ad9f-24693bee767a admin admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:20:43.474128 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=90,keystone:INSERT=10 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:20:43.477683 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 793/1591] 10.222.0.18 () {60 vars in 1483 bytes} [Tue Sep 13 03:20:43 2022] PUT /identity/v3/projects/9b7c5a1fd8a24526b7859fdb030ccb95/users/83bd9ab163ba4229bcfd810345430564/roles/1eb7a5b5203945c29e6d22d653f47847 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 03:20:43.478403 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfaa1fc6-c27f-459a-ad9f-24693bee767a admin admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:20:43.480454 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 799/1592] 10.222.0.18 () {60 vars in 1144 bytes} [Tue Sep 13 03:20:43 2022] POST /identity/v3/users => generated 293 bytes in 41 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 03:20:43.481151 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90ea5573-c727-4415-b75e-952c33e7c761 admin admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:20:43.491463 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90ea5573-c727-4415-b75e-952c33e7c761 admin admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:20:43.544076 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfaa1fc6-c27f-459a-ad9f-24693bee767a admin admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:20:43.549238 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-559905c1-4d43-4c30-9512-42215ab83008 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76c5181fc7be45279ac00017bb4640b3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:20:43.549745 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afbf62bf-105a-46a7-a511-72396c43f7c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76c5181fc7be45279ac00017bb4640b3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:20:43.549868 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-559905c1-4d43-4c30-9512-42215ab83008 admin admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:20:43.549963 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afbf62bf-105a-46a7-a511-72396c43f7c6 admin admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:20:43.550068 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-559905c1-4d43-4c30-9512-42215ab83008 admin admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:20:43.550245 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afbf62bf-105a-46a7-a511-72396c43f7c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:20:43.550354 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-559905c1-4d43-4c30-9512-42215ab83008 admin admin] PATH_INFO: `/v3/roles` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:20:43.550528 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afbf62bf-105a-46a7-a511-72396c43f7c6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:20:43.550928 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-559905c1-4d43-4c30-9512-42215ab83008 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:20:43.551199 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afbf62bf-105a-46a7-a511-72396c43f7c6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:20:43.552555 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-559905c1-4d43-4c30-9512-42215ab83008 admin admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:20:43.552847 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afbf62bf-105a-46a7-a511-72396c43f7c6 admin admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:20:43.562162 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 800/1593] 10.222.0.18 () {58 vars in 1122 bytes} [Tue Sep 13 03:20:43 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 03:20:43.562642 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 794/1594] 10.222.0.18 () {58 vars in 1122 bytes} [Tue Sep 13 03:20:43 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 03:20:43.562974 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-559905c1-4d43-4c30-9512-42215ab83008 admin admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:20:43.563443 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afbf62bf-105a-46a7-a511-72396c43f7c6 admin admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:20:43.572887 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-559905c1-4d43-4c30-9512-42215ab83008 admin admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:20:43.573371 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afbf62bf-105a-46a7-a511-72396c43f7c6 admin admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:20:43.577432 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d3abbfb-0f15-4705-87f3-22eddeeffe24 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76c5181fc7be45279ac00017bb4640b3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:20:43.578023 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d3abbfb-0f15-4705-87f3-22eddeeffe24 admin admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:20:43.578137 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d233c541-2f6f-4f7c-8fb7-2806455bead7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76c5181fc7be45279ac00017bb4640b3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:20:43.578315 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d3abbfb-0f15-4705-87f3-22eddeeffe24 admin admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:20:43.578584 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d3abbfb-0f15-4705-87f3-22eddeeffe24 admin admin] PATH_INFO: `/v3/roles` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:20:43.578726 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d233c541-2f6f-4f7c-8fb7-2806455bead7 admin admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:20:43.578941 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d233c541-2f6f-4f7c-8fb7-2806455bead7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:20:43.579212 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d233c541-2f6f-4f7c-8fb7-2806455bead7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:20:43.579319 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d3abbfb-0f15-4705-87f3-22eddeeffe24 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:20:43.579837 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d233c541-2f6f-4f7c-8fb7-2806455bead7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:20:43.580768 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d3abbfb-0f15-4705-87f3-22eddeeffe24 admin admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:20:43.581425 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d233c541-2f6f-4f7c-8fb7-2806455bead7 admin admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:20:43.594910 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 795/1595] 10.222.0.18 () {58 vars in 1122 bytes} [Tue Sep 13 03:20:43 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 03:20:43.595076 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 801/1596] 10.222.0.18 () {58 vars in 1122 bytes} [Tue Sep 13 03:20:43 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 03:20:43.595161 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d233c541-2f6f-4f7c-8fb7-2806455bead7 admin admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:20:43.595258 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d3abbfb-0f15-4705-87f3-22eddeeffe24 admin admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:20:43.605062 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d3abbfb-0f15-4705-87f3-22eddeeffe24 admin admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:20:43.605278 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d233c541-2f6f-4f7c-8fb7-2806455bead7 admin admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:20:43.609570 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5df69e4a-ee8c-4d21-a4a7-bb41f9c01db5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76c5181fc7be45279ac00017bb4640b3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:20:43.610051 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c79c595e-f0ef-44aa-bcb2-51773edf0ec3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76c5181fc7be45279ac00017bb4640b3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:20:43.610169 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5df69e4a-ee8c-4d21-a4a7-bb41f9c01db5 admin admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:20:43.610391 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5df69e4a-ee8c-4d21-a4a7-bb41f9c01db5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:20:43.610644 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c79c595e-f0ef-44aa-bcb2-51773edf0ec3 admin admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:20:43.610747 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5df69e4a-ee8c-4d21-a4a7-bb41f9c01db5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:20:43.610921 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c79c595e-f0ef-44aa-bcb2-51773edf0ec3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:20:43.611190 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c79c595e-f0ef-44aa-bcb2-51773edf0ec3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:20:43.611326 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5df69e4a-ee8c-4d21-a4a7-bb41f9c01db5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:20:43.611809 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c79c595e-f0ef-44aa-bcb2-51773edf0ec3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:20:43.612867 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5df69e4a-ee8c-4d21-a4a7-bb41f9c01db5 admin admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:20:43.613360 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c79c595e-f0ef-44aa-bcb2-51773edf0ec3 admin admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:20:43.622684 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 802/1597] 10.222.0.18 () {58 vars in 1122 bytes} [Tue Sep 13 03:20:43 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 03:20:43.623871 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 796/1598] 10.222.0.18 () {58 vars in 1122 bytes} [Tue Sep 13 03:20:43 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 03:20:43.624010 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e5df568-97d6-4ea5-8328-506fd66f26a3 None None] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:20:43.624233 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e5df568-97d6-4ea5-8328-506fd66f26a3 None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:20:43.624423 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e5df568-97d6-4ea5-8328-506fd66f26a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:20:43.624536 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c79c595e-f0ef-44aa-bcb2-51773edf0ec3 admin admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:20:43.634003 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c79c595e-f0ef-44aa-bcb2-51773edf0ec3 admin admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:20:43.638956 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afc16491-52e0-44f7-b797-9c2d8c922f31 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76c5181fc7be45279ac00017bb4640b3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:20:43.639727 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc16491-52e0-44f7-b797-9c2d8c922f31 admin admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:20:43.640010 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc16491-52e0-44f7-b797-9c2d8c922f31 admin admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:20:43.640285 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc16491-52e0-44f7-b797-9c2d8c922f31 admin admin] PATH_INFO: `/v3/roles` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:20:43.640914 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afc16491-52e0-44f7-b797-9c2d8c922f31 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:20:43.642497 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afc16491-52e0-44f7-b797-9c2d8c922f31 admin admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:20:43.642950 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.auth.core [None req-0e5df568-97d6-4ea5-8328-506fd66f26a3 None None] MFA Rules not processed for user `49f280ab086b4610855c4ffa378517d9`. Rule list: `[]` (Enabled: `True`). {{(pid=81400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 03:20:43.644123 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-0e5df568-97d6-4ea5-8328-506fd66f26a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:20:43.652582 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 803/1599] 10.222.0.18 () {58 vars in 958 bytes} [Tue Sep 13 03:20:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 03:20:43.656084 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 797/1600] 10.222.0.18 () {58 vars in 1122 bytes} [Tue Sep 13 03:20:43 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 03:20:43.656084 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afc16491-52e0-44f7-b797-9c2d8c922f31 admin admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:20:43.660476 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e5df568-97d6-4ea5-8328-506fd66f26a3 None None] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:20:43.664791 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afc16491-52e0-44f7-b797-9c2d8c922f31 admin admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:20:43.669370 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e9005d1-c26d-4233-a978-fa4d9ddae21d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76c5181fc7be45279ac00017bb4640b3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:20:43.669958 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e9005d1-c26d-4233-a978-fa4d9ddae21d admin admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:20:43.670504 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e9005d1-c26d-4233-a978-fa4d9ddae21d admin admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:20:43.670804 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e9005d1-c26d-4233-a978-fa4d9ddae21d admin admin] PATH_INFO: `/v3/roles` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:20:43.671173 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e5df568-97d6-4ea5-8328-506fd66f26a3 None None] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:20:43.671432 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e9005d1-c26d-4233-a978-fa4d9ddae21d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:20:43.673023 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e9005d1-c26d-4233-a978-fa4d9ddae21d admin admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:20:43.675878 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e80e943-f7b0-46cb-8760-43eec3ebc75d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76c5181fc7be45279ac00017bb4640b3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:20:43.676557 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e80e943-f7b0-46cb-8760-43eec3ebc75d admin admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:20:43.676891 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e80e943-f7b0-46cb-8760-43eec3ebc75d admin admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:20:43.677226 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e80e943-f7b0-46cb-8760-43eec3ebc75d admin admin] PATH_INFO: `/v3/roles` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:20:43.677952 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e80e943-f7b0-46cb-8760-43eec3ebc75d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:20:43.679654 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e80e943-f7b0-46cb-8760-43eec3ebc75d admin admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:20:43.683095 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 798/1601] 10.222.0.18 () {58 vars in 1122 bytes} [Tue Sep 13 03:20:43 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 03:20:43.683855 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e9005d1-c26d-4233-a978-fa4d9ddae21d admin admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:20:43.690419 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 804/1602] 10.222.0.18 () {58 vars in 1122 bytes} [Tue Sep 13 03:20:43 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 03:20:43.690706 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e80e943-f7b0-46cb-8760-43eec3ebc75d admin admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:20:43.694951 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e9005d1-c26d-4233-a978-fa4d9ddae21d admin admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:20:43.699765 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4406f55-d207-4452-821d-d8a985513e69 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76c5181fc7be45279ac00017bb4640b3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:20:43.700243 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e80e943-f7b0-46cb-8760-43eec3ebc75d admin admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:20:43.700409 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4406f55-d207-4452-821d-d8a985513e69 admin admin] REQUEST_METHOD: `PUT` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:20:43.700690 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4406f55-d207-4452-821d-d8a985513e69 admin admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:20:43.700963 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4406f55-d207-4452-821d-d8a985513e69 admin admin] PATH_INFO: `/v3/projects/6cc08951599b4506b42b24e9a8fd9eed/users/23038257634a4029856b0b8cb855235e/roles/425d4b481c7846e0bc07e7e805d5b15f` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:20:43.702870 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4406f55-d207-4452-821d-d8a985513e69 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6cc08951599b4506b42b24e9a8fd9eed, user_id=23038257634a4029856b0b8cb855235e, role_id=425d4b481c7846e0bc07e7e805d5b15f)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:20:43.704688 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05c5e1f9-8243-4725-935c-5a8de65cb89d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76c5181fc7be45279ac00017bb4640b3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:20:43.705294 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c5e1f9-8243-4725-935c-5a8de65cb89d admin admin] REQUEST_METHOD: `PUT` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:20:43.705542 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4406f55-d207-4452-821d-d8a985513e69 admin admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:20:43.705647 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c5e1f9-8243-4725-935c-5a8de65cb89d admin admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:20:43.705816 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c5e1f9-8243-4725-935c-5a8de65cb89d admin admin] PATH_INFO: `/v3/projects/9d8c2872e912420696a20ed363471888/users/10ac3e249fb948d6a53010d5accd7097/roles/1eb7a5b5203945c29e6d22d653f47847` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:20:43.717320 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 799/1603] 10.222.0.18 () {60 vars in 1483 bytes} [Tue Sep 13 03:20:43 2022] PUT /identity/v3/projects/6cc08951599b4506b42b24e9a8fd9eed/users/23038257634a4029856b0b8cb855235e/roles/425d4b481c7846e0bc07e7e805d5b15f => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 03:20:43.718101 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4406f55-d207-4452-821d-d8a985513e69 admin admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:20:43.721656 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05c5e1f9-8243-4725-935c-5a8de65cb89d admin admin] RBAC: Authorizing `identity:create_grant(project_id=9d8c2872e912420696a20ed363471888, user_id=10ac3e249fb948d6a53010d5accd7097, role_id=1eb7a5b5203945c29e6d22d653f47847)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:20:43.724376 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05c5e1f9-8243-4725-935c-5a8de65cb89d admin admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:20:43.738206 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 805/1604] 10.222.0.18 () {60 vars in 1483 bytes} [Tue Sep 13 03:20:43 2022] PUT /identity/v3/projects/9d8c2872e912420696a20ed363471888/users/10ac3e249fb948d6a53010d5accd7097/roles/1eb7a5b5203945c29e6d22d653f47847 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 03:20:43.739086 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05c5e1f9-8243-4725-935c-5a8de65cb89d admin admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:20:43.798718 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4406f55-d207-4452-821d-d8a985513e69 admin admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:20:43.801865 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05c5e1f9-8243-4725-935c-5a8de65cb89d admin admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:20:43.803648 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c37956cf-d459-43af-8d14-60f467af09b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76c5181fc7be45279ac00017bb4640b3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:20:43.804247 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c37956cf-d459-43af-8d14-60f467af09b8 admin admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:20:43.804529 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c37956cf-d459-43af-8d14-60f467af09b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:20:43.804800 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c37956cf-d459-43af-8d14-60f467af09b8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:20:43.805432 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c37956cf-d459-43af-8d14-60f467af09b8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:20:43.806488 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c78fdd7-c746-4f75-b8bd-f75b1452e487 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76c5181fc7be45279ac00017bb4640b3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:20:43.807158 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c78fdd7-c746-4f75-b8bd-f75b1452e487 admin admin] REQUEST_METHOD: `PUT` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:20:43.807365 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c37956cf-d459-43af-8d14-60f467af09b8 admin admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:20:43.807465 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c78fdd7-c746-4f75-b8bd-f75b1452e487 admin admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:20:43.807706 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c78fdd7-c746-4f75-b8bd-f75b1452e487 admin admin] PATH_INFO: `/v3/projects/bb8f67ac0f3c48b2933b6ae34d4fd838/users/80f6f9a952d84ccab0de028d58aa6914/roles/1eb7a5b5203945c29e6d22d653f47847` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:20:43.808716 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:20:43.819884 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 800/1605] 10.222.0.18 () {58 vars in 1122 bytes} [Tue Sep 13 03:20:43 2022] GET /identity/v3/roles => generated 1420 bytes in 102 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 03:20:43.820719 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c37956cf-d459-43af-8d14-60f467af09b8 admin admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:20:43.824721 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c78fdd7-c746-4f75-b8bd-f75b1452e487 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bb8f67ac0f3c48b2933b6ae34d4fd838, user_id=80f6f9a952d84ccab0de028d58aa6914, role_id=1eb7a5b5203945c29e6d22d653f47847)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:20:43.827530 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c78fdd7-c746-4f75-b8bd-f75b1452e487 admin admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:20:43.832557 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c37956cf-d459-43af-8d14-60f467af09b8 admin admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:20:43.839285 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7390f0a4-6982-43cf-a389-36e82100b89b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76c5181fc7be45279ac00017bb4640b3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:20:43.839889 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7390f0a4-6982-43cf-a389-36e82100b89b admin admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:20:43.840158 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7390f0a4-6982-43cf-a389-36e82100b89b admin admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:20:43.840424 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7390f0a4-6982-43cf-a389-36e82100b89b admin admin] PATH_INFO: `/v3/roles` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:20:43.841047 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7390f0a4-6982-43cf-a389-36e82100b89b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:20:43.845951 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 806/1606] 10.222.0.18 () {60 vars in 1483 bytes} [Tue Sep 13 03:20:43 2022] PUT /identity/v3/projects/bb8f67ac0f3c48b2933b6ae34d4fd838/users/80f6f9a952d84ccab0de028d58aa6914/roles/1eb7a5b5203945c29e6d22d653f47847 => generated 0 bytes in 103 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 03:20:43.846113 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7390f0a4-6982-43cf-a389-36e82100b89b admin admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:20:43.846221 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c78fdd7-c746-4f75-b8bd-f75b1452e487 admin admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:20:43.847354 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-8c78fdd7-c746-4f75-b8bd-f75b1452e487 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:20:43.855810 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 801/1607] 10.222.0.18 () {58 vars in 1122 bytes} [Tue Sep 13 03:20:43 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 03:20:43.855810 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7390f0a4-6982-43cf-a389-36e82100b89b admin admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:20:43.864643 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7390f0a4-6982-43cf-a389-36e82100b89b admin admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:20:43.867456 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:20:43.907873 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c78fdd7-c746-4f75-b8bd-f75b1452e487 admin admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:20:43.911551 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d4bfcc1-6e93-481a-9d64-50b785242e92 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdffbc851086441ebd01022d1e4c2609', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b9d94f6a68f7470d9494f112175620dc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:20:43.912183 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d4bfcc1-6e93-481a-9d64-50b785242e92 service neutron] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:20:43.912468 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d4bfcc1-6e93-481a-9d64-50b785242e92 service neutron] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:20:43.912734 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d4bfcc1-6e93-481a-9d64-50b785242e92 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:20:43.921627 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a60f7f6-a8a8-4b93-9440-cf930a5a9bed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76c5181fc7be45279ac00017bb4640b3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:20:43.922764 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a60f7f6-a8a8-4b93-9440-cf930a5a9bed admin admin] REQUEST_METHOD: `PUT` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:20:43.923167 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a60f7f6-a8a8-4b93-9440-cf930a5a9bed admin admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:20:43.923448 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a60f7f6-a8a8-4b93-9440-cf930a5a9bed admin admin] PATH_INFO: `/v3/projects/53d5e604b12545a68da3655cb6b881c5/users/616dd34b9fde4ed6b42ec22421ebb504/roles/1eb7a5b5203945c29e6d22d653f47847` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:20:43.938193 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a60f7f6-a8a8-4b93-9440-cf930a5a9bed admin admin] RBAC: Authorizing `identity:create_grant(project_id=53d5e604b12545a68da3655cb6b881c5, user_id=616dd34b9fde4ed6b42ec22421ebb504, role_id=1eb7a5b5203945c29e6d22d653f47847)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:20:43.940904 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a60f7f6-a8a8-4b93-9440-cf930a5a9bed admin admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:20:43.955580 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 802/1608] 10.222.0.18 () {60 vars in 1483 bytes} [Tue Sep 13 03:20:43 2022] PUT /identity/v3/projects/53d5e604b12545a68da3655cb6b881c5/users/616dd34b9fde4ed6b42ec22421ebb504/roles/1eb7a5b5203945c29e6d22d653f47847 => generated 0 bytes in 102 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 03:20:43.956483 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a60f7f6-a8a8-4b93-9440-cf930a5a9bed admin admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:20:43.975400 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d4bfcc1-6e93-481a-9d64-50b785242e92 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:20:43.976842 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d4bfcc1-6e93-481a-9d64-50b785242e92 service neutron] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:20:43.987384 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 807/1609] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Sep 13 03:20:43 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 145 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 03:20:43.988827 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a45f62-7061-41fc-839d-9d2438a58e16 None None] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:20:43.989236 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a45f62-7061-41fc-839d-9d2438a58e16 None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:20:43.989522 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a45f62-7061-41fc-839d-9d2438a58e16 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:20:44.022040 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.auth.core [None req-a2a45f62-7061-41fc-839d-9d2438a58e16 None None] MFA Rules not processed for user `23038257634a4029856b0b8cb855235e`. Rule list: `[]` (Enabled: `True`). {{(pid=81400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 03:20:44.022636 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a60f7f6-a8a8-4b93-9440-cf930a5a9bed admin admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:20:44.023377 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-a2a45f62-7061-41fc-839d-9d2438a58e16 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:20:44.027652 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fffebbe3-d793-4bba-bec5-adff37b63bf4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76c5181fc7be45279ac00017bb4640b3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:20:44.028334 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fffebbe3-d793-4bba-bec5-adff37b63bf4 admin admin] REQUEST_METHOD: `PUT` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:20:44.028607 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fffebbe3-d793-4bba-bec5-adff37b63bf4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:20:44.028883 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fffebbe3-d793-4bba-bec5-adff37b63bf4 admin admin] PATH_INFO: `/v3/projects/d018975f5fd14c808fc149f9f178b185/users/77e5e1a2032f408faaa53ba4cb2f5dc1/roles/1eb7a5b5203945c29e6d22d653f47847` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:20:44.044458 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fffebbe3-d793-4bba-bec5-adff37b63bf4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d018975f5fd14c808fc149f9f178b185, user_id=77e5e1a2032f408faaa53ba4cb2f5dc1, role_id=1eb7a5b5203945c29e6d22d653f47847)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:20:44.047443 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fffebbe3-d793-4bba-bec5-adff37b63bf4 admin admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:20:44.074687 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 803/1610] 10.222.0.18 () {60 vars in 1483 bytes} [Tue Sep 13 03:20:43 2022] PUT /identity/v3/projects/d018975f5fd14c808fc149f9f178b185/users/77e5e1a2032f408faaa53ba4cb2f5dc1/roles/1eb7a5b5203945c29e6d22d653f47847 => generated 0 bytes in 114 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 03:20:44.077515 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a13c16-68bc-475d-8d99-8ebb339ea1bd None None] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:20:44.078212 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a13c16-68bc-475d-8d99-8ebb339ea1bd None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:20:44.078993 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a13c16-68bc-475d-8d99-8ebb339ea1bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:20:44.110124 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 808/1611] 10.222.0.18 () {58 vars in 958 bytes} [Tue Sep 13 03:20:43 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 03:20:44.110671 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.auth.core [None req-95a13c16-68bc-475d-8d99-8ebb339ea1bd None None] MFA Rules not processed for user `10ac3e249fb948d6a53010d5accd7097`. Rule list: `[]` (Enabled: `True`). {{(pid=81399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 03:20:44.111933 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-95a13c16-68bc-475d-8d99-8ebb339ea1bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:20:44.119675 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2a45f62-7061-41fc-839d-9d2438a58e16 None None] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:20:44.134498 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2a45f62-7061-41fc-839d-9d2438a58e16 None None] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:20:44.159144 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:20:44.189968 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:20:44.194755 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 804/1612] 10.222.0.18 () {58 vars in 958 bytes} [Tue Sep 13 03:20:44 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 03:20:44.196977 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1bdead1-7171-4f77-b5b1-94a9b3de5eed None None] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:20:44.197287 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1bdead1-7171-4f77-b5b1-94a9b3de5eed None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:20:44.197630 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1bdead1-7171-4f77-b5b1-94a9b3de5eed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:20:44.205336 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df6b1c11-fd0f-4ed5-9466-731d399c9b73 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76c5181fc7be45279ac00017bb4640b3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:20:44.207042 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df6b1c11-fd0f-4ed5-9466-731d399c9b73 admin admin] REQUEST_METHOD: `PUT` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:20:44.207391 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df6b1c11-fd0f-4ed5-9466-731d399c9b73 admin admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:20:44.207731 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df6b1c11-fd0f-4ed5-9466-731d399c9b73 admin admin] PATH_INFO: `/v3/projects/609f8dcac1034d09a04192f4238b2857/users/b85258f01e1f436585eb6b9a4f3813b8/roles/1eb7a5b5203945c29e6d22d653f47847` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:20:44.220358 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.auth.core [None req-b1bdead1-7171-4f77-b5b1-94a9b3de5eed None None] MFA Rules not processed for user `49f280ab086b4610855c4ffa378517d9`. Rule list: `[]` (Enabled: `True`). {{(pid=81399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 03:20:44.221717 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-b1bdead1-7171-4f77-b5b1-94a9b3de5eed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:20:44.226923 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df6b1c11-fd0f-4ed5-9466-731d399c9b73 admin admin] RBAC: Authorizing `identity:create_grant(project_id=609f8dcac1034d09a04192f4238b2857, user_id=b85258f01e1f436585eb6b9a4f3813b8, role_id=1eb7a5b5203945c29e6d22d653f47847)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:20:44.229836 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 805/1613] 10.222.0.18 () {58 vars in 958 bytes} [Tue Sep 13 03:20:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 03:20:44.233096 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df6b1c11-fd0f-4ed5-9466-731d399c9b73 admin admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:20:44.233233 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1bdead1-7171-4f77-b5b1-94a9b3de5eed None None] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:20:44.242579 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1bdead1-7171-4f77-b5b1-94a9b3de5eed None None] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:20:44.253602 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 809/1614] 10.222.0.18 () {60 vars in 1483 bytes} [Tue Sep 13 03:20:44 2022] PUT /identity/v3/projects/609f8dcac1034d09a04192f4238b2857/users/b85258f01e1f436585eb6b9a4f3813b8/roles/1eb7a5b5203945c29e6d22d653f47847 => generated 0 bytes in 143 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 03:20:44.255042 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a7a2e9-5d94-4c78-be2a-078a44d563aa None None] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:20:44.255366 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a7a2e9-5d94-4c78-be2a-078a44d563aa None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:20:44.257066 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a7a2e9-5d94-4c78-be2a-078a44d563aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:20:44.278549 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.auth.core [None req-c8a7a2e9-5d94-4c78-be2a-078a44d563aa None None] MFA Rules not processed for user `49f280ab086b4610855c4ffa378517d9`. Rule list: `[]` (Enabled: `True`). {{(pid=81400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 03:20:44.279767 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-c8a7a2e9-5d94-4c78-be2a-078a44d563aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:20:44.309203 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9b544e2-831a-4995-841c-b024e0762561 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76c5181fc7be45279ac00017bb4640b3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:20:44.310012 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b544e2-831a-4995-841c-b024e0762561 admin admin] REQUEST_METHOD: `PUT` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:20:44.310304 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b544e2-831a-4995-841c-b024e0762561 admin admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:20:44.310741 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b544e2-831a-4995-841c-b024e0762561 admin admin] PATH_INFO: `/v3/projects/676c9aac6fa348b89a2d5cef106f1a74/users/4f76a7c6fe9347b0bea5f701bccb6150/roles/1eb7a5b5203945c29e6d22d653f47847` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:20:44.327377 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9b544e2-831a-4995-841c-b024e0762561 admin admin] RBAC: Authorizing `identity:create_grant(project_id=676c9aac6fa348b89a2d5cef106f1a74, user_id=4f76a7c6fe9347b0bea5f701bccb6150, role_id=1eb7a5b5203945c29e6d22d653f47847)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:20:44.330065 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9b544e2-831a-4995-841c-b024e0762561 admin admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:20:44.345445 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 806/1615] 10.222.0.18 () {60 vars in 1483 bytes} [Tue Sep 13 03:20:44 2022] PUT /identity/v3/projects/676c9aac6fa348b89a2d5cef106f1a74/users/4f76a7c6fe9347b0bea5f701bccb6150/roles/1eb7a5b5203945c29e6d22d653f47847 => generated 0 bytes in 115 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 03:20:44.347968 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-906d4f4c-a644-4a3f-ae6a-f799b23c6474 None None] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:20:44.347968 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-906d4f4c-a644-4a3f-ae6a-f799b23c6474 None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:20:44.347968 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-906d4f4c-a644-4a3f-ae6a-f799b23c6474 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:20:44.351927 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 810/1616] 10.222.0.18 () {58 vars in 958 bytes} [Tue Sep 13 03:20:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 03:20:44.352684 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8a7a2e9-5d94-4c78-be2a-078a44d563aa None None] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:20:44.379812 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.auth.core [None req-906d4f4c-a644-4a3f-ae6a-f799b23c6474 None None] MFA Rules not processed for user `77e5e1a2032f408faaa53ba4cb2f5dc1`. Rule list: `[]` (Enabled: `True`). {{(pid=81399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 03:20:44.381187 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-906d4f4c-a644-4a3f-ae6a-f799b23c6474 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:20:44.413119 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8a7a2e9-5d94-4c78-be2a-078a44d563aa None None] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:20:44.465426 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 807/1617] 10.222.0.18 () {58 vars in 958 bytes} [Tue Sep 13 03:20:44 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 03:20:44.465426 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47266010-2633-4df2-b861-1452229ed043 None None] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:20:44.465426 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47266010-2633-4df2-b861-1452229ed043 None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:20:44.466127 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47266010-2633-4df2-b861-1452229ed043 None None] PATH_INFO: `/` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:20:44.467205 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 808/1618] 10.222.0.18 () {54 vars in 962 bytes} [Tue Sep 13 03:20:44 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 03:20:44.467999 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47266010-2633-4df2-b861-1452229ed043 None None] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:20:44.479093 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e64c9b7f-0cbf-4c9a-939d-4af70098d03b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '924ee3f608d34a18b0ddc4695f102a56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b9d94f6a68f7470d9494f112175620dc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:20:44.480546 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e64c9b7f-0cbf-4c9a-939d-4af70098d03b service cinder] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:20:44.481182 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e64c9b7f-0cbf-4c9a-939d-4af70098d03b service cinder] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:20:44.481743 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e64c9b7f-0cbf-4c9a-939d-4af70098d03b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:20:44.551032 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:20:44.563653 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:20:44.565299 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47266010-2633-4df2-b861-1452229ed043 None None] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:20:44.573899 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d98d5026-21cd-4374-97e0-0754877419ed service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdffbc851086441ebd01022d1e4c2609', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b9d94f6a68f7470d9494f112175620dc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:20:44.575128 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d98d5026-21cd-4374-97e0-0754877419ed service neutron] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:20:44.575751 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d98d5026-21cd-4374-97e0-0754877419ed service neutron] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:20:44.576210 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d98d5026-21cd-4374-97e0-0754877419ed service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:20:44.586860 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e64c9b7f-0cbf-4c9a-939d-4af70098d03b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:20:44.588312 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e64c9b7f-0cbf-4c9a-939d-4af70098d03b service cinder] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:20:44.601148 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 811/1619] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Sep 13 03:20:44 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 249 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 03:20:44.602581 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d5be7a5-2696-45a1-9891-5b448c39dc75 None None] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:20:44.602954 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d5be7a5-2696-45a1-9891-5b448c39dc75 None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:20:44.603293 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d5be7a5-2696-45a1-9891-5b448c39dc75 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:20:44.626805 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.auth.core [None req-4d5be7a5-2696-45a1-9891-5b448c39dc75 None None] MFA Rules not processed for user `49f280ab086b4610855c4ffa378517d9`. Rule list: `[]` (Enabled: `True`). {{(pid=81400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 03:20:44.628233 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-4d5be7a5-2696-45a1-9891-5b448c39dc75 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:20:44.674466 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d98d5026-21cd-4374-97e0-0754877419ed service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:20:44.679635 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d98d5026-21cd-4374-97e0-0754877419ed service neutron] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:20:44.694901 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 809/1620] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Sep 13 03:20:44 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 227 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 03:20:44.696535 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2de61152-cd99-451b-b286-d347056b2ec0 None None] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:20:44.696823 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2de61152-cd99-451b-b286-d347056b2ec0 None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:20:44.697093 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2de61152-cd99-451b-b286-d347056b2ec0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:20:44.714604 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 812/1621] 10.222.0.18 () {58 vars in 958 bytes} [Tue Sep 13 03:20:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 03:20:44.717920 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.auth.core [None req-2de61152-cd99-451b-b286-d347056b2ec0 None None] MFA Rules not processed for user `49f280ab086b4610855c4ffa378517d9`. Rule list: `[]` (Enabled: `True`). {{(pid=81399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 03:20:44.719554 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d5be7a5-2696-45a1-9891-5b448c39dc75 None None] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:20:44.720881 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-2de61152-cd99-451b-b286-d347056b2ec0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:20:44.733086 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 810/1622] 10.222.0.18 () {58 vars in 958 bytes} [Tue Sep 13 03:20:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 03:20:44.736954 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d5be7a5-2696-45a1-9891-5b448c39dc75 None None] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:20:44.738219 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa00b09a-05c2-43cf-987e-028ad9c45a3d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdffbc851086441ebd01022d1e4c2609', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b9d94f6a68f7470d9494f112175620dc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:20:44.738363 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2de61152-cd99-451b-b286-d347056b2ec0 None None] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:20:44.739212 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa00b09a-05c2-43cf-987e-028ad9c45a3d service neutron] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:20:44.740692 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa00b09a-05c2-43cf-987e-028ad9c45a3d service neutron] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:20:44.741155 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa00b09a-05c2-43cf-987e-028ad9c45a3d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:20:44.751747 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2de61152-cd99-451b-b286-d347056b2ec0 None None] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:20:44.753913 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa00b09a-05c2-43cf-987e-028ad9c45a3d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:20:44.755726 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa00b09a-05c2-43cf-987e-028ad9c45a3d service neutron] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:20:44.763253 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e6463ae-9f6a-402c-97f7-f85ade73d0ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76c5181fc7be45279ac00017bb4640b3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:20:44.764016 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e6463ae-9f6a-402c-97f7-f85ade73d0ea admin admin] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:20:44.764292 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e6463ae-9f6a-402c-97f7-f85ade73d0ea admin admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:20:44.764561 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e6463ae-9f6a-402c-97f7-f85ade73d0ea admin admin] PATH_INFO: `/v3/projects` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:20:44.765196 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e6463ae-9f6a-402c-97f7-f85ade73d0ea admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:20:44.767150 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e6463ae-9f6a-402c-97f7-f85ade73d0ea admin admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:20:44.767696 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 813/1623] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Sep 13 03:20:44 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 53 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 03:20:44.769096 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a0c19d-c816-4480-a0d4-cc5c5daa9590 None None] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:20:44.769377 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a0c19d-c816-4480-a0d4-cc5c5daa9590 None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:20:44.769647 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a0c19d-c816-4480-a0d4-cc5c5daa9590 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:20:44.803737 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 811/1624] 10.222.0.18 () {60 vars in 1153 bytes} [Tue Sep 13 03:20:44 2022] POST /identity/v3/projects => generated 373 bytes in 66 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 03:20:44.804670 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e6463ae-9f6a-402c-97f7-f85ade73d0ea admin admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:20:44.817787 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.auth.core [None req-17a0c19d-c816-4480-a0d4-cc5c5daa9590 None None] MFA Rules not processed for user `c7377f5bee4a49959182e1bd34bc97a4`. Rule list: `[]` (Enabled: `True`). {{(pid=81400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 03:20:44.821687 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-17a0c19d-c816-4480-a0d4-cc5c5daa9590 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:20:44.868013 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e6463ae-9f6a-402c-97f7-f85ade73d0ea admin admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:20:44.871892 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a37feb3d-e7fb-4dad-af8d-bf085ceda454 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdffbc851086441ebd01022d1e4c2609', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b9d94f6a68f7470d9494f112175620dc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:20:44.872482 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a37feb3d-e7fb-4dad-af8d-bf085ceda454 service neutron] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:20:44.873015 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a37feb3d-e7fb-4dad-af8d-bf085ceda454 service neutron] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:20:44.873015 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a37feb3d-e7fb-4dad-af8d-bf085ceda454 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:20:44.906491 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 814/1625] 10.222.0.18 () {58 vars in 1059 bytes} [Tue Sep 13 03:20:44 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 03:20:44.907263 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17a0c19d-c816-4480-a0d4-cc5c5daa9590 None None] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:20:44.922030 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17a0c19d-c816-4480-a0d4-cc5c5daa9590 None None] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:20:44.927311 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df156f76-a672-46ae-bcbe-cf61d3880c0b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdffbc851086441ebd01022d1e4c2609', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b9d94f6a68f7470d9494f112175620dc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:20:44.928244 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df156f76-a672-46ae-bcbe-cf61d3880c0b service neutron] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:20:44.928596 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df156f76-a672-46ae-bcbe-cf61d3880c0b service neutron] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:20:44.929274 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df156f76-a672-46ae-bcbe-cf61d3880c0b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:20:44.969610 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a37feb3d-e7fb-4dad-af8d-bf085ceda454 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:20:44.971212 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a37feb3d-e7fb-4dad-af8d-bf085ceda454 service neutron] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:20:44.984302 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 812/1626] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Sep 13 03:20:44 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 180 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 03:20:44.985280 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a37feb3d-e7fb-4dad-af8d-bf085ceda454 service neutron] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:20:44.992390 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:20:45.016329 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:20:45.042072 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df156f76-a672-46ae-bcbe-cf61d3880c0b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:20:45.044687 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df156f76-a672-46ae-bcbe-cf61d3880c0b service neutron] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:20:45.071669 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 815/1627] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Sep 13 03:20:44 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 03:20:45.072448 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df156f76-a672-46ae-bcbe-cf61d3880c0b service neutron] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:20:45.094973 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a37feb3d-e7fb-4dad-af8d-bf085ceda454 service neutron] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:20:45.095755 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df156f76-a672-46ae-bcbe-cf61d3880c0b service neutron] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:20:45.100314 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f7596c7-b851-4579-bc00-43d9f9bca349 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '924ee3f608d34a18b0ddc4695f102a56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b9d94f6a68f7470d9494f112175620dc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:20:45.100930 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f7596c7-b851-4579-bc00-43d9f9bca349 service cinder] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:20:45.101371 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f7596c7-b851-4579-bc00-43d9f9bca349 service cinder] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:20:45.101781 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f7596c7-b851-4579-bc00-43d9f9bca349 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:20:45.107351 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56d2d806-dd95-4fa4-9d76-adc7f8c3a103 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76c5181fc7be45279ac00017bb4640b3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:20:45.107986 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d2d806-dd95-4fa4-9d76-adc7f8c3a103 admin admin] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:20:45.108265 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d2d806-dd95-4fa4-9d76-adc7f8c3a103 admin admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:20:45.108538 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d2d806-dd95-4fa4-9d76-adc7f8c3a103 admin admin] PATH_INFO: `/v3/users` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:20:45.109223 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56d2d806-dd95-4fa4-9d76-adc7f8c3a103 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:20:45.112339 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56d2d806-dd95-4fa4-9d76-adc7f8c3a103 admin admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:20:45.157965 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 816/1628] 10.222.0.18 () {60 vars in 1144 bytes} [Tue Sep 13 03:20:45 2022] POST /identity/v3/users => generated 293 bytes in 83 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 03:20:45.157965 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56d2d806-dd95-4fa4-9d76-adc7f8c3a103 admin admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:20:45.188624 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56d2d806-dd95-4fa4-9d76-adc7f8c3a103 admin admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:20:45.204159 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15664297-923c-401d-be70-a006770f905e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7377f5bee4a49959182e1bd34bc97a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b9d94f6a68f7470d9494f112175620dc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:20:45.204924 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15664297-923c-401d-be70-a006770f905e service nova] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:20:45.205206 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15664297-923c-401d-be70-a006770f905e service nova] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:20:45.205475 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15664297-923c-401d-be70-a006770f905e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:20:45.293347 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f7596c7-b851-4579-bc00-43d9f9bca349 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:20:45.295658 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f7596c7-b851-4579-bc00-43d9f9bca349 service cinder] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:20:45.317851 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 813/1629] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Sep 13 03:20:44 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 333 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 03:20:45.320739 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f7596c7-b851-4579-bc00-43d9f9bca349 service cinder] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:20:45.335839 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f7596c7-b851-4579-bc00-43d9f9bca349 service cinder] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:20:45.336442 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15664297-923c-401d-be70-a006770f905e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:20:45.338077 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15664297-923c-401d-be70-a006770f905e service nova] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:20:45.341532 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-587636b5-7713-4596-a24c-1f5837388d13 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76c5181fc7be45279ac00017bb4640b3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:20:45.342156 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-587636b5-7713-4596-a24c-1f5837388d13 admin admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:20:45.342431 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-587636b5-7713-4596-a24c-1f5837388d13 admin admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:20:45.342722 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-587636b5-7713-4596-a24c-1f5837388d13 admin admin] PATH_INFO: `/v3/roles` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:20:45.343495 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-587636b5-7713-4596-a24c-1f5837388d13 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:20:45.345345 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-587636b5-7713-4596-a24c-1f5837388d13 admin admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:20:45.358322 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 814/1630] 10.222.0.18 () {58 vars in 1122 bytes} [Tue Sep 13 03:20:45 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 03:20:45.361782 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-587636b5-7713-4596-a24c-1f5837388d13 admin admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:20:45.362882 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 817/1631] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Sep 13 03:20:45 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 207 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 03:20:45.373512 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-587636b5-7713-4596-a24c-1f5837388d13 admin admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:20:45.379194 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17d60ea1-c940-4f1f-9f64-04cc968c46c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76c5181fc7be45279ac00017bb4640b3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:20:45.380094 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17d60ea1-c940-4f1f-9f64-04cc968c46c2 admin admin] REQUEST_METHOD: `PUT` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:20:45.380369 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17d60ea1-c940-4f1f-9f64-04cc968c46c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:20:45.381383 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17d60ea1-c940-4f1f-9f64-04cc968c46c2 admin admin] PATH_INFO: `/v3/projects/4563823041354123a4dd7a3913fcd0e6/users/38473290033c4666a1010621d9514173/roles/425d4b481c7846e0bc07e7e805d5b15f` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:20:45.399032 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17d60ea1-c940-4f1f-9f64-04cc968c46c2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4563823041354123a4dd7a3913fcd0e6, user_id=38473290033c4666a1010621d9514173, role_id=425d4b481c7846e0bc07e7e805d5b15f)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:20:45.403428 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17d60ea1-c940-4f1f-9f64-04cc968c46c2 admin admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:20:45.421675 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 815/1632] 10.222.0.18 () {60 vars in 1483 bytes} [Tue Sep 13 03:20:45 2022] PUT /identity/v3/projects/4563823041354123a4dd7a3913fcd0e6/users/38473290033c4666a1010621d9514173/roles/425d4b481c7846e0bc07e7e805d5b15f => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 03:20:45.426659 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15664297-923c-401d-be70-a006770f905e service nova] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:20:45.445949 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15664297-923c-401d-be70-a006770f905e service nova] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:20:45.578293 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ba20336-5e7c-4eed-a561-aae91260ed6e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76c5181fc7be45279ac00017bb4640b3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:20:45.579802 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba20336-5e7c-4eed-a561-aae91260ed6e admin admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:20:45.580119 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba20336-5e7c-4eed-a561-aae91260ed6e admin admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:20:45.580391 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba20336-5e7c-4eed-a561-aae91260ed6e admin admin] PATH_INFO: `/v3/roles` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:20:45.581048 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ba20336-5e7c-4eed-a561-aae91260ed6e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:20:45.582723 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ba20336-5e7c-4eed-a561-aae91260ed6e admin admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:20:45.616341 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 818/1633] 10.222.0.18 () {58 vars in 1122 bytes} [Tue Sep 13 03:20:45 2022] GET /identity/v3/roles => generated 1420 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 03:20:45.621275 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17d60ea1-c940-4f1f-9f64-04cc968c46c2 admin admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:20:45.635735 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17d60ea1-c940-4f1f-9f64-04cc968c46c2 admin admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:20:45.649354 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7f2a9cd-8e34-4492-b706-4fe06aaab47c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76c5181fc7be45279ac00017bb4640b3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:20:45.650059 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f2a9cd-8e34-4492-b706-4fe06aaab47c admin admin] REQUEST_METHOD: `PUT` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:20:45.650337 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f2a9cd-8e34-4492-b706-4fe06aaab47c admin admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:20:45.650603 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f2a9cd-8e34-4492-b706-4fe06aaab47c admin admin] PATH_INFO: `/v3/projects/4563823041354123a4dd7a3913fcd0e6/users/38473290033c4666a1010621d9514173/roles/425d4b481c7846e0bc07e7e805d5b15f` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:20:45.658262 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7f2a9cd-8e34-4492-b706-4fe06aaab47c admin admin] RBAC: Authorizing `identity:create_grant(project_id=4563823041354123a4dd7a3913fcd0e6, user_id=38473290033c4666a1010621d9514173, role_id=425d4b481c7846e0bc07e7e805d5b15f)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:20:45.661069 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7f2a9cd-8e34-4492-b706-4fe06aaab47c admin admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:20:45.708874 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9535361b-6b9f-4eec-a814-8a6dfea0eb66 None None] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:20:45.709154 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9535361b-6b9f-4eec-a814-8a6dfea0eb66 None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:20:45.709432 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9535361b-6b9f-4eec-a814-8a6dfea0eb66 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:20:45.712622 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 816/1634] 10.222.0.18 () {60 vars in 1483 bytes} [Tue Sep 13 03:20:45 2022] PUT /identity/v3/projects/4563823041354123a4dd7a3913fcd0e6/users/38473290033c4666a1010621d9514173/roles/425d4b481c7846e0bc07e7e805d5b15f => generated 0 bytes in 77 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 03:20:45.745059 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.auth.core [None req-9535361b-6b9f-4eec-a814-8a6dfea0eb66 None None] MFA Rules not processed for user `38473290033c4666a1010621d9514173`. Rule list: `[]` (Enabled: `True`). {{(pid=81400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 03:20:45.747392 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-9535361b-6b9f-4eec-a814-8a6dfea0eb66 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:20:45.810390 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:20:45.851371 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 819/1635] 10.222.0.18 () {58 vars in 958 bytes} [Tue Sep 13 03:20:45 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 03:20:45.859980 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7f2a9cd-8e34-4492-b706-4fe06aaab47c admin admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:20:45.882667 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b71c254e-4aad-46a4-b6cf-4b05a1f1aca5 None None] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:20:45.883604 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b71c254e-4aad-46a4-b6cf-4b05a1f1aca5 None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:20:45.885639 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b71c254e-4aad-46a4-b6cf-4b05a1f1aca5 None None] PATH_INFO: `/` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:20:45.891160 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 820/1636] 10.222.0.18 () {54 vars in 912 bytes} [Tue Sep 13 03:20:45 2022] GET /identity => generated 270 bytes in 12 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 03:20:45.898818 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea6acee9-95a4-4e08-ab86-0b5baf535fbf None None] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:20:45.908199 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea6acee9-95a4-4e08-ab86-0b5baf535fbf None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:20:45.908199 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea6acee9-95a4-4e08-ab86-0b5baf535fbf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:20:45.941890 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.auth.core [None req-ea6acee9-95a4-4e08-ab86-0b5baf535fbf None None] MFA Rules not processed for user `c7377f5bee4a49959182e1bd34bc97a4`. Rule list: `[]` (Enabled: `True`). {{(pid=81400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 03:20:45.943867 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-ea6acee9-95a4-4e08-ab86-0b5baf535fbf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:20:45.996813 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7f2a9cd-8e34-4492-b706-4fe06aaab47c admin admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:20:46.004224 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cafcf29-739e-4e29-86a1-f0ddb76affb6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '924ee3f608d34a18b0ddc4695f102a56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b9d94f6a68f7470d9494f112175620dc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:20:46.005505 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cafcf29-739e-4e29-86a1-f0ddb76affb6 service cinder] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:20:46.005791 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cafcf29-739e-4e29-86a1-f0ddb76affb6 service cinder] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:20:46.006157 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cafcf29-739e-4e29-86a1-f0ddb76affb6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:20:46.033390 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cafcf29-739e-4e29-86a1-f0ddb76affb6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:20:46.035099 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cafcf29-739e-4e29-86a1-f0ddb76affb6 service cinder] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:20:46.054920 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 817/1637] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Sep 13 03:20:45 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 196 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 03:20:46.112554 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 821/1638] 10.222.0.18 () {58 vars in 1009 bytes} [Tue Sep 13 03:20:45 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 216 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 03:20:46.133839 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cafcf29-739e-4e29-86a1-f0ddb76affb6 service cinder] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:20:46.148106 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cafcf29-739e-4e29-86a1-f0ddb76affb6 service cinder] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:20:46.155379 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8440510-0379-4c0e-b9f7-3a2fd6cd88fe service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '924ee3f608d34a18b0ddc4695f102a56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b9d94f6a68f7470d9494f112175620dc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:20:46.156265 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8440510-0379-4c0e-b9f7-3a2fd6cd88fe service cinder] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:20:46.156608 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8440510-0379-4c0e-b9f7-3a2fd6cd88fe service cinder] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:20:46.157077 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8440510-0379-4c0e-b9f7-3a2fd6cd88fe service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:20:46.181624 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8440510-0379-4c0e-b9f7-3a2fd6cd88fe service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:20:46.184516 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8440510-0379-4c0e-b9f7-3a2fd6cd88fe service cinder] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:20:46.220080 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 818/1639] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Sep 13 03:20:46 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 03:20:46.894774 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea6acee9-95a4-4e08-ab86-0b5baf535fbf None None] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:20:47.116763 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea6acee9-95a4-4e08-ab86-0b5baf535fbf None None] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:20:47.128965 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b130083f-9a53-4353-9885-886d0f83a825 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76c5181fc7be45279ac00017bb4640b3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:20:47.129837 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b130083f-9a53-4353-9885-886d0f83a825 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:20:47.130255 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b130083f-9a53-4353-9885-886d0f83a825 admin admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:20:47.131394 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b130083f-9a53-4353-9885-886d0f83a825 admin admin] PATH_INFO: `/v3/users/77e5e1a2032f408faaa53ba4cb2f5dc1` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:20:47.194151 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b130083f-9a53-4353-9885-886d0f83a825 admin admin] RBAC: Authorizing `identity:delete_user(user_id=77e5e1a2032f408faaa53ba4cb2f5dc1)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:20:47.194151 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: WARNING py.warnings [None req-b130083f-9a53-4353-9885-886d0f83a825 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 13 03:20:47.194151 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: warnings.warn(msg) Sep 13 03:20:47.194151 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: Sep 13 03:20:47.194151 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b130083f-9a53-4353-9885-886d0f83a825 admin admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:20:47.357000 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-b130083f-9a53-4353-9885-886d0f83a825 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '77e5e1a2032f408faaa53ba4cb2f5dc1'} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:20:47.379296 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-b130083f-9a53-4353-9885-886d0f83a825 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '77e5e1a2032f408faaa53ba4cb2f5dc1'} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:20:47.380021 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-b130083f-9a53-4353-9885-886d0f83a825 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '77e5e1a2032f408faaa53ba4cb2f5dc1'} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:20:47.421203 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-b130083f-9a53-4353-9885-886d0f83a825 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '77e5e1a2032f408faaa53ba4cb2f5dc1'} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:20:47.438981 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:20:47.452980 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-b130083f-9a53-4353-9885-886d0f83a825 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '77e5e1a2032f408faaa53ba4cb2f5dc1'} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:20:47.472583 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 822/1640] 10.222.0.18 () {58 vars in 1224 bytes} [Tue Sep 13 03:20:46 2022] DELETE /identity/v3/users/77e5e1a2032f408faaa53ba4cb2f5dc1 => generated 0 bytes in 575 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 03:20:47.473959 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb37954d-4ebc-4681-8cdc-2c599b1408a7 None None] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:20:47.475031 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb37954d-4ebc-4681-8cdc-2c599b1408a7 None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:20:47.475132 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb37954d-4ebc-4681-8cdc-2c599b1408a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:20:47.509644 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.auth.core [None req-cb37954d-4ebc-4681-8cdc-2c599b1408a7 None None] MFA Rules not processed for user `49f280ab086b4610855c4ffa378517d9`. Rule list: `[]` (Enabled: `True`). {{(pid=81399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 03:20:47.511096 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-cb37954d-4ebc-4681-8cdc-2c599b1408a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:20:47.534704 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:20:47.570443 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3613637-b124-43f4-88b4-aeaba0161add None None] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:20:47.571472 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3613637-b124-43f4-88b4-aeaba0161add None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:20:47.572067 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3613637-b124-43f4-88b4-aeaba0161add None None] PATH_INFO: `/` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:20:47.575002 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 823/1641] 10.222.0.18 () {54 vars in 962 bytes} [Tue Sep 13 03:20:47 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 03:20:47.584112 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f09bc252-1eea-43f3-b67f-395b749bcca8 None None] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:20:47.584660 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f09bc252-1eea-43f3-b67f-395b749bcca8 None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:20:47.585130 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f09bc252-1eea-43f3-b67f-395b749bcca8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:20:47.620481 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.auth.core [None req-f09bc252-1eea-43f3-b67f-395b749bcca8 None None] MFA Rules not processed for user `c7377f5bee4a49959182e1bd34bc97a4`. Rule list: `[]` (Enabled: `True`). {{(pid=81400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 03:20:47.622710 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-f09bc252-1eea-43f3-b67f-395b749bcca8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:20:47.656408 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 819/1642] 10.222.0.18 () {58 vars in 958 bytes} [Tue Sep 13 03:20:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 13 03:20:47.765211 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 824/1643] 10.222.0.18 () {58 vars in 1059 bytes} [Tue Sep 13 03:20:47 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 03:20:47.771683 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-627bdecf-5653-4124-bd53-81bfcfe9c5af None None] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:20:47.772079 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-627bdecf-5653-4124-bd53-81bfcfe9c5af None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:20:47.772428 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-627bdecf-5653-4124-bd53-81bfcfe9c5af None None] PATH_INFO: `/` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:20:47.773946 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 820/1644] 10.222.0.18 () {54 vars in 912 bytes} [Tue Sep 13 03:20:47 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 03:20:47.781623 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99138394-2d14-4a2f-bb40-995b213aedbb None None] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:20:47.781927 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99138394-2d14-4a2f-bb40-995b213aedbb None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:20:47.782202 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99138394-2d14-4a2f-bb40-995b213aedbb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:20:47.791428 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-627bdecf-5653-4124-bd53-81bfcfe9c5af None None] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:20:47.796725 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-627bdecf-5653-4124-bd53-81bfcfe9c5af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:20:47.846022 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.auth.core [None req-99138394-2d14-4a2f-bb40-995b213aedbb None None] MFA Rules not processed for user `c7377f5bee4a49959182e1bd34bc97a4`. Rule list: `[]` (Enabled: `True`). {{(pid=81400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 03:20:47.849838 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-99138394-2d14-4a2f-bb40-995b213aedbb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:20:47.884564 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 825/1645] 10.222.0.18 () {58 vars in 1009 bytes} [Tue Sep 13 03:20:47 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 03:20:47.898531 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99138394-2d14-4a2f-bb40-995b213aedbb None None] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:20:47.900118 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-99138394-2d14-4a2f-bb40-995b213aedbb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:20:47.984722 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99138394-2d14-4a2f-bb40-995b213aedbb None None] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:20:47.997847 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73de665b-0bea-45c7-bb43-c5a44889b9c1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '924ee3f608d34a18b0ddc4695f102a56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b9d94f6a68f7470d9494f112175620dc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:20:48.000341 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73de665b-0bea-45c7-bb43-c5a44889b9c1 service cinder] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:20:48.000622 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73de665b-0bea-45c7-bb43-c5a44889b9c1 service cinder] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:20:48.000903 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73de665b-0bea-45c7-bb43-c5a44889b9c1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:20:48.018557 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73de665b-0bea-45c7-bb43-c5a44889b9c1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:20:48.020685 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73de665b-0bea-45c7-bb43-c5a44889b9c1 service cinder] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:20:48.035483 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-627bdecf-5653-4124-bd53-81bfcfe9c5af None None] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:20:48.042999 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 826/1646] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Sep 13 03:20:47 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 144 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 13 03:20:48.055424 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21404808-23e6-4987-93e3-b92f47155bc5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdffbc851086441ebd01022d1e4c2609', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b9d94f6a68f7470d9494f112175620dc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:20:48.056076 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21404808-23e6-4987-93e3-b92f47155bc5 service neutron] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:20:48.056365 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21404808-23e6-4987-93e3-b92f47155bc5 service neutron] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:20:48.056643 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21404808-23e6-4987-93e3-b92f47155bc5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:20:48.080604 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21404808-23e6-4987-93e3-b92f47155bc5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:20:48.082259 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21404808-23e6-4987-93e3-b92f47155bc5 service neutron] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:20:48.108149 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 821/1647] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Sep 13 03:20:47 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 317 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 03:20:50.225398 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73de665b-0bea-45c7-bb43-c5a44889b9c1 service cinder] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:20:50.229734 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-73de665b-0bea-45c7-bb43-c5a44889b9c1 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:20:50.270463 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73de665b-0bea-45c7-bb43-c5a44889b9c1 service cinder] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:20:50.284311 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78b4e4d7-15a5-444a-8a9d-d0f5ecc5c46b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76c5181fc7be45279ac00017bb4640b3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:20:50.285261 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b4e4d7-15a5-444a-8a9d-d0f5ecc5c46b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:20:50.286900 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b4e4d7-15a5-444a-8a9d-d0f5ecc5c46b admin admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:20:50.287339 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b4e4d7-15a5-444a-8a9d-d0f5ecc5c46b admin admin] PATH_INFO: `/v3/projects/d018975f5fd14c808fc149f9f178b185` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:20:50.288683 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78b4e4d7-15a5-444a-8a9d-d0f5ecc5c46b admin admin] RBAC: Authorizing `identity:delete_project(project_id=d018975f5fd14c808fc149f9f178b185)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:20:50.297020 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78b4e4d7-15a5-444a-8a9d-d0f5ecc5c46b admin admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:20:50.386089 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-78b4e4d7-15a5-444a-8a9d-d0f5ecc5c46b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd018975f5fd14c808fc149f9f178b185', 'request_id': 'req-78b4e4d7-15a5-444a-8a9d-d0f5ecc5c46b'} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:20:50.395647 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-78b4e4d7-15a5-444a-8a9d-d0f5ecc5c46b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd018975f5fd14c808fc149f9f178b185', 'request_id': 'req-78b4e4d7-15a5-444a-8a9d-d0f5ecc5c46b'} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:20:50.406169 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-78b4e4d7-15a5-444a-8a9d-d0f5ecc5c46b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd018975f5fd14c808fc149f9f178b185', 'request_id': 'req-78b4e4d7-15a5-444a-8a9d-d0f5ecc5c46b'} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:20:50.436700 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-78b4e4d7-15a5-444a-8a9d-d0f5ecc5c46b admin admin] The token cache is being invalidate because project d018975f5fd14c808fc149f9f178b185 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81400) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 03:20:50.437333 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-78b4e4d7-15a5-444a-8a9d-d0f5ecc5c46b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:20:50.438074 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-78b4e4d7-15a5-444a-8a9d-d0f5ecc5c46b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:20:50.441520 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 827/1648] 10.222.0.18 () {58 vars in 1233 bytes} [Tue Sep 13 03:20:50 2022] DELETE /identity/v3/projects/d018975f5fd14c808fc149f9f178b185 => generated 0 bytes in 217 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 03:20:50.446207 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21404808-23e6-4987-93e3-b92f47155bc5 service neutron] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:20:50.457800 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21404808-23e6-4987-93e3-b92f47155bc5 service neutron] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:20:50.582348 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42336f87-5f94-46e0-a750-bd8da2e754cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76c5181fc7be45279ac00017bb4640b3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:20:50.583150 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42336f87-5f94-46e0-a750-bd8da2e754cc admin admin] REQUEST_METHOD: `DELETE` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:20:50.583465 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42336f87-5f94-46e0-a750-bd8da2e754cc admin admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:20:50.583799 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42336f87-5f94-46e0-a750-bd8da2e754cc admin admin] PATH_INFO: `/v3/users/38473290033c4666a1010621d9514173` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:20:50.585089 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42336f87-5f94-46e0-a750-bd8da2e754cc admin admin] RBAC: Authorizing `identity:delete_user(user_id=38473290033c4666a1010621d9514173)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:20:50.587403 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42336f87-5f94-46e0-a750-bd8da2e754cc admin admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:20:50.793716 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-42336f87-5f94-46e0-a750-bd8da2e754cc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '38473290033c4666a1010621d9514173'} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:20:50.812429 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-42336f87-5f94-46e0-a750-bd8da2e754cc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '38473290033c4666a1010621d9514173'} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:20:50.817124 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78b4e4d7-15a5-444a-8a9d-d0f5ecc5c46b admin admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:20:50.818310 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:20:50.819224 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-78b4e4d7-15a5-444a-8a9d-d0f5ecc5c46b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:20:50.839523 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:20:50.849002 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-42336f87-5f94-46e0-a750-bd8da2e754cc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '38473290033c4666a1010621d9514173'} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:20:50.850918 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78b4e4d7-15a5-444a-8a9d-d0f5ecc5c46b admin admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:20:50.851181 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-42336f87-5f94-46e0-a750-bd8da2e754cc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '38473290033c4666a1010621d9514173'} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:20:50.871885 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d7e1293-5cef-4a28-8e42-faa7bb9d2cac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76c5181fc7be45279ac00017bb4640b3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:20:50.872551 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d7e1293-5cef-4a28-8e42-faa7bb9d2cac admin admin] REQUEST_METHOD: `DELETE` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:20:50.872832 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d7e1293-5cef-4a28-8e42-faa7bb9d2cac admin admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:20:50.873097 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d7e1293-5cef-4a28-8e42-faa7bb9d2cac admin admin] PATH_INFO: `/v3/users/10ac3e249fb948d6a53010d5accd7097` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:20:50.875608 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d7e1293-5cef-4a28-8e42-faa7bb9d2cac admin admin] RBAC: Authorizing `identity:delete_user(user_id=10ac3e249fb948d6a53010d5accd7097)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:20:50.883145 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d7e1293-5cef-4a28-8e42-faa7bb9d2cac admin admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:20:50.888174 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-42336f87-5f94-46e0-a750-bd8da2e754cc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '38473290033c4666a1010621d9514173'} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:20:50.896839 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 822/1649] 10.222.0.18 () {58 vars in 1224 bytes} [Tue Sep 13 03:20:50 2022] DELETE /identity/v3/users/38473290033c4666a1010621d9514173 => generated 0 bytes in 451 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 03:20:51.006011 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-4d7e1293-5cef-4a28-8e42-faa7bb9d2cac admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '10ac3e249fb948d6a53010d5accd7097'} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:20:51.029997 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-4d7e1293-5cef-4a28-8e42-faa7bb9d2cac admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '10ac3e249fb948d6a53010d5accd7097'} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:20:51.030849 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-4d7e1293-5cef-4a28-8e42-faa7bb9d2cac admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '10ac3e249fb948d6a53010d5accd7097'} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:20:51.062753 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-4d7e1293-5cef-4a28-8e42-faa7bb9d2cac admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '10ac3e249fb948d6a53010d5accd7097'} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:20:51.082947 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-4d7e1293-5cef-4a28-8e42-faa7bb9d2cac admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '10ac3e249fb948d6a53010d5accd7097'} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:20:51.102570 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 828/1650] 10.222.0.18 () {58 vars in 1224 bytes} [Tue Sep 13 03:20:50 2022] DELETE /identity/v3/users/10ac3e249fb948d6a53010d5accd7097 => generated 0 bytes in 286 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 03:20:51.107809 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06b5e8c5-ce3a-4b74-a663-acb0f1d6abf4 None None] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:20:51.108134 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06b5e8c5-ce3a-4b74-a663-acb0f1d6abf4 None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:20:51.108357 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06b5e8c5-ce3a-4b74-a663-acb0f1d6abf4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:20:51.135542 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.auth.core [None req-06b5e8c5-ce3a-4b74-a663-acb0f1d6abf4 None None] MFA Rules not processed for user `49f280ab086b4610855c4ffa378517d9`. Rule list: `[]` (Enabled: `True`). {{(pid=81399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 03:20:51.135542 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-06b5e8c5-ce3a-4b74-a663-acb0f1d6abf4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:20:51.285349 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 823/1651] 10.222.0.18 () {58 vars in 958 bytes} [Tue Sep 13 03:20:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 03:20:51.306742 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d7e1293-5cef-4a28-8e42-faa7bb9d2cac admin admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:20:51.308200 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-4d7e1293-5cef-4a28-8e42-faa7bb9d2cac admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:20:51.494535 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d7e1293-5cef-4a28-8e42-faa7bb9d2cac admin admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:20:51.502729 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78002750-c38f-4c6e-8975-2242682ecd07 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdffbc851086441ebd01022d1e4c2609', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b9d94f6a68f7470d9494f112175620dc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:20:51.503686 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78002750-c38f-4c6e-8975-2242682ecd07 service neutron] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:20:51.504104 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78002750-c38f-4c6e-8975-2242682ecd07 service neutron] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:20:51.504524 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78002750-c38f-4c6e-8975-2242682ecd07 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:20:51.531553 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78002750-c38f-4c6e-8975-2242682ecd07 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:20:51.534192 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78002750-c38f-4c6e-8975-2242682ecd07 service neutron] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:20:51.571235 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 829/1652] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Sep 13 03:20:51 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 265 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 03:20:52.155710 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06b5e8c5-ce3a-4b74-a663-acb0f1d6abf4 None None] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:20:52.157719 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-06b5e8c5-ce3a-4b74-a663-acb0f1d6abf4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:20:52.180791 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06b5e8c5-ce3a-4b74-a663-acb0f1d6abf4 None None] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:20:52.186273 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed88b679-7020-4ac5-9aa0-251c081d18f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76c5181fc7be45279ac00017bb4640b3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:20:52.187302 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed88b679-7020-4ac5-9aa0-251c081d18f0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:20:52.187779 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed88b679-7020-4ac5-9aa0-251c081d18f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:20:52.188227 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed88b679-7020-4ac5-9aa0-251c081d18f0 admin admin] PATH_INFO: `/v3/projects/4563823041354123a4dd7a3913fcd0e6` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:20:52.189630 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed88b679-7020-4ac5-9aa0-251c081d18f0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4563823041354123a4dd7a3913fcd0e6)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:20:52.190890 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: WARNING py.warnings [None req-ed88b679-7020-4ac5-9aa0-251c081d18f0 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 13 03:20:52.190890 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: warnings.warn(msg) Sep 13 03:20:52.190890 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: Sep 13 03:20:52.192464 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed88b679-7020-4ac5-9aa0-251c081d18f0 admin admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:20:52.278514 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-ed88b679-7020-4ac5-9aa0-251c081d18f0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4563823041354123a4dd7a3913fcd0e6', 'request_id': 'req-ed88b679-7020-4ac5-9aa0-251c081d18f0'} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:20:52.287835 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-ed88b679-7020-4ac5-9aa0-251c081d18f0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4563823041354123a4dd7a3913fcd0e6', 'request_id': 'req-ed88b679-7020-4ac5-9aa0-251c081d18f0'} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:20:52.306723 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-ed88b679-7020-4ac5-9aa0-251c081d18f0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4563823041354123a4dd7a3913fcd0e6', 'request_id': 'req-ed88b679-7020-4ac5-9aa0-251c081d18f0'} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:20:52.336998 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-ed88b679-7020-4ac5-9aa0-251c081d18f0 admin admin] The token cache is being invalidate because project 4563823041354123a4dd7a3913fcd0e6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81399) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 03:20:52.337303 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-ed88b679-7020-4ac5-9aa0-251c081d18f0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:20:52.338491 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-ed88b679-7020-4ac5-9aa0-251c081d18f0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:20:52.353864 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 824/1653] 10.222.0.18 () {58 vars in 1233 bytes} [Tue Sep 13 03:20:52 2022] DELETE /identity/v3/projects/4563823041354123a4dd7a3913fcd0e6 => generated 0 bytes in 196 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 03:20:52.355811 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11895865-551d-4838-8adc-8a0bf44d2708 None None] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:20:52.356259 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11895865-551d-4838-8adc-8a0bf44d2708 None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:20:52.356360 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11895865-551d-4838-8adc-8a0bf44d2708 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:20:52.379291 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.auth.core [None req-11895865-551d-4838-8adc-8a0bf44d2708 None None] MFA Rules not processed for user `49f280ab086b4610855c4ffa378517d9`. Rule list: `[]` (Enabled: `True`). {{(pid=81400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 03:20:52.381165 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-11895865-551d-4838-8adc-8a0bf44d2708 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:20:52.508523 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 830/1654] 10.222.0.18 () {58 vars in 958 bytes} [Tue Sep 13 03:20:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 03:20:52.517336 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c79b6a10-9d12-4d49-b5f5-97abb3f048d6 None None] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:20:52.517924 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c79b6a10-9d12-4d49-b5f5-97abb3f048d6 None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:20:52.518498 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c79b6a10-9d12-4d49-b5f5-97abb3f048d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:20:52.579445 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.auth.core [None req-c79b6a10-9d12-4d49-b5f5-97abb3f048d6 None None] MFA Rules not processed for user `49f280ab086b4610855c4ffa378517d9`. Rule list: `[]` (Enabled: `True`). {{(pid=81399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 03:20:52.581609 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-c79b6a10-9d12-4d49-b5f5-97abb3f048d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:20:52.596195 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 825/1655] 10.222.0.18 () {58 vars in 958 bytes} [Tue Sep 13 03:20:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 13 03:20:52.601607 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11895865-551d-4838-8adc-8a0bf44d2708 None None] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:20:52.618489 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11895865-551d-4838-8adc-8a0bf44d2708 None None] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:20:52.623943 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90877886-bbb9-48c3-9fb6-fdbe296dc919 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76c5181fc7be45279ac00017bb4640b3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:20:52.625892 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90877886-bbb9-48c3-9fb6-fdbe296dc919 admin admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:20:52.627219 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90877886-bbb9-48c3-9fb6-fdbe296dc919 admin admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:20:52.627702 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90877886-bbb9-48c3-9fb6-fdbe296dc919 admin admin] PATH_INFO: `/v3/domains` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:20:52.628808 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90877886-bbb9-48c3-9fb6-fdbe296dc919 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 03:20:52.629281 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90877886-bbb9-48c3-9fb6-fdbe296dc919 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:20:52.631718 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90877886-bbb9-48c3-9fb6-fdbe296dc919 admin admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:20:52.658343 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 831/1656] 10.222.0.18 () {58 vars in 1153 bytes} [Tue Sep 13 03:20:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 03:20:52.664439 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c79b6a10-9d12-4d49-b5f5-97abb3f048d6 None None] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:20:52.677456 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c79b6a10-9d12-4d49-b5f5-97abb3f048d6 None None] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:20:52.682948 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d4cd148-1d4d-4f81-b0e2-e70c2866cfad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76c5181fc7be45279ac00017bb4640b3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:20:52.683909 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d4cd148-1d4d-4f81-b0e2-e70c2866cfad admin admin] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:20:52.684198 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d4cd148-1d4d-4f81-b0e2-e70c2866cfad admin admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:20:52.684475 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d4cd148-1d4d-4f81-b0e2-e70c2866cfad admin admin] PATH_INFO: `/v3/projects` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:20:52.685130 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d4cd148-1d4d-4f81-b0e2-e70c2866cfad admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:20:52.687572 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d4cd148-1d4d-4f81-b0e2-e70c2866cfad admin admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:20:52.761921 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90877886-bbb9-48c3-9fb6-fdbe296dc919 admin admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:20:52.762063 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 826/1657] 10.222.0.18 () {60 vars in 1153 bytes} [Tue Sep 13 03:20:52 2022] POST /identity/v3/projects => generated 367 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 03:20:52.778984 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90877886-bbb9-48c3-9fb6-fdbe296dc919 admin admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:20:52.791292 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:20:52.916320 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d56931a3-39f3-4f94-a082-9a76d1ada644 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76c5181fc7be45279ac00017bb4640b3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:20:52.917109 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d56931a3-39f3-4f94-a082-9a76d1ada644 admin admin] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:20:52.917489 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d56931a3-39f3-4f94-a082-9a76d1ada644 admin admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:20:52.917861 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d56931a3-39f3-4f94-a082-9a76d1ada644 admin admin] PATH_INFO: `/v3/users` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:20:52.918625 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d56931a3-39f3-4f94-a082-9a76d1ada644 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:20:52.921010 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d56931a3-39f3-4f94-a082-9a76d1ada644 admin admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:20:52.975152 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 832/1658] 10.222.0.18 () {60 vars in 1144 bytes} [Tue Sep 13 03:20:52 2022] POST /identity/v3/users => generated 284 bytes in 211 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 03:20:52.977648 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d4cd148-1d4d-4f81-b0e2-e70c2866cfad admin admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:20:52.998313 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d4cd148-1d4d-4f81-b0e2-e70c2866cfad admin admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:20:53.029741 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-251ab8d0-af0f-47fb-9c9a-9978f15ed62a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76c5181fc7be45279ac00017bb4640b3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:20:53.038302 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-251ab8d0-af0f-47fb-9c9a-9978f15ed62a admin admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:20:53.038604 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-251ab8d0-af0f-47fb-9c9a-9978f15ed62a admin admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:20:53.038925 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-251ab8d0-af0f-47fb-9c9a-9978f15ed62a admin admin] PATH_INFO: `/v3/roles` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:20:53.039634 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-251ab8d0-af0f-47fb-9c9a-9978f15ed62a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:20:53.050824 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-251ab8d0-af0f-47fb-9c9a-9978f15ed62a admin admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:20:53.101860 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d56931a3-39f3-4f94-a082-9a76d1ada644 admin admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:20:53.104832 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 827/1659] 10.222.0.18 () {58 vars in 1122 bytes} [Tue Sep 13 03:20:52 2022] GET /identity/v3/roles => generated 1420 bytes in 128 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 03:20:53.116344 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d56931a3-39f3-4f94-a082-9a76d1ada644 admin admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:20:53.123396 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a592da5-3314-4c03-a28c-439ba5862227 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76c5181fc7be45279ac00017bb4640b3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:20:53.124084 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a592da5-3314-4c03-a28c-439ba5862227 admin admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:20:53.124434 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a592da5-3314-4c03-a28c-439ba5862227 admin admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:20:53.124777 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a592da5-3314-4c03-a28c-439ba5862227 admin admin] PATH_INFO: `/v3/roles` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:20:53.125501 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a592da5-3314-4c03-a28c-439ba5862227 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:20:53.127264 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a592da5-3314-4c03-a28c-439ba5862227 admin admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:20:53.145975 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 833/1660] 10.222.0.18 () {58 vars in 1122 bytes} [Tue Sep 13 03:20:53 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 03:20:53.150096 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-251ab8d0-af0f-47fb-9c9a-9978f15ed62a admin admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:20:53.183665 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-251ab8d0-af0f-47fb-9c9a-9978f15ed62a admin admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:20:53.203480 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c206309f-5024-4b56-9f24-1e585ab00ac8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76c5181fc7be45279ac00017bb4640b3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:20:53.203625 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c206309f-5024-4b56-9f24-1e585ab00ac8 admin admin] REQUEST_METHOD: `PUT` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:20:53.204063 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c206309f-5024-4b56-9f24-1e585ab00ac8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:20:53.205091 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c206309f-5024-4b56-9f24-1e585ab00ac8 admin admin] PATH_INFO: `/v3/projects/15143b2e72aa40f78184abf09d8bd9c1/users/b3313cfa155542c3afc5cd2c56af6dc7/roles/1eb7a5b5203945c29e6d22d653f47847` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:20:53.234018 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c206309f-5024-4b56-9f24-1e585ab00ac8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=15143b2e72aa40f78184abf09d8bd9c1, user_id=b3313cfa155542c3afc5cd2c56af6dc7, role_id=1eb7a5b5203945c29e6d22d653f47847)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:20:53.238777 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c206309f-5024-4b56-9f24-1e585ab00ac8 admin admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:20:53.246975 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:DELETE=9,keystone:SELECT=88,keystone:INSERT=3 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:20:53.277869 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 828/1661] 10.222.0.18 () {60 vars in 1483 bytes} [Tue Sep 13 03:20:53 2022] PUT /identity/v3/projects/15143b2e72aa40f78184abf09d8bd9c1/users/b3313cfa155542c3afc5cd2c56af6dc7/roles/1eb7a5b5203945c29e6d22d653f47847 => generated 0 bytes in 120 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 03:20:53.280047 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ec4d3e-da9a-4121-bd1b-0b19dfdcf3c0 None None] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:20:53.280368 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ec4d3e-da9a-4121-bd1b-0b19dfdcf3c0 None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:20:53.280641 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ec4d3e-da9a-4121-bd1b-0b19dfdcf3c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:20:53.342252 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.auth.core [None req-65ec4d3e-da9a-4121-bd1b-0b19dfdcf3c0 None None] MFA Rules not processed for user `b3313cfa155542c3afc5cd2c56af6dc7`. Rule list: `[]` (Enabled: `True`). {{(pid=81400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 03:20:53.348150 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-65ec4d3e-da9a-4121-bd1b-0b19dfdcf3c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:20:53.521584 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 834/1662] 10.222.0.18 () {58 vars in 958 bytes} [Tue Sep 13 03:20:53 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 239 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 03:20:53.523921 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c206309f-5024-4b56-9f24-1e585ab00ac8 admin admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:20:53.688937 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c206309f-5024-4b56-9f24-1e585ab00ac8 admin admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:20:53.707591 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19058a0e-bfc6-4928-9b71-ff1af705a89a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76c5181fc7be45279ac00017bb4640b3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:20:53.708096 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19058a0e-bfc6-4928-9b71-ff1af705a89a admin admin] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:20:53.708407 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19058a0e-bfc6-4928-9b71-ff1af705a89a admin admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:20:53.708680 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19058a0e-bfc6-4928-9b71-ff1af705a89a admin admin] PATH_INFO: `/v3/projects` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:20:53.709331 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19058a0e-bfc6-4928-9b71-ff1af705a89a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:20:53.711455 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19058a0e-bfc6-4928-9b71-ff1af705a89a admin admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:20:53.762906 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 829/1663] 10.222.0.18 () {60 vars in 1153 bytes} [Tue Sep 13 03:20:53 2022] POST /identity/v3/projects => generated 367 bytes in 239 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 03:20:53.769453 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65ec4d3e-da9a-4121-bd1b-0b19dfdcf3c0 None None] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:20:53.771228 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19058a0e-bfc6-4928-9b71-ff1af705a89a admin admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:20:53.773198 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-65ec4d3e-da9a-4121-bd1b-0b19dfdcf3c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:20:53.804066 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65ec4d3e-da9a-4121-bd1b-0b19dfdcf3c0 None None] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:20:53.936003 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19058a0e-bfc6-4928-9b71-ff1af705a89a admin admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:20:53.942980 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33cad174-654c-4469-8bc3-0519c13a0d9c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76c5181fc7be45279ac00017bb4640b3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:20:53.943808 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33cad174-654c-4469-8bc3-0519c13a0d9c admin admin] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:20:53.944243 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33cad174-654c-4469-8bc3-0519c13a0d9c admin admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:20:53.944666 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33cad174-654c-4469-8bc3-0519c13a0d9c admin admin] PATH_INFO: `/v3/users` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:20:53.945463 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33cad174-654c-4469-8bc3-0519c13a0d9c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:20:53.947690 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33cad174-654c-4469-8bc3-0519c13a0d9c admin admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:20:53.982480 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 830/1664] 10.222.0.18 () {60 vars in 1144 bytes} [Tue Sep 13 03:20:53 2022] POST /identity/v3/users => generated 290 bytes in 212 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 03:20:53.989713 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30206dc0-057f-4ddb-bb32-67f7447b809e None None] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:20:53.989905 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e009412-3ce7-4407-b7ec-7c8e408f0065 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76c5181fc7be45279ac00017bb4640b3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:20:53.990577 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e009412-3ce7-4407-b7ec-7c8e408f0065 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:20:53.990883 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e009412-3ce7-4407-b7ec-7c8e408f0065 admin admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:20:53.991154 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e009412-3ce7-4407-b7ec-7c8e408f0065 admin admin] PATH_INFO: `/v3/projects/9d8c2872e912420696a20ed363471888` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:20:53.991863 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30206dc0-057f-4ddb-bb32-67f7447b809e None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:20:53.992379 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e009412-3ce7-4407-b7ec-7c8e408f0065 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9d8c2872e912420696a20ed363471888)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:20:53.993866 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30206dc0-057f-4ddb-bb32-67f7447b809e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:20:53.994451 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e009412-3ce7-4407-b7ec-7c8e408f0065 admin admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:20:54.040070 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:20:54.051770 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.auth.core [None req-30206dc0-057f-4ddb-bb32-67f7447b809e None None] MFA Rules not processed for user `83bd9ab163ba4229bcfd810345430564`. Rule list: `[]` (Enabled: `True`). {{(pid=81399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 03:20:54.053508 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-30206dc0-057f-4ddb-bb32-67f7447b809e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:20:54.156469 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-5e009412-3ce7-4407-b7ec-7c8e408f0065 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9d8c2872e912420696a20ed363471888', 'request_id': 'req-5e009412-3ce7-4407-b7ec-7c8e408f0065'} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:20:54.169318 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-5e009412-3ce7-4407-b7ec-7c8e408f0065 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9d8c2872e912420696a20ed363471888', 'request_id': 'req-5e009412-3ce7-4407-b7ec-7c8e408f0065'} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:20:54.188445 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-5e009412-3ce7-4407-b7ec-7c8e408f0065 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9d8c2872e912420696a20ed363471888', 'request_id': 'req-5e009412-3ce7-4407-b7ec-7c8e408f0065'} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:20:54.203645 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:20:54.216696 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 831/1665] 10.222.0.18 () {58 vars in 958 bytes} [Tue Sep 13 03:20:53 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 228 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 03:20:54.217938 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30206dc0-057f-4ddb-bb32-67f7447b809e None None] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:20:54.224863 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-5e009412-3ce7-4407-b7ec-7c8e408f0065 admin admin] The token cache is being invalidate because project 9d8c2872e912420696a20ed363471888 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81400) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 03:20:54.225383 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-5e009412-3ce7-4407-b7ec-7c8e408f0065 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:20:54.227040 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-5e009412-3ce7-4407-b7ec-7c8e408f0065 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:20:54.229211 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 835/1666] 10.222.0.18 () {58 vars in 1233 bytes} [Tue Sep 13 03:20:53 2022] DELETE /identity/v3/projects/9d8c2872e912420696a20ed363471888 => generated 0 bytes in 463 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 03:20:54.241827 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30206dc0-057f-4ddb-bb32-67f7447b809e None None] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:20:54.242177 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e009412-3ce7-4407-b7ec-7c8e408f0065 admin admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:20:54.245139 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-5e009412-3ce7-4407-b7ec-7c8e408f0065 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:20:54.392094 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-953a4168-b5df-437d-9efd-7206bb4e8254 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76c5181fc7be45279ac00017bb4640b3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:20:54.394190 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-953a4168-b5df-437d-9efd-7206bb4e8254 admin admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:20:54.394784 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-953a4168-b5df-437d-9efd-7206bb4e8254 admin admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:20:54.395264 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-953a4168-b5df-437d-9efd-7206bb4e8254 admin admin] PATH_INFO: `/v3/roles` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:20:54.396793 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-953a4168-b5df-437d-9efd-7206bb4e8254 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:20:54.398878 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-953a4168-b5df-437d-9efd-7206bb4e8254 admin admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:20:54.427198 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 832/1667] 10.222.0.18 () {58 vars in 1122 bytes} [Tue Sep 13 03:20:54 2022] GET /identity/v3/roles => generated 1420 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 03:20:54.428929 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-247d9aef-c77b-442b-9ad6-926490819d6c None None] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:20:54.429943 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-247d9aef-c77b-442b-9ad6-926490819d6c None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:20:54.431647 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-247d9aef-c77b-442b-9ad6-926490819d6c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:20:54.454524 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e009412-3ce7-4407-b7ec-7c8e408f0065 admin admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:20:54.473312 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00255e62-e59b-4f1a-9ab3-b165018a3334 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7377f5bee4a49959182e1bd34bc97a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b9d94f6a68f7470d9494f112175620dc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:20:54.474629 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00255e62-e59b-4f1a-9ab3-b165018a3334 service nova] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:20:54.475709 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00255e62-e59b-4f1a-9ab3-b165018a3334 service nova] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:20:54.476166 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00255e62-e59b-4f1a-9ab3-b165018a3334 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:20:54.479991 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.auth.core [None req-247d9aef-c77b-442b-9ad6-926490819d6c None None] MFA Rules not processed for user `49f280ab086b4610855c4ffa378517d9`. Rule list: `[]` (Enabled: `True`). {{(pid=81399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 03:20:54.481211 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-00255e62-e59b-4f1a-9ab3-b165018a3334 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:20:54.485782 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-247d9aef-c77b-442b-9ad6-926490819d6c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:20:54.512514 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 833/1668] 10.222.0.18 () {58 vars in 958 bytes} [Tue Sep 13 03:20:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 03:20:54.513733 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-247d9aef-c77b-442b-9ad6-926490819d6c None None] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:20:54.517287 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-247d9aef-c77b-442b-9ad6-926490819d6c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:20:54.547425 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-247d9aef-c77b-442b-9ad6-926490819d6c None None] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:20:54.558554 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fdce878-dcdc-4b97-95a4-9a3d6bf081a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76c5181fc7be45279ac00017bb4640b3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:20:54.559590 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fdce878-dcdc-4b97-95a4-9a3d6bf081a7 admin admin] REQUEST_METHOD: `PUT` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:20:54.560056 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fdce878-dcdc-4b97-95a4-9a3d6bf081a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:20:54.560504 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fdce878-dcdc-4b97-95a4-9a3d6bf081a7 admin admin] PATH_INFO: `/v3/projects/4397caa381624321bedbd87854af542e/users/dceabdb575394818b6504bc03dc9958f/roles/425d4b481c7846e0bc07e7e805d5b15f` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:20:54.633710 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fdce878-dcdc-4b97-95a4-9a3d6bf081a7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4397caa381624321bedbd87854af542e, user_id=dceabdb575394818b6504bc03dc9958f, role_id=425d4b481c7846e0bc07e7e805d5b15f)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:20:54.637104 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fdce878-dcdc-4b97-95a4-9a3d6bf081a7 admin admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:20:54.666469 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 834/1669] 10.222.0.18 () {60 vars in 1483 bytes} [Tue Sep 13 03:20:54 2022] PUT /identity/v3/projects/4397caa381624321bedbd87854af542e/users/dceabdb575394818b6504bc03dc9958f/roles/425d4b481c7846e0bc07e7e805d5b15f => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 03:20:54.669543 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf4ef1d-09db-48c2-ad44-7d2472b5cd5c None None] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:20:54.669840 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf4ef1d-09db-48c2-ad44-7d2472b5cd5c None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:20:54.670366 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf4ef1d-09db-48c2-ad44-7d2472b5cd5c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:20:54.688303 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00255e62-e59b-4f1a-9ab3-b165018a3334 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:20:54.695055 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00255e62-e59b-4f1a-9ab3-b165018a3334 service nova] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:20:54.714288 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.auth.core [None req-aaf4ef1d-09db-48c2-ad44-7d2472b5cd5c None None] MFA Rules not processed for user `49f280ab086b4610855c4ffa378517d9`. Rule list: `[]` (Enabled: `True`). {{(pid=81399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 03:20:54.717748 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-aaf4ef1d-09db-48c2-ad44-7d2472b5cd5c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:20:54.722039 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 836/1670] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Sep 13 03:20:54 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 481 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 03:20:54.723164 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00255e62-e59b-4f1a-9ab3-b165018a3334 service nova] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:20:54.752739 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00255e62-e59b-4f1a-9ab3-b165018a3334 service nova] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:20:54.886781 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:20:54.955162 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 835/1671] 10.222.0.18 () {58 vars in 958 bytes} [Tue Sep 13 03:20:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 285 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 03:20:54.958716 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaf4ef1d-09db-48c2-ad44-7d2472b5cd5c None None] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:20:54.973325 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-010d384d-4123-4ab9-8f09-1e825badf97b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76c5181fc7be45279ac00017bb4640b3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:20:54.974497 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-010d384d-4123-4ab9-8f09-1e825badf97b admin admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:20:54.974816 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-010d384d-4123-4ab9-8f09-1e825badf97b admin admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:20:54.975265 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-010d384d-4123-4ab9-8f09-1e825badf97b admin admin] PATH_INFO: `/v3/roles` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:20:54.981405 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-010d384d-4123-4ab9-8f09-1e825badf97b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:20:54.986896 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-010d384d-4123-4ab9-8f09-1e825badf97b admin admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:20:54.992536 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaf4ef1d-09db-48c2-ad44-7d2472b5cd5c None None] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:20:55.003586 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d711743-7a84-400a-8d26-90d80d64bcd0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76c5181fc7be45279ac00017bb4640b3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:20:55.004445 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d711743-7a84-400a-8d26-90d80d64bcd0 admin admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:20:55.004607 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d711743-7a84-400a-8d26-90d80d64bcd0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:20:55.004964 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d711743-7a84-400a-8d26-90d80d64bcd0 admin admin] PATH_INFO: `/v3/domains` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:20:55.005740 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d711743-7a84-400a-8d26-90d80d64bcd0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 03:20:55.006304 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d711743-7a84-400a-8d26-90d80d64bcd0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:20:55.008168 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d711743-7a84-400a-8d26-90d80d64bcd0 admin admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:20:55.044639 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 837/1672] 10.222.0.18 () {58 vars in 1122 bytes} [Tue Sep 13 03:20:54 2022] GET /identity/v3/roles => generated 1420 bytes in 322 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 03:20:55.048840 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-010d384d-4123-4ab9-8f09-1e825badf97b admin admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:20:55.078229 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 836/1673] 10.222.0.18 () {58 vars in 1153 bytes} [Tue Sep 13 03:20:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 122 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 03:20:55.079386 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d711743-7a84-400a-8d26-90d80d64bcd0 admin admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:20:55.094555 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-010d384d-4123-4ab9-8f09-1e825badf97b admin admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:20:55.110027 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d711743-7a84-400a-8d26-90d80d64bcd0 admin admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:20:55.113104 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e857059e-c7d0-4485-9637-c6f91326a90e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76c5181fc7be45279ac00017bb4640b3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:20:55.113791 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e857059e-c7d0-4485-9637-c6f91326a90e admin admin] REQUEST_METHOD: `PUT` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:20:55.114060 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e857059e-c7d0-4485-9637-c6f91326a90e admin admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:20:55.114324 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e857059e-c7d0-4485-9637-c6f91326a90e admin admin] PATH_INFO: `/v3/projects/4397caa381624321bedbd87854af542e/users/dceabdb575394818b6504bc03dc9958f/roles/425d4b481c7846e0bc07e7e805d5b15f` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:20:55.121211 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e857059e-c7d0-4485-9637-c6f91326a90e admin admin] RBAC: Authorizing `identity:create_grant(project_id=4397caa381624321bedbd87854af542e, user_id=dceabdb575394818b6504bc03dc9958f, role_id=425d4b481c7846e0bc07e7e805d5b15f)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:20:55.128560 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e857059e-c7d0-4485-9637-c6f91326a90e admin admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:20:55.131424 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0181a276-4172-4d6c-85bf-316cdcd0e210 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76c5181fc7be45279ac00017bb4640b3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:20:55.132050 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0181a276-4172-4d6c-85bf-316cdcd0e210 admin admin] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:20:55.132372 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0181a276-4172-4d6c-85bf-316cdcd0e210 admin admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:20:55.132579 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0181a276-4172-4d6c-85bf-316cdcd0e210 admin admin] PATH_INFO: `/v3/projects` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:20:55.133360 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0181a276-4172-4d6c-85bf-316cdcd0e210 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:20:55.142318 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0181a276-4172-4d6c-85bf-316cdcd0e210 admin admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:20:55.173692 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 838/1674] 10.222.0.18 () {60 vars in 1483 bytes} [Tue Sep 13 03:20:55 2022] PUT /identity/v3/projects/4397caa381624321bedbd87854af542e/users/dceabdb575394818b6504bc03dc9958f/roles/425d4b481c7846e0bc07e7e805d5b15f => generated 0 bytes in 128 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 03:20:55.190040 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f21cc512-3623-4e3e-9c9d-4529321870b9 None None] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:20:55.190536 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f21cc512-3623-4e3e-9c9d-4529321870b9 None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:20:55.191029 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f21cc512-3623-4e3e-9c9d-4529321870b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:20:55.237435 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 837/1675] 10.222.0.18 () {60 vars in 1153 bytes} [Tue Sep 13 03:20:55 2022] POST /identity/v3/projects => generated 375 bytes in 158 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 03:20:55.244353 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0181a276-4172-4d6c-85bf-316cdcd0e210 admin admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:20:55.283543 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.auth.core [None req-f21cc512-3623-4e3e-9c9d-4529321870b9 None None] MFA Rules not processed for user `dceabdb575394818b6504bc03dc9958f`. Rule list: `[]` (Enabled: `True`). {{(pid=81400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 03:20:55.285135 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-f21cc512-3623-4e3e-9c9d-4529321870b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:20:55.373924 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:20:55.472816 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0181a276-4172-4d6c-85bf-316cdcd0e210 admin admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:20:55.492783 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7cd8c2e-954e-4cd4-ab9a-d5abb8573f02 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76c5181fc7be45279ac00017bb4640b3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:20:55.493096 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7cd8c2e-954e-4cd4-ab9a-d5abb8573f02 admin admin] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:20:55.493346 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7cd8c2e-954e-4cd4-ab9a-d5abb8573f02 admin admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:20:55.493606 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7cd8c2e-954e-4cd4-ab9a-d5abb8573f02 admin admin] PATH_INFO: `/v3/users` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:20:55.494236 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7cd8c2e-954e-4cd4-ab9a-d5abb8573f02 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:20:55.496182 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7cd8c2e-954e-4cd4-ab9a-d5abb8573f02 admin admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:20:55.554968 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 839/1676] 10.222.0.18 () {58 vars in 958 bytes} [Tue Sep 13 03:20:55 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 366 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 03:20:55.572764 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f21cc512-3623-4e3e-9c9d-4529321870b9 None None] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:20:55.580247 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-f21cc512-3623-4e3e-9c9d-4529321870b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:20:55.581353 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 838/1677] 10.222.0.18 () {60 vars in 1144 bytes} [Tue Sep 13 03:20:55 2022] POST /identity/v3/users => generated 288 bytes in 337 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 03:20:55.585349 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7cd8c2e-954e-4cd4-ab9a-d5abb8573f02 admin admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:20:55.616239 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7cd8c2e-954e-4cd4-ab9a-d5abb8573f02 admin admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:20:55.635376 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-613f5087-6cbe-4ed9-8cdf-6246e07e956d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76c5181fc7be45279ac00017bb4640b3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:20:55.636071 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-613f5087-6cbe-4ed9-8cdf-6246e07e956d admin admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:20:55.636345 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-613f5087-6cbe-4ed9-8cdf-6246e07e956d admin admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:20:55.636614 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-613f5087-6cbe-4ed9-8cdf-6246e07e956d admin admin] PATH_INFO: `/v3/roles` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:20:55.637280 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-613f5087-6cbe-4ed9-8cdf-6246e07e956d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:20:55.639048 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-613f5087-6cbe-4ed9-8cdf-6246e07e956d admin admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:20:55.678436 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 839/1678] 10.222.0.18 () {58 vars in 1122 bytes} [Tue Sep 13 03:20:55 2022] GET /identity/v3/roles => generated 1420 bytes in 93 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 03:20:55.679392 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-613f5087-6cbe-4ed9-8cdf-6246e07e956d admin admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:20:55.710532 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-613f5087-6cbe-4ed9-8cdf-6246e07e956d admin admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:20:55.726930 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c9b0117-eeb1-4b26-8be0-ef0c70bc740c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76c5181fc7be45279ac00017bb4640b3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:20:55.731169 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c9b0117-eeb1-4b26-8be0-ef0c70bc740c admin admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:20:55.731459 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c9b0117-eeb1-4b26-8be0-ef0c70bc740c admin admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:20:55.731763 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c9b0117-eeb1-4b26-8be0-ef0c70bc740c admin admin] PATH_INFO: `/v3/roles` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:20:55.732431 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c9b0117-eeb1-4b26-8be0-ef0c70bc740c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:20:55.734043 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c9b0117-eeb1-4b26-8be0-ef0c70bc740c admin admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:20:55.763284 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f21cc512-3623-4e3e-9c9d-4529321870b9 None None] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:20:55.771737 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5a78f55-e45a-4b3f-be62-9648976da5e1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '924ee3f608d34a18b0ddc4695f102a56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b9d94f6a68f7470d9494f112175620dc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:20:55.774660 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5a78f55-e45a-4b3f-be62-9648976da5e1 service cinder] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:20:55.774960 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5a78f55-e45a-4b3f-be62-9648976da5e1 service cinder] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:20:55.775958 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5a78f55-e45a-4b3f-be62-9648976da5e1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:20:55.776736 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 840/1679] 10.222.0.18 () {58 vars in 1122 bytes} [Tue Sep 13 03:20:55 2022] GET /identity/v3/roles => generated 1420 bytes in 98 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 03:20:55.780480 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c9b0117-eeb1-4b26-8be0-ef0c70bc740c admin admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:20:55.807749 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5a78f55-e45a-4b3f-be62-9648976da5e1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:20:55.809228 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5a78f55-e45a-4b3f-be62-9648976da5e1 service cinder] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:20:55.811837 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c9b0117-eeb1-4b26-8be0-ef0c70bc740c admin admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:20:55.833027 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c029e03-c8f5-4cf8-98e4-7736df23ca51 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76c5181fc7be45279ac00017bb4640b3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:20:55.833760 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c029e03-c8f5-4cf8-98e4-7736df23ca51 admin admin] REQUEST_METHOD: `PUT` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:20:55.834036 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c029e03-c8f5-4cf8-98e4-7736df23ca51 admin admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:20:55.834306 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c029e03-c8f5-4cf8-98e4-7736df23ca51 admin admin] PATH_INFO: `/v3/projects/08cd984973f5443e85cd79d3f0e76834/users/c11cf177df3d47caafca8b5fe2076778/roles/1eb7a5b5203945c29e6d22d653f47847` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:20:55.842529 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 840/1680] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Sep 13 03:20:55 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 273 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 03:20:55.956020 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c029e03-c8f5-4cf8-98e4-7736df23ca51 admin admin] RBAC: Authorizing `identity:create_grant(project_id=08cd984973f5443e85cd79d3f0e76834, user_id=c11cf177df3d47caafca8b5fe2076778, role_id=1eb7a5b5203945c29e6d22d653f47847)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:20:55.967661 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c029e03-c8f5-4cf8-98e4-7736df23ca51 admin admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:20:56.002449 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 841/1681] 10.222.0.18 () {60 vars in 1483 bytes} [Tue Sep 13 03:20:55 2022] PUT /identity/v3/projects/08cd984973f5443e85cd79d3f0e76834/users/c11cf177df3d47caafca8b5fe2076778/roles/1eb7a5b5203945c29e6d22d653f47847 => generated 0 bytes in 213 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 03:20:56.004439 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c7cf56f-c012-4bdf-b5f4-5f645573cc85 None None] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:20:56.004722 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c7cf56f-c012-4bdf-b5f4-5f645573cc85 None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:20:56.004994 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c7cf56f-c012-4bdf-b5f4-5f645573cc85 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:20:56.080129 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.auth.core [None req-5c7cf56f-c012-4bdf-b5f4-5f645573cc85 None None] MFA Rules not processed for user `c11cf177df3d47caafca8b5fe2076778`. Rule list: `[]` (Enabled: `True`). {{(pid=81400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 03:20:56.081457 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-5c7cf56f-c012-4bdf-b5f4-5f645573cc85 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:20:56.184212 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:20:56.250250 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 841/1682] 10.222.0.18 () {58 vars in 958 bytes} [Tue Sep 13 03:20:56 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 247 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 03:20:56.263779 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c029e03-c8f5-4cf8-98e4-7736df23ca51 admin admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:20:56.267432 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-6c029e03-c8f5-4cf8-98e4-7736df23ca51 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:20:56.428367 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c029e03-c8f5-4cf8-98e4-7736df23ca51 admin admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:20:56.469121 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf1ee7c0-ecd0-4b4b-adfe-bf77682f9c61 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7377f5bee4a49959182e1bd34bc97a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b9d94f6a68f7470d9494f112175620dc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:20:56.471810 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf1ee7c0-ecd0-4b4b-adfe-bf77682f9c61 service nova] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:20:56.474386 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf1ee7c0-ecd0-4b4b-adfe-bf77682f9c61 service nova] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:20:56.475812 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf1ee7c0-ecd0-4b4b-adfe-bf77682f9c61 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:20:56.539587 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf1ee7c0-ecd0-4b4b-adfe-bf77682f9c61 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:20:56.550123 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf1ee7c0-ecd0-4b4b-adfe-bf77682f9c61 service nova] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:20:56.604542 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 842/1683] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Sep 13 03:20:56 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 341 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 03:20:56.993668 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c7cf56f-c012-4bdf-b5f4-5f645573cc85 None None] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:20:57.233153 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c7cf56f-c012-4bdf-b5f4-5f645573cc85 None None] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:20:57.249028 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f94acb91-a90f-4c23-9722-edcd095fee34 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76c5181fc7be45279ac00017bb4640b3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:20:57.249688 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f94acb91-a90f-4c23-9722-edcd095fee34 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:20:57.249964 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f94acb91-a90f-4c23-9722-edcd095fee34 admin admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:20:57.250230 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f94acb91-a90f-4c23-9722-edcd095fee34 admin admin] PATH_INFO: `/v3/users/c11cf177df3d47caafca8b5fe2076778` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:20:57.263570 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f94acb91-a90f-4c23-9722-edcd095fee34 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c11cf177df3d47caafca8b5fe2076778)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:20:57.270001 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f94acb91-a90f-4c23-9722-edcd095fee34 admin admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:20:57.417377 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-f94acb91-a90f-4c23-9722-edcd095fee34 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c11cf177df3d47caafca8b5fe2076778'} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:20:57.461374 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-f94acb91-a90f-4c23-9722-edcd095fee34 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c11cf177df3d47caafca8b5fe2076778'} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:20:57.461906 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-f94acb91-a90f-4c23-9722-edcd095fee34 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c11cf177df3d47caafca8b5fe2076778'} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:20:57.510040 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-f94acb91-a90f-4c23-9722-edcd095fee34 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c11cf177df3d47caafca8b5fe2076778'} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:20:57.534473 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-f94acb91-a90f-4c23-9722-edcd095fee34 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c11cf177df3d47caafca8b5fe2076778'} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:20:57.574879 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 842/1684] 10.222.0.18 () {58 vars in 1224 bytes} [Tue Sep 13 03:20:56 2022] DELETE /identity/v3/users/c11cf177df3d47caafca8b5fe2076778 => generated 0 bytes in 579 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 03:20:57.576490 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52b906fe-a12f-4095-9afc-399012c88df1 None None] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:20:57.576923 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52b906fe-a12f-4095-9afc-399012c88df1 None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:20:57.577022 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52b906fe-a12f-4095-9afc-399012c88df1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:20:57.648574 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.auth.core [None req-52b906fe-a12f-4095-9afc-399012c88df1 None None] MFA Rules not processed for user `49f280ab086b4610855c4ffa378517d9`. Rule list: `[]` (Enabled: `True`). {{(pid=81399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 03:20:57.648574 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-52b906fe-a12f-4095-9afc-399012c88df1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:20:57.866173 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:20:57.886514 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 843/1685] 10.222.0.18 () {58 vars in 958 bytes} [Tue Sep 13 03:20:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 311 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 03:20:57.961616 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f94acb91-a90f-4c23-9722-edcd095fee34 admin admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:20:57.963055 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-f94acb91-a90f-4c23-9722-edcd095fee34 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:20:58.163317 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f94acb91-a90f-4c23-9722-edcd095fee34 admin admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:20:58.176532 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1508974f-fe19-44b3-aecc-15e5374aefd4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdffbc851086441ebd01022d1e4c2609', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b9d94f6a68f7470d9494f112175620dc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:20:58.177810 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1508974f-fe19-44b3-aecc-15e5374aefd4 service neutron] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:20:58.178093 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1508974f-fe19-44b3-aecc-15e5374aefd4 service neutron] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:20:58.178361 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1508974f-fe19-44b3-aecc-15e5374aefd4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:20:58.205751 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1508974f-fe19-44b3-aecc-15e5374aefd4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:20:58.215441 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1508974f-fe19-44b3-aecc-15e5374aefd4 service neutron] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:20:58.232830 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 843/1686] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Sep 13 03:20:57 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 272 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 03:20:59.244950 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52b906fe-a12f-4095-9afc-399012c88df1 None None] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:20:59.246331 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-52b906fe-a12f-4095-9afc-399012c88df1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:20:59.268348 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52b906fe-a12f-4095-9afc-399012c88df1 None None] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:20:59.281745 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85ef6f85-a6e7-4deb-928a-c13b55aeade4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76c5181fc7be45279ac00017bb4640b3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:20:59.282377 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ef6f85-a6e7-4deb-928a-c13b55aeade4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:20:59.283003 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ef6f85-a6e7-4deb-928a-c13b55aeade4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:20:59.283475 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ef6f85-a6e7-4deb-928a-c13b55aeade4 admin admin] PATH_INFO: `/v3/users/b3313cfa155542c3afc5cd2c56af6dc7` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:20:59.287713 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85ef6f85-a6e7-4deb-928a-c13b55aeade4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b3313cfa155542c3afc5cd2c56af6dc7)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:20:59.289994 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85ef6f85-a6e7-4deb-928a-c13b55aeade4 admin admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:20:59.394726 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-85ef6f85-a6e7-4deb-928a-c13b55aeade4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b3313cfa155542c3afc5cd2c56af6dc7'} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:20:59.416937 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-85ef6f85-a6e7-4deb-928a-c13b55aeade4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b3313cfa155542c3afc5cd2c56af6dc7'} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:20:59.435790 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-85ef6f85-a6e7-4deb-928a-c13b55aeade4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b3313cfa155542c3afc5cd2c56af6dc7'} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:20:59.443646 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-85ef6f85-a6e7-4deb-928a-c13b55aeade4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b3313cfa155542c3afc5cd2c56af6dc7'} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:20:59.461525 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-85ef6f85-a6e7-4deb-928a-c13b55aeade4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b3313cfa155542c3afc5cd2c56af6dc7'} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:20:59.480607 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 844/1687] 10.222.0.18 () {58 vars in 1224 bytes} [Tue Sep 13 03:20:59 2022] DELETE /identity/v3/users/b3313cfa155542c3afc5cd2c56af6dc7 => generated 0 bytes in 236 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 03:20:59.484938 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-541c56b2-5e23-40ea-8908-aa58380125a9 None None] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:20:59.485880 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-541c56b2-5e23-40ea-8908-aa58380125a9 None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:20:59.486369 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-541c56b2-5e23-40ea-8908-aa58380125a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:20:59.500467 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:20:59.523627 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.auth.core [None req-541c56b2-5e23-40ea-8908-aa58380125a9 None None] MFA Rules not processed for user `49f280ab086b4610855c4ffa378517d9`. Rule list: `[]` (Enabled: `True`). {{(pid=81400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 03:20:59.525263 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-541c56b2-5e23-40ea-8908-aa58380125a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:20:59.653077 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 844/1688] 10.222.0.18 () {58 vars in 958 bytes} [Tue Sep 13 03:20:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 03:20:59.732964 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85ef6f85-a6e7-4deb-928a-c13b55aeade4 admin admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:20:59.734659 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-85ef6f85-a6e7-4deb-928a-c13b55aeade4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:20:59.870101 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85ef6f85-a6e7-4deb-928a-c13b55aeade4 admin admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:20:59.874063 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb024ad2-e5d1-46b5-b19d-b8ec52b3c328 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdffbc851086441ebd01022d1e4c2609', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b9d94f6a68f7470d9494f112175620dc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:20:59.874730 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb024ad2-e5d1-46b5-b19d-b8ec52b3c328 service neutron] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:20:59.875016 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb024ad2-e5d1-46b5-b19d-b8ec52b3c328 service neutron] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:20:59.875293 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb024ad2-e5d1-46b5-b19d-b8ec52b3c328 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:20:59.900579 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb024ad2-e5d1-46b5-b19d-b8ec52b3c328 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:20:59.902119 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb024ad2-e5d1-46b5-b19d-b8ec52b3c328 service neutron] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:20:59.913679 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 845/1689] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Sep 13 03:20:59 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 181 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 03:21:00.147215 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb249439-ca3d-4fe8-b639-8d5419290904 None None] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:21:00.147482 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb249439-ca3d-4fe8-b639-8d5419290904 None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:21:00.148092 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb249439-ca3d-4fe8-b639-8d5419290904 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:21:00.228140 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.auth.core [None req-eb249439-ca3d-4fe8-b639-8d5419290904 None None] MFA Rules not processed for user `80f6f9a952d84ccab0de028d58aa6914`. Rule list: `[]` (Enabled: `True`). {{(pid=81400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 03:21:00.229950 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-eb249439-ca3d-4fe8-b639-8d5419290904 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:21:00.388665 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 845/1690] 10.222.0.18 () {58 vars in 958 bytes} [Tue Sep 13 03:21:00 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 243 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 03:21:00.414973 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb024ad2-e5d1-46b5-b19d-b8ec52b3c328 service neutron] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:21:00.416427 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-fb024ad2-e5d1-46b5-b19d-b8ec52b3c328 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:21:00.571295 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb024ad2-e5d1-46b5-b19d-b8ec52b3c328 service neutron] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:21:00.582069 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06db06d6-a2f6-4912-a1ee-f8072eaba5a5 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7377f5bee4a49959182e1bd34bc97a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b9d94f6a68f7470d9494f112175620dc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:21:00.583691 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06db06d6-a2f6-4912-a1ee-f8072eaba5a5 service nova] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:21:00.584180 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06db06d6-a2f6-4912-a1ee-f8072eaba5a5 service nova] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:21:00.584638 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06db06d6-a2f6-4912-a1ee-f8072eaba5a5 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:21:00.597989 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06db06d6-a2f6-4912-a1ee-f8072eaba5a5 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:21:00.600613 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06db06d6-a2f6-4912-a1ee-f8072eaba5a5 service nova] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:21:00.611248 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:21:00.627365 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 846/1691] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Sep 13 03:21:00 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 213 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 03:21:00.770539 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb249439-ca3d-4fe8-b639-8d5419290904 None None] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:21:00.772704 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-eb249439-ca3d-4fe8-b639-8d5419290904 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:21:00.808414 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb249439-ca3d-4fe8-b639-8d5419290904 None None] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:21:00.816915 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4dd8c6a-3813-424c-a30c-2e50e619c651 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76c5181fc7be45279ac00017bb4640b3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:21:00.817620 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4dd8c6a-3813-424c-a30c-2e50e619c651 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:21:00.818647 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4dd8c6a-3813-424c-a30c-2e50e619c651 admin admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:21:00.818934 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4dd8c6a-3813-424c-a30c-2e50e619c651 admin admin] PATH_INFO: `/v3/projects/08cd984973f5443e85cd79d3f0e76834` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:21:00.821176 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4dd8c6a-3813-424c-a30c-2e50e619c651 admin admin] RBAC: Authorizing `identity:delete_project(project_id=08cd984973f5443e85cd79d3f0e76834)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:21:00.823149 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4dd8c6a-3813-424c-a30c-2e50e619c651 admin admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:21:00.949502 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-e4dd8c6a-3813-424c-a30c-2e50e619c651 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '08cd984973f5443e85cd79d3f0e76834', 'request_id': 'req-e4dd8c6a-3813-424c-a30c-2e50e619c651'} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:21:00.964872 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-e4dd8c6a-3813-424c-a30c-2e50e619c651 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '08cd984973f5443e85cd79d3f0e76834', 'request_id': 'req-e4dd8c6a-3813-424c-a30c-2e50e619c651'} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:21:00.980990 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-e4dd8c6a-3813-424c-a30c-2e50e619c651 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '08cd984973f5443e85cd79d3f0e76834', 'request_id': 'req-e4dd8c6a-3813-424c-a30c-2e50e619c651'} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:21:01.016682 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-e4dd8c6a-3813-424c-a30c-2e50e619c651 admin admin] The token cache is being invalidate because project 08cd984973f5443e85cd79d3f0e76834 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81400) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 03:21:01.017366 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-e4dd8c6a-3813-424c-a30c-2e50e619c651 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:21:01.018785 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-e4dd8c6a-3813-424c-a30c-2e50e619c651 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:21:01.021724 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 846/1692] 10.222.0.18 () {58 vars in 1233 bytes} [Tue Sep 13 03:21:00 2022] DELETE /identity/v3/projects/08cd984973f5443e85cd79d3f0e76834 => generated 0 bytes in 251 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 03:21:01.026500 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7963af21-24af-4cc7-be83-d574a193e323 None None] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:21:01.027125 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7963af21-24af-4cc7-be83-d574a193e323 None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:21:01.027599 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7963af21-24af-4cc7-be83-d574a193e323 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:21:01.067285 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.auth.core [None req-7963af21-24af-4cc7-be83-d574a193e323 None None] MFA Rules not processed for user `49f280ab086b4610855c4ffa378517d9`. Rule list: `[]` (Enabled: `True`). {{(pid=81399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 03:21:01.069777 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-7963af21-24af-4cc7-be83-d574a193e323 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:21:01.193884 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 847/1693] 10.222.0.18 () {58 vars in 958 bytes} [Tue Sep 13 03:21:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 03:21:01.194824 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a507123-d441-4467-b903-fa49979b0450 None None] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:21:01.195111 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a507123-d441-4467-b903-fa49979b0450 None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:21:01.195382 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a507123-d441-4467-b903-fa49979b0450 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:21:01.238849 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.auth.core [None req-0a507123-d441-4467-b903-fa49979b0450 None None] MFA Rules not processed for user `49f280ab086b4610855c4ffa378517d9`. Rule list: `[]` (Enabled: `True`). {{(pid=81400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 03:21:01.240821 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-0a507123-d441-4467-b903-fa49979b0450 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:21:01.263578 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 847/1694] 10.222.0.18 () {58 vars in 958 bytes} [Tue Sep 13 03:21:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 70 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 03:21:01.267244 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7963af21-24af-4cc7-be83-d574a193e323 None None] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:21:01.282759 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7963af21-24af-4cc7-be83-d574a193e323 None None] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:21:01.295208 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-932d3323-2abb-4f44-b144-524764c33835 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76c5181fc7be45279ac00017bb4640b3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:21:01.297091 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-932d3323-2abb-4f44-b144-524764c33835 admin admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:21:01.297850 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-932d3323-2abb-4f44-b144-524764c33835 admin admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:21:01.298130 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-932d3323-2abb-4f44-b144-524764c33835 admin admin] PATH_INFO: `/v3/domains` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:21:01.300393 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-932d3323-2abb-4f44-b144-524764c33835 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 03:21:01.300721 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-932d3323-2abb-4f44-b144-524764c33835 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:21:01.303598 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-932d3323-2abb-4f44-b144-524764c33835 admin admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:21:01.332890 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 848/1695] 10.222.0.18 () {58 vars in 1153 bytes} [Tue Sep 13 03:21:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 66 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 03:21:01.338118 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a507123-d441-4467-b903-fa49979b0450 None None] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:21:01.352103 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a507123-d441-4467-b903-fa49979b0450 None None] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:21:01.364133 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c859649-9954-4fd3-a76c-49c5a787d6d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76c5181fc7be45279ac00017bb4640b3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:21:01.366117 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c859649-9954-4fd3-a76c-49c5a787d6d6 admin admin] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:21:01.367543 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c859649-9954-4fd3-a76c-49c5a787d6d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:21:01.367919 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c859649-9954-4fd3-a76c-49c5a787d6d6 admin admin] PATH_INFO: `/v3/projects` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:21:01.369835 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c859649-9954-4fd3-a76c-49c5a787d6d6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:21:01.377229 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c859649-9954-4fd3-a76c-49c5a787d6d6 admin admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:21:01.408004 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:21:01.412576 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-932d3323-2abb-4f44-b144-524764c33835 admin admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:21:01.414328 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-932d3323-2abb-4f44-b144-524764c33835 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:21:01.438252 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-932d3323-2abb-4f44-b144-524764c33835 admin admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:21:01.452213 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e59f2252-96be-4411-ab2f-992f6793b7c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76c5181fc7be45279ac00017bb4640b3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:21:01.454415 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e59f2252-96be-4411-ab2f-992f6793b7c8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:21:01.455022 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e59f2252-96be-4411-ab2f-992f6793b7c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:21:01.456288 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e59f2252-96be-4411-ab2f-992f6793b7c8 admin admin] PATH_INFO: `/v3/projects/15143b2e72aa40f78184abf09d8bd9c1` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:21:01.458970 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e59f2252-96be-4411-ab2f-992f6793b7c8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=15143b2e72aa40f78184abf09d8bd9c1)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:21:01.465965 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e59f2252-96be-4411-ab2f-992f6793b7c8 admin admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:21:01.481876 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 848/1696] 10.222.0.18 () {60 vars in 1153 bytes} [Tue Sep 13 03:21:01 2022] POST /identity/v3/projects => generated 363 bytes in 142 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 03:21:01.482711 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef418682-6d16-43a7-90f4-aaa01d905d4d None None] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:21:01.485174 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef418682-6d16-43a7-90f4-aaa01d905d4d None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:21:01.485569 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef418682-6d16-43a7-90f4-aaa01d905d4d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:21:01.547795 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.auth.core [None req-ef418682-6d16-43a7-90f4-aaa01d905d4d None None] MFA Rules not processed for user `83bd9ab163ba4229bcfd810345430564`. Rule list: `[]` (Enabled: `True`). {{(pid=81400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 03:21:01.553570 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-ef418682-6d16-43a7-90f4-aaa01d905d4d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:21:01.598011 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-e59f2252-96be-4411-ab2f-992f6793b7c8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '15143b2e72aa40f78184abf09d8bd9c1', 'request_id': 'req-e59f2252-96be-4411-ab2f-992f6793b7c8'} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:21:01.618894 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-e59f2252-96be-4411-ab2f-992f6793b7c8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '15143b2e72aa40f78184abf09d8bd9c1', 'request_id': 'req-e59f2252-96be-4411-ab2f-992f6793b7c8'} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:21:01.635316 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-e59f2252-96be-4411-ab2f-992f6793b7c8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '15143b2e72aa40f78184abf09d8bd9c1', 'request_id': 'req-e59f2252-96be-4411-ab2f-992f6793b7c8'} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:21:01.665623 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-e59f2252-96be-4411-ab2f-992f6793b7c8 admin admin] The token cache is being invalidate because project 15143b2e72aa40f78184abf09d8bd9c1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81399) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 03:21:01.665922 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-e59f2252-96be-4411-ab2f-992f6793b7c8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:21:01.668901 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-e59f2252-96be-4411-ab2f-992f6793b7c8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:21:01.671583 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 849/1697] 10.222.0.18 () {58 vars in 1233 bytes} [Tue Sep 13 03:21:01 2022] DELETE /identity/v3/projects/15143b2e72aa40f78184abf09d8bd9c1 => generated 0 bytes in 260 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 13 03:21:01.672353 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e59f2252-96be-4411-ab2f-992f6793b7c8 admin admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:21:01.673716 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-e59f2252-96be-4411-ab2f-992f6793b7c8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:21:01.725692 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 849/1698] 10.222.0.18 () {58 vars in 958 bytes} [Tue Sep 13 03:21:01 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 246 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 03:21:01.743807 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef418682-6d16-43a7-90f4-aaa01d905d4d None None] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:21:01.748567 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-ef418682-6d16-43a7-90f4-aaa01d905d4d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:21:01.840942 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e59f2252-96be-4411-ab2f-992f6793b7c8 admin admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:21:01.852540 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c3bd7dc-2c89-4965-a88e-ee43276ccd31 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76c5181fc7be45279ac00017bb4640b3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:21:01.853650 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c3bd7dc-2c89-4965-a88e-ee43276ccd31 admin admin] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:21:01.853650 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c3bd7dc-2c89-4965-a88e-ee43276ccd31 admin admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:21:01.853834 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c3bd7dc-2c89-4965-a88e-ee43276ccd31 admin admin] PATH_INFO: `/v3/users` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:21:01.854488 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c3bd7dc-2c89-4965-a88e-ee43276ccd31 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:21:01.860570 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c3bd7dc-2c89-4965-a88e-ee43276ccd31 admin admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:21:01.903145 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:21:01.920605 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 850/1699] 10.222.0.18 () {60 vars in 1144 bytes} [Tue Sep 13 03:21:01 2022] POST /identity/v3/users => generated 282 bytes in 247 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 03:21:01.921467 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c3bd7dc-2c89-4965-a88e-ee43276ccd31 admin admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:21:01.946212 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c3bd7dc-2c89-4965-a88e-ee43276ccd31 admin admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:21:01.961023 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c828081-9413-4525-b9ee-6700ea988c5c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76c5181fc7be45279ac00017bb4640b3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:21:01.961023 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c828081-9413-4525-b9ee-6700ea988c5c admin admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:21:01.961407 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c828081-9413-4525-b9ee-6700ea988c5c admin admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:21:01.961407 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c828081-9413-4525-b9ee-6700ea988c5c admin admin] PATH_INFO: `/v3/roles` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:21:01.961983 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c828081-9413-4525-b9ee-6700ea988c5c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:21:01.967758 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c828081-9413-4525-b9ee-6700ea988c5c admin admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:21:01.988094 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef418682-6d16-43a7-90f4-aaa01d905d4d None None] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:21:01.994836 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67f008d8-53fd-4843-a6f2-b870302a0cfa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76c5181fc7be45279ac00017bb4640b3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:21:01.995931 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f008d8-53fd-4843-a6f2-b870302a0cfa admin admin] REQUEST_METHOD: `DELETE` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:21:01.996395 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f008d8-53fd-4843-a6f2-b870302a0cfa admin admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:21:01.996906 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f008d8-53fd-4843-a6f2-b870302a0cfa admin admin] PATH_INFO: `/v3/users/dceabdb575394818b6504bc03dc9958f` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:21:01.998296 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 851/1700] 10.222.0.18 () {58 vars in 1122 bytes} [Tue Sep 13 03:21:01 2022] GET /identity/v3/roles => generated 1420 bytes in 77 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 03:21:01.999033 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67f008d8-53fd-4843-a6f2-b870302a0cfa admin admin] RBAC: Authorizing `identity:delete_user(user_id=dceabdb575394818b6504bc03dc9958f)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:21:01.999366 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c828081-9413-4525-b9ee-6700ea988c5c admin admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:21:02.002760 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67f008d8-53fd-4843-a6f2-b870302a0cfa admin admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:21:02.011815 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c828081-9413-4525-b9ee-6700ea988c5c admin admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:21:02.016691 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5fc17cb-fa78-4051-8fe1-5663296c9f68 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76c5181fc7be45279ac00017bb4640b3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:21:02.017321 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5fc17cb-fa78-4051-8fe1-5663296c9f68 admin admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:21:02.017588 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5fc17cb-fa78-4051-8fe1-5663296c9f68 admin admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:21:02.017883 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5fc17cb-fa78-4051-8fe1-5663296c9f68 admin admin] PATH_INFO: `/v3/roles` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:21:02.018826 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5fc17cb-fa78-4051-8fe1-5663296c9f68 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:21:02.020448 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5fc17cb-fa78-4051-8fe1-5663296c9f68 admin admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:21:02.033032 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 852/1701] 10.222.0.18 () {58 vars in 1122 bytes} [Tue Sep 13 03:21:01 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 03:21:02.038753 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5fc17cb-fa78-4051-8fe1-5663296c9f68 admin admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:21:02.062859 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5fc17cb-fa78-4051-8fe1-5663296c9f68 admin admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:21:02.074304 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bad39a64-5191-4de5-8fa6-86d7f73a376c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76c5181fc7be45279ac00017bb4640b3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:21:02.077371 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bad39a64-5191-4de5-8fa6-86d7f73a376c admin admin] REQUEST_METHOD: `PUT` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:21:02.078063 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bad39a64-5191-4de5-8fa6-86d7f73a376c admin admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:21:02.079706 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bad39a64-5191-4de5-8fa6-86d7f73a376c admin admin] PATH_INFO: `/v3/projects/abfd705df8b14fb08d55dcdf98c6d7cc/users/6b344f309db7491d976dbfb78bc8a334/roles/1eb7a5b5203945c29e6d22d653f47847` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:21:02.100665 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-67f008d8-53fd-4843-a6f2-b870302a0cfa admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'dceabdb575394818b6504bc03dc9958f'} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:21:02.113927 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-67f008d8-53fd-4843-a6f2-b870302a0cfa admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'dceabdb575394818b6504bc03dc9958f'} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:21:02.115213 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-67f008d8-53fd-4843-a6f2-b870302a0cfa admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'dceabdb575394818b6504bc03dc9958f'} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:21:02.134962 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bad39a64-5191-4de5-8fa6-86d7f73a376c admin admin] RBAC: Authorizing `identity:create_grant(project_id=abfd705df8b14fb08d55dcdf98c6d7cc, user_id=6b344f309db7491d976dbfb78bc8a334, role_id=1eb7a5b5203945c29e6d22d653f47847)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:21:02.138508 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bad39a64-5191-4de5-8fa6-86d7f73a376c admin admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:21:02.172550 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 853/1702] 10.222.0.18 () {60 vars in 1483 bytes} [Tue Sep 13 03:21:02 2022] PUT /identity/v3/projects/abfd705df8b14fb08d55dcdf98c6d7cc/users/6b344f309db7491d976dbfb78bc8a334/roles/1eb7a5b5203945c29e6d22d653f47847 => generated 0 bytes in 135 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 03:21:02.183149 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f15ca0f4-21f7-4984-9285-ae03a250d9dc None None] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:21:02.183948 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f15ca0f4-21f7-4984-9285-ae03a250d9dc None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:21:02.183948 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f15ca0f4-21f7-4984-9285-ae03a250d9dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:21:02.207577 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-67f008d8-53fd-4843-a6f2-b870302a0cfa admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'dceabdb575394818b6504bc03dc9958f'} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:21:02.255517 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-67f008d8-53fd-4843-a6f2-b870302a0cfa admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'dceabdb575394818b6504bc03dc9958f'} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:21:02.262220 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.auth.core [None req-f15ca0f4-21f7-4984-9285-ae03a250d9dc None None] MFA Rules not processed for user `6b344f309db7491d976dbfb78bc8a334`. Rule list: `[]` (Enabled: `True`). {{(pid=81399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 03:21:02.263480 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-f15ca0f4-21f7-4984-9285-ae03a250d9dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:21:02.307579 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 850/1703] 10.222.0.18 () {58 vars in 1224 bytes} [Tue Sep 13 03:21:01 2022] DELETE /identity/v3/users/dceabdb575394818b6504bc03dc9958f => generated 0 bytes in 579 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 03:21:02.476701 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67f008d8-53fd-4843-a6f2-b870302a0cfa admin admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:21:02.479065 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 854/1704] 10.222.0.18 () {58 vars in 958 bytes} [Tue Sep 13 03:21:02 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 297 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 13 03:21:02.480941 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-67f008d8-53fd-4843-a6f2-b870302a0cfa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:21:02.542692 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:21:02.674246 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67f008d8-53fd-4843-a6f2-b870302a0cfa admin admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:21:02.696197 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07c3a1c9-d6ee-43e8-ad25-0d1fa22f38c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76c5181fc7be45279ac00017bb4640b3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:21:02.697313 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07c3a1c9-d6ee-43e8-ad25-0d1fa22f38c1 admin admin] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:21:02.698278 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07c3a1c9-d6ee-43e8-ad25-0d1fa22f38c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:21:02.698799 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07c3a1c9-d6ee-43e8-ad25-0d1fa22f38c1 admin admin] PATH_INFO: `/v3/projects` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:21:02.699813 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07c3a1c9-d6ee-43e8-ad25-0d1fa22f38c1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:21:02.702731 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07c3a1c9-d6ee-43e8-ad25-0d1fa22f38c1 admin admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:21:02.803139 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 851/1705] 10.222.0.18 () {60 vars in 1153 bytes} [Tue Sep 13 03:21:02 2022] POST /identity/v3/projects => generated 365 bytes in 327 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 03:21:02.807584 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f15ca0f4-21f7-4984-9285-ae03a250d9dc None None] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:21:02.948310 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f15ca0f4-21f7-4984-9285-ae03a250d9dc None None] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:21:03.020003 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:21:03.038572 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6fc5766-2b55-40f4-8f07-57ca9386ff11 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76c5181fc7be45279ac00017bb4640b3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:21:03.039921 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6fc5766-2b55-40f4-8f07-57ca9386ff11 admin admin] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:21:03.040208 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6fc5766-2b55-40f4-8f07-57ca9386ff11 admin admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:21:03.040490 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6fc5766-2b55-40f4-8f07-57ca9386ff11 admin admin] PATH_INFO: `/v3/users` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:21:03.041136 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6fc5766-2b55-40f4-8f07-57ca9386ff11 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:21:03.045658 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6fc5766-2b55-40f4-8f07-57ca9386ff11 admin admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:21:03.087892 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 855/1706] 10.222.0.18 () {60 vars in 1144 bytes} [Tue Sep 13 03:21:02 2022] POST /identity/v3/users => generated 289 bytes in 281 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 03:21:03.091413 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07c3a1c9-d6ee-43e8-ad25-0d1fa22f38c1 admin admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:21:03.103430 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07c3a1c9-d6ee-43e8-ad25-0d1fa22f38c1 admin admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:21:03.109249 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dd9b5b8-f6d1-4956-8d03-e629b05413e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76c5181fc7be45279ac00017bb4640b3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:21:03.109872 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dd9b5b8-f6d1-4956-8d03-e629b05413e6 admin admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:21:03.110144 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dd9b5b8-f6d1-4956-8d03-e629b05413e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:21:03.110411 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dd9b5b8-f6d1-4956-8d03-e629b05413e6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:21:03.111103 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dd9b5b8-f6d1-4956-8d03-e629b05413e6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:21:03.112706 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dd9b5b8-f6d1-4956-8d03-e629b05413e6 admin admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:21:03.126529 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 852/1707] 10.222.0.18 () {58 vars in 1122 bytes} [Tue Sep 13 03:21:03 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 03:21:03.130846 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6fc5766-2b55-40f4-8f07-57ca9386ff11 admin admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:21:03.144926 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6fc5766-2b55-40f4-8f07-57ca9386ff11 admin admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:21:03.150762 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b02e80b8-be0e-4fa3-b3be-43f77bea03fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76c5181fc7be45279ac00017bb4640b3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:21:03.151445 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b02e80b8-be0e-4fa3-b3be-43f77bea03fb admin admin] REQUEST_METHOD: `PUT` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:21:03.151728 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b02e80b8-be0e-4fa3-b3be-43f77bea03fb admin admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:21:03.152008 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b02e80b8-be0e-4fa3-b3be-43f77bea03fb admin admin] PATH_INFO: `/v3/projects/49ef5d64ceef40229724e226a98bf998/users/e0d9caad4d6d4f72ad4807230776d96e/roles/425d4b481c7846e0bc07e7e805d5b15f` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:21:03.182109 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b02e80b8-be0e-4fa3-b3be-43f77bea03fb admin admin] RBAC: Authorizing `identity:create_grant(project_id=49ef5d64ceef40229724e226a98bf998, user_id=e0d9caad4d6d4f72ad4807230776d96e, role_id=425d4b481c7846e0bc07e7e805d5b15f)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:21:03.189960 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b02e80b8-be0e-4fa3-b3be-43f77bea03fb admin admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:21:03.214663 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 856/1708] 10.222.0.18 () {60 vars in 1483 bytes} [Tue Sep 13 03:21:03 2022] PUT /identity/v3/projects/49ef5d64ceef40229724e226a98bf998/users/e0d9caad4d6d4f72ad4807230776d96e/roles/425d4b481c7846e0bc07e7e805d5b15f => generated 0 bytes in 84 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 03:21:03.218492 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dd9b5b8-f6d1-4956-8d03-e629b05413e6 admin admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:21:03.232939 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dd9b5b8-f6d1-4956-8d03-e629b05413e6 admin admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:21:03.325847 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5ec6b51-76a8-423f-b3e1-8d9a5c7f77e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76c5181fc7be45279ac00017bb4640b3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:21:03.327159 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5ec6b51-76a8-423f-b3e1-8d9a5c7f77e4 admin admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:21:03.327605 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5ec6b51-76a8-423f-b3e1-8d9a5c7f77e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:21:03.328060 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5ec6b51-76a8-423f-b3e1-8d9a5c7f77e4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:21:03.328885 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5ec6b51-76a8-423f-b3e1-8d9a5c7f77e4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:21:03.331307 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5ec6b51-76a8-423f-b3e1-8d9a5c7f77e4 admin admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:21:03.386950 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b02e80b8-be0e-4fa3-b3be-43f77bea03fb admin admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:21:03.413716 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b02e80b8-be0e-4fa3-b3be-43f77bea03fb admin admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:21:03.414721 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 853/1709] 10.222.0.18 () {58 vars in 1122 bytes} [Tue Sep 13 03:21:03 2022] GET /identity/v3/roles => generated 1420 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 03:21:03.432949 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef8fa963-d187-4e3c-9ff0-ba33f778f81c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76c5181fc7be45279ac00017bb4640b3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:21:03.433569 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef8fa963-d187-4e3c-9ff0-ba33f778f81c admin admin] REQUEST_METHOD: `PUT` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:21:03.433847 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef8fa963-d187-4e3c-9ff0-ba33f778f81c admin admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:21:03.434116 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef8fa963-d187-4e3c-9ff0-ba33f778f81c admin admin] PATH_INFO: `/v3/projects/49ef5d64ceef40229724e226a98bf998/users/e0d9caad4d6d4f72ad4807230776d96e/roles/425d4b481c7846e0bc07e7e805d5b15f` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:21:03.437676 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef8fa963-d187-4e3c-9ff0-ba33f778f81c admin admin] RBAC: Authorizing `identity:create_grant(project_id=49ef5d64ceef40229724e226a98bf998, user_id=e0d9caad4d6d4f72ad4807230776d96e, role_id=425d4b481c7846e0bc07e7e805d5b15f)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:21:03.441384 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef8fa963-d187-4e3c-9ff0-ba33f778f81c admin admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:21:03.483378 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea92a3b9-dbe3-4f04-90db-98fda0332820 None None] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:21:03.483378 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea92a3b9-dbe3-4f04-90db-98fda0332820 None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:21:03.483640 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea92a3b9-dbe3-4f04-90db-98fda0332820 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:21:03.490476 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 857/1710] 10.222.0.18 () {60 vars in 1483 bytes} [Tue Sep 13 03:21:03 2022] PUT /identity/v3/projects/49ef5d64ceef40229724e226a98bf998/users/e0d9caad4d6d4f72ad4807230776d96e/roles/425d4b481c7846e0bc07e7e805d5b15f => generated 0 bytes in 91 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 03:21:03.576903 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.auth.core [None req-ea92a3b9-dbe3-4f04-90db-98fda0332820 None None] MFA Rules not processed for user `e0d9caad4d6d4f72ad4807230776d96e`. Rule list: `[]` (Enabled: `True`). {{(pid=81400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 03:21:03.579875 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-ea92a3b9-dbe3-4f04-90db-98fda0332820 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:21:03.584728 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef8fa963-d187-4e3c-9ff0-ba33f778f81c admin admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:21:03.586494 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-ef8fa963-d187-4e3c-9ff0-ba33f778f81c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:21:03.708809 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 854/1711] 10.222.0.18 () {58 vars in 958 bytes} [Tue Sep 13 03:21:03 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 227 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 03:21:03.723603 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea92a3b9-dbe3-4f04-90db-98fda0332820 None None] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:21:03.725484 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-ea92a3b9-dbe3-4f04-90db-98fda0332820 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:21:03.737463 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef8fa963-d187-4e3c-9ff0-ba33f778f81c admin admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:21:03.743343 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d2985d7-4c4f-4d50-bfdb-f54aba6a1c6f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76c5181fc7be45279ac00017bb4640b3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:21:03.744545 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d2985d7-4c4f-4d50-bfdb-f54aba6a1c6f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:21:03.744545 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d2985d7-4c4f-4d50-bfdb-f54aba6a1c6f admin admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:21:03.744741 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d2985d7-4c4f-4d50-bfdb-f54aba6a1c6f admin admin] PATH_INFO: `/v3/projects/4397caa381624321bedbd87854af542e` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:21:03.745965 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d2985d7-4c4f-4d50-bfdb-f54aba6a1c6f admin admin] RBAC: Authorizing `identity:delete_project(project_id=4397caa381624321bedbd87854af542e)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:21:03.748555 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d2985d7-4c4f-4d50-bfdb-f54aba6a1c6f admin admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:21:03.790908 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:21:03.856659 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-3d2985d7-4c4f-4d50-bfdb-f54aba6a1c6f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4397caa381624321bedbd87854af542e', 'request_id': 'req-3d2985d7-4c4f-4d50-bfdb-f54aba6a1c6f'} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:21:03.870661 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-3d2985d7-4c4f-4d50-bfdb-f54aba6a1c6f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4397caa381624321bedbd87854af542e', 'request_id': 'req-3d2985d7-4c4f-4d50-bfdb-f54aba6a1c6f'} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:21:03.882693 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-3d2985d7-4c4f-4d50-bfdb-f54aba6a1c6f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4397caa381624321bedbd87854af542e', 'request_id': 'req-3d2985d7-4c4f-4d50-bfdb-f54aba6a1c6f'} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:21:03.914175 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-3d2985d7-4c4f-4d50-bfdb-f54aba6a1c6f admin admin] The token cache is being invalidate because project 4397caa381624321bedbd87854af542e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81399) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 03:21:03.914680 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-3d2985d7-4c4f-4d50-bfdb-f54aba6a1c6f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:21:03.915579 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-3d2985d7-4c4f-4d50-bfdb-f54aba6a1c6f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:21:03.917986 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 858/1712] 10.222.0.18 () {58 vars in 1233 bytes} [Tue Sep 13 03:21:03 2022] DELETE /identity/v3/projects/4397caa381624321bedbd87854af542e => generated 0 bytes in 334 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 03:21:03.930564 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb975066-52c6-4dda-916f-13dd8940def4 None None] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:21:03.930887 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb975066-52c6-4dda-916f-13dd8940def4 None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:21:03.931173 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb975066-52c6-4dda-916f-13dd8940def4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:21:03.981339 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.auth.core [None req-eb975066-52c6-4dda-916f-13dd8940def4 None None] MFA Rules not processed for user `49f280ab086b4610855c4ffa378517d9`. Rule list: `[]` (Enabled: `True`). {{(pid=81399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 03:21:03.982578 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-eb975066-52c6-4dda-916f-13dd8940def4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:21:03.982710 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea92a3b9-dbe3-4f04-90db-98fda0332820 None None] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:21:04.052692 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=87,keystone:INSERT=6,keystone:DELETE=7 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:21:04.119574 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2941483f-af3d-45a7-9a3e-9284404ed428 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '924ee3f608d34a18b0ddc4695f102a56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b9d94f6a68f7470d9494f112175620dc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:21:04.121592 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2941483f-af3d-45a7-9a3e-9284404ed428 service cinder] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:21:04.122079 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2941483f-af3d-45a7-9a3e-9284404ed428 service cinder] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:21:04.123240 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2941483f-af3d-45a7-9a3e-9284404ed428 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:21:04.128285 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-2941483f-af3d-45a7-9a3e-9284404ed428 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:21:04.154374 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 859/1713] 10.222.0.18 () {58 vars in 958 bytes} [Tue Sep 13 03:21:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 224 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 03:21:04.162443 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-092aab4b-213d-4d74-ad9c-bd30aede6362 None None] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:21:04.162944 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-092aab4b-213d-4d74-ad9c-bd30aede6362 None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:21:04.163388 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-092aab4b-213d-4d74-ad9c-bd30aede6362 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:21:04.210658 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.auth.core [None req-092aab4b-213d-4d74-ad9c-bd30aede6362 None None] MFA Rules not processed for user `49f280ab086b4610855c4ffa378517d9`. Rule list: `[]` (Enabled: `True`). {{(pid=81399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 03:21:04.212196 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-092aab4b-213d-4d74-ad9c-bd30aede6362 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:21:04.231795 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 860/1714] 10.222.0.18 () {58 vars in 958 bytes} [Tue Sep 13 03:21:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 71 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 13 03:21:04.247173 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-092aab4b-213d-4d74-ad9c-bd30aede6362 None None] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:21:04.276101 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-092aab4b-213d-4d74-ad9c-bd30aede6362 None None] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:21:04.296836 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2941483f-af3d-45a7-9a3e-9284404ed428 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:21:04.298416 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2941483f-af3d-45a7-9a3e-9284404ed428 service cinder] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:21:04.303620 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec85be1f-b45f-472f-bf4d-a85b66ddab42 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76c5181fc7be45279ac00017bb4640b3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:21:04.304265 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec85be1f-b45f-472f-bf4d-a85b66ddab42 admin admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:21:04.304545 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec85be1f-b45f-472f-bf4d-a85b66ddab42 admin admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:21:04.304816 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec85be1f-b45f-472f-bf4d-a85b66ddab42 admin admin] PATH_INFO: `/v3/domains` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:21:04.305475 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec85be1f-b45f-472f-bf4d-a85b66ddab42 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 03:21:04.305781 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec85be1f-b45f-472f-bf4d-a85b66ddab42 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:21:04.307455 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec85be1f-b45f-472f-bf4d-a85b66ddab42 admin admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:21:04.331520 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 855/1715] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Sep 13 03:21:03 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 608 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 13 03:21:04.363074 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 861/1716] 10.222.0.18 () {58 vars in 1153 bytes} [Tue Sep 13 03:21:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 121 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 03:21:04.367957 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2941483f-af3d-45a7-9a3e-9284404ed428 service cinder] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:21:04.383480 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2941483f-af3d-45a7-9a3e-9284404ed428 service cinder] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:21:04.397205 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca0ef311-0dd9-4cd2-9f34-ae5988eb505e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76c5181fc7be45279ac00017bb4640b3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:21:04.400396 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca0ef311-0dd9-4cd2-9f34-ae5988eb505e admin admin] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:21:04.401784 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca0ef311-0dd9-4cd2-9f34-ae5988eb505e admin admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:21:04.412890 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca0ef311-0dd9-4cd2-9f34-ae5988eb505e admin admin] PATH_INFO: `/v3/projects` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:21:04.413779 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca0ef311-0dd9-4cd2-9f34-ae5988eb505e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:21:04.430358 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca0ef311-0dd9-4cd2-9f34-ae5988eb505e admin admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:21:04.476495 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51f8b0e6-c230-4002-82be-1c9a046c27f6 None None] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:21:04.476783 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51f8b0e6-c230-4002-82be-1c9a046c27f6 None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:21:04.477080 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51f8b0e6-c230-4002-82be-1c9a046c27f6 None None] PATH_INFO: `/` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:21:04.478442 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 862/1717] 10.222.0.18 () {54 vars in 912 bytes} [Tue Sep 13 03:21:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 03:21:04.504958 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f8801f8-99d5-4527-a587-c8827f77a0cc None None] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:21:04.505599 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f8801f8-99d5-4527-a587-c8827f77a0cc None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:21:04.505599 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f8801f8-99d5-4527-a587-c8827f77a0cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:21:04.545771 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 856/1718] 10.222.0.18 () {60 vars in 1153 bytes} [Tue Sep 13 03:21:04 2022] POST /identity/v3/projects => generated 379 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 03:21:04.547986 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca0ef311-0dd9-4cd2-9f34-ae5988eb505e admin admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:21:04.600268 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.auth.core [None req-1f8801f8-99d5-4527-a587-c8827f77a0cc None None] MFA Rules not processed for user `cdffbc851086441ebd01022d1e4c2609`. Rule list: `[]` (Enabled: `True`). {{(pid=81399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 03:21:04.602451 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-1f8801f8-99d5-4527-a587-c8827f77a0cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:21:04.767350 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:21:04.788781 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca0ef311-0dd9-4cd2-9f34-ae5988eb505e admin admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:21:04.793365 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 863/1719] 10.222.0.18 () {58 vars in 1009 bytes} [Tue Sep 13 03:21:04 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 289 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 13 03:21:04.803741 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97bac567-27ab-49c1-b453-04b9cd40059b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76c5181fc7be45279ac00017bb4640b3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:21:04.805052 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97bac567-27ab-49c1-b453-04b9cd40059b admin admin] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:21:04.805608 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97bac567-27ab-49c1-b453-04b9cd40059b admin admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:21:04.805978 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97bac567-27ab-49c1-b453-04b9cd40059b admin admin] PATH_INFO: `/v3/users` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:21:04.807026 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97bac567-27ab-49c1-b453-04b9cd40059b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:21:04.811464 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97bac567-27ab-49c1-b453-04b9cd40059b admin admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:21:04.825224 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f8801f8-99d5-4527-a587-c8827f77a0cc None None] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:21:04.827366 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-1f8801f8-99d5-4527-a587-c8827f77a0cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:21:04.852720 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f8801f8-99d5-4527-a587-c8827f77a0cc None None] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:21:04.858052 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 857/1720] 10.222.0.18 () {60 vars in 1144 bytes} [Tue Sep 13 03:21:04 2022] POST /identity/v3/users => generated 290 bytes in 312 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 03:21:04.859986 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97bac567-27ab-49c1-b453-04b9cd40059b admin admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:21:04.861419 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6548145-a10c-4709-b203-ca73541d4604 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdffbc851086441ebd01022d1e4c2609', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b9d94f6a68f7470d9494f112175620dc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:21:04.863723 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6548145-a10c-4709-b203-ca73541d4604 service neutron] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:21:04.864010 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6548145-a10c-4709-b203-ca73541d4604 service neutron] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:21:04.866206 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6548145-a10c-4709-b203-ca73541d4604 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:21:04.887387 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6548145-a10c-4709-b203-ca73541d4604 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:21:04.887570 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97bac567-27ab-49c1-b453-04b9cd40059b admin admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:21:04.892708 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6548145-a10c-4709-b203-ca73541d4604 service neutron] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:21:04.894208 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dabd19af-db80-494c-adcc-572b5e92edb8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76c5181fc7be45279ac00017bb4640b3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:21:04.895140 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dabd19af-db80-494c-adcc-572b5e92edb8 admin admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:21:04.895619 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dabd19af-db80-494c-adcc-572b5e92edb8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:21:04.898012 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dabd19af-db80-494c-adcc-572b5e92edb8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:21:04.898858 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dabd19af-db80-494c-adcc-572b5e92edb8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:21:04.900794 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dabd19af-db80-494c-adcc-572b5e92edb8 admin admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:21:04.915927 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 864/1721] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Sep 13 03:21:04 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 03:21:04.930898 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 858/1722] 10.222.0.18 () {58 vars in 1122 bytes} [Tue Sep 13 03:21:04 2022] GET /identity/v3/roles => generated 1420 bytes in 71 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 03:21:04.933090 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6548145-a10c-4709-b203-ca73541d4604 service neutron] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:21:04.961719 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6548145-a10c-4709-b203-ca73541d4604 service neutron] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:21:04.981884 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acddca6c-2196-4a0d-a629-275c9e5ed52e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76c5181fc7be45279ac00017bb4640b3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:21:04.982525 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acddca6c-2196-4a0d-a629-275c9e5ed52e admin admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:21:04.982854 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acddca6c-2196-4a0d-a629-275c9e5ed52e admin admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:21:04.983131 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acddca6c-2196-4a0d-a629-275c9e5ed52e admin admin] PATH_INFO: `/v3/roles` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:21:04.983783 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acddca6c-2196-4a0d-a629-275c9e5ed52e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:21:04.985388 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acddca6c-2196-4a0d-a629-275c9e5ed52e admin admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:21:05.006399 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 865/1723] 10.222.0.18 () {58 vars in 1122 bytes} [Tue Sep 13 03:21:04 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 03:21:05.010218 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dabd19af-db80-494c-adcc-572b5e92edb8 admin admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:21:05.030803 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dabd19af-db80-494c-adcc-572b5e92edb8 admin admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:21:05.063853 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-867c8e1c-02e8-4cef-b648-d56a3890ef53 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76c5181fc7be45279ac00017bb4640b3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:21:05.064562 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-867c8e1c-02e8-4cef-b648-d56a3890ef53 admin admin] REQUEST_METHOD: `PUT` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:21:05.064846 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-867c8e1c-02e8-4cef-b648-d56a3890ef53 admin admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:21:05.065124 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-867c8e1c-02e8-4cef-b648-d56a3890ef53 admin admin] PATH_INFO: `/v3/projects/11c5a74585df4cd195f06c9b2aadc3f9/users/b640f179ec9247398eb7f26376dde7d4/roles/1eb7a5b5203945c29e6d22d653f47847` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:21:05.131199 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-867c8e1c-02e8-4cef-b648-d56a3890ef53 admin admin] RBAC: Authorizing `identity:create_grant(project_id=11c5a74585df4cd195f06c9b2aadc3f9, user_id=b640f179ec9247398eb7f26376dde7d4, role_id=1eb7a5b5203945c29e6d22d653f47847)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:21:05.134337 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-867c8e1c-02e8-4cef-b648-d56a3890ef53 admin admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:21:05.158456 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 859/1724] 10.222.0.18 () {60 vars in 1483 bytes} [Tue Sep 13 03:21:05 2022] PUT /identity/v3/projects/11c5a74585df4cd195f06c9b2aadc3f9/users/b640f179ec9247398eb7f26376dde7d4/roles/1eb7a5b5203945c29e6d22d653f47847 => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 03:21:05.171446 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f36ff6e-0458-4fff-b662-4d8d1f380f84 None None] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:21:05.171743 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f36ff6e-0458-4fff-b662-4d8d1f380f84 None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:21:05.172023 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f36ff6e-0458-4fff-b662-4d8d1f380f84 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:21:05.243116 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.auth.core [None req-9f36ff6e-0458-4fff-b662-4d8d1f380f84 None None] MFA Rules not processed for user `b640f179ec9247398eb7f26376dde7d4`. Rule list: `[]` (Enabled: `True`). {{(pid=81399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 03:21:05.247554 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-9f36ff6e-0458-4fff-b662-4d8d1f380f84 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:21:05.374377 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-867c8e1c-02e8-4cef-b648-d56a3890ef53 admin admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:21:05.381471 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-867c8e1c-02e8-4cef-b648-d56a3890ef53 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:21:05.442186 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 866/1725] 10.222.0.18 () {58 vars in 958 bytes} [Tue Sep 13 03:21:05 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 272 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 03:21:05.447986 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a3e9b2a-b8fe-4921-8776-a1c9ff3e4efa None None] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:21:05.450497 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a3e9b2a-b8fe-4921-8776-a1c9ff3e4efa None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:21:05.450850 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a3e9b2a-b8fe-4921-8776-a1c9ff3e4efa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:21:05.486506 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.auth.core [None req-1a3e9b2a-b8fe-4921-8776-a1c9ff3e4efa None None] MFA Rules not processed for user `80f6f9a952d84ccab0de028d58aa6914`. Rule list: `[]` (Enabled: `True`). {{(pid=81399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 03:21:05.487831 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-1a3e9b2a-b8fe-4921-8776-a1c9ff3e4efa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:21:05.525226 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:21:05.552812 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-867c8e1c-02e8-4cef-b648-d56a3890ef53 admin admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:21:05.557957 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c7c86f6-83ed-41bc-b6eb-986c20aa91e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76c5181fc7be45279ac00017bb4640b3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:21:05.558644 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c7c86f6-83ed-41bc-b6eb-986c20aa91e6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:21:05.559685 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c7c86f6-83ed-41bc-b6eb-986c20aa91e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:21:05.559962 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c7c86f6-83ed-41bc-b6eb-986c20aa91e6 admin admin] PATH_INFO: `/v3/users/6b344f309db7491d976dbfb78bc8a334` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:21:05.561272 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c7c86f6-83ed-41bc-b6eb-986c20aa91e6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6b344f309db7491d976dbfb78bc8a334)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:21:05.563312 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c7c86f6-83ed-41bc-b6eb-986c20aa91e6 admin admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:21:05.673787 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-7c7c86f6-83ed-41bc-b6eb-986c20aa91e6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6b344f309db7491d976dbfb78bc8a334'} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:21:05.694985 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-7c7c86f6-83ed-41bc-b6eb-986c20aa91e6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6b344f309db7491d976dbfb78bc8a334'} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:21:05.695459 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-7c7c86f6-83ed-41bc-b6eb-986c20aa91e6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6b344f309db7491d976dbfb78bc8a334'} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:21:05.709807 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 867/1726] 10.222.0.18 () {58 vars in 958 bytes} [Tue Sep 13 03:21:05 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 267 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 03:21:05.711056 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a3e9b2a-b8fe-4921-8776-a1c9ff3e4efa None None] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:21:05.725980 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-7c7c86f6-83ed-41bc-b6eb-986c20aa91e6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6b344f309db7491d976dbfb78bc8a334'} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:21:05.727113 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a3e9b2a-b8fe-4921-8776-a1c9ff3e4efa None None] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:21:05.732642 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-215e9627-70fb-49ab-b572-2c9b852e4d75 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76c5181fc7be45279ac00017bb4640b3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:21:05.733436 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-215e9627-70fb-49ab-b572-2c9b852e4d75 admin admin] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:21:05.733861 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-215e9627-70fb-49ab-b572-2c9b852e4d75 admin admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:21:05.734276 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-215e9627-70fb-49ab-b572-2c9b852e4d75 admin admin] PATH_INFO: `/v3/projects` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:21:05.735916 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-215e9627-70fb-49ab-b572-2c9b852e4d75 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:21:05.742397 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-215e9627-70fb-49ab-b572-2c9b852e4d75 admin admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:21:05.748149 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-7c7c86f6-83ed-41bc-b6eb-986c20aa91e6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6b344f309db7491d976dbfb78bc8a334'} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:21:05.763623 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 860/1727] 10.222.0.18 () {58 vars in 1224 bytes} [Tue Sep 13 03:21:05 2022] DELETE /identity/v3/users/6b344f309db7491d976dbfb78bc8a334 => generated 0 bytes in 387 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 13 03:21:05.771184 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61019e9f-1ceb-47aa-aa87-4c6b40f086c4 None None] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:21:05.771184 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61019e9f-1ceb-47aa-aa87-4c6b40f086c4 None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:21:05.771184 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61019e9f-1ceb-47aa-aa87-4c6b40f086c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:21:05.828824 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 868/1728] 10.222.0.18 () {60 vars in 1153 bytes} [Tue Sep 13 03:21:05 2022] POST /identity/v3/projects => generated 381 bytes in 115 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 03:21:05.834900 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-215e9627-70fb-49ab-b572-2c9b852e4d75 admin admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:21:05.838425 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.auth.core [None req-61019e9f-1ceb-47aa-aa87-4c6b40f086c4 None None] MFA Rules not processed for user `49f280ab086b4610855c4ffa378517d9`. Rule list: `[]` (Enabled: `True`). {{(pid=81400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 03:21:05.839692 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-61019e9f-1ceb-47aa-aa87-4c6b40f086c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:21:05.842498 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-215e9627-70fb-49ab-b572-2c9b852e4d75 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:21:05.940197 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=85,keystone:INSERT=5,keystone:DELETE=10 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:21:05.995937 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-215e9627-70fb-49ab-b572-2c9b852e4d75 admin admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:21:06.006785 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b7e7ebd-3d60-4b65-b17e-9f67b75b12e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76c5181fc7be45279ac00017bb4640b3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:21:06.007414 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b7e7ebd-3d60-4b65-b17e-9f67b75b12e7 admin admin] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:21:06.008418 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b7e7ebd-3d60-4b65-b17e-9f67b75b12e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:21:06.008696 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b7e7ebd-3d60-4b65-b17e-9f67b75b12e7 admin admin] PATH_INFO: `/v3/users` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:21:06.009335 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b7e7ebd-3d60-4b65-b17e-9f67b75b12e7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:21:06.013064 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b7e7ebd-3d60-4b65-b17e-9f67b75b12e7 admin admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:21:06.040981 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 861/1729] 10.222.0.18 () {58 vars in 958 bytes} [Tue Sep 13 03:21:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 274 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 03:21:06.075590 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 869/1730] 10.222.0.18 () {60 vars in 1144 bytes} [Tue Sep 13 03:21:05 2022] POST /identity/v3/users => generated 297 bytes in 241 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 03:21:06.079770 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61019e9f-1ceb-47aa-aa87-4c6b40f086c4 None None] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:21:06.093540 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b7e7ebd-3d60-4b65-b17e-9f67b75b12e7 admin admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:21:06.098157 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-9b7e7ebd-3d60-4b65-b17e-9f67b75b12e7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:21:06.100677 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61019e9f-1ceb-47aa-aa87-4c6b40f086c4 None None] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:21:06.116387 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c2af45a-b9a0-4f63-9fe2-312209fb4728 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76c5181fc7be45279ac00017bb4640b3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:21:06.119360 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c2af45a-b9a0-4f63-9fe2-312209fb4728 admin admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:21:06.120768 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c2af45a-b9a0-4f63-9fe2-312209fb4728 admin admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:21:06.122402 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c2af45a-b9a0-4f63-9fe2-312209fb4728 admin admin] PATH_INFO: `/v3/roles` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:21:06.124929 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c2af45a-b9a0-4f63-9fe2-312209fb4728 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:21:06.130191 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c2af45a-b9a0-4f63-9fe2-312209fb4728 admin admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:21:06.187186 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 862/1731] 10.222.0.18 () {58 vars in 1122 bytes} [Tue Sep 13 03:21:06 2022] GET /identity/v3/roles => generated 1420 bytes in 105 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 13 03:21:06.189230 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c2af45a-b9a0-4f63-9fe2-312209fb4728 admin admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:21:06.232148 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c2af45a-b9a0-4f63-9fe2-312209fb4728 admin admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:21:06.258642 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a8ff3c9-69cf-4879-81ab-7f721900dee4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76c5181fc7be45279ac00017bb4640b3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:21:06.261715 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a8ff3c9-69cf-4879-81ab-7f721900dee4 admin admin] REQUEST_METHOD: `PUT` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:21:06.262509 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a8ff3c9-69cf-4879-81ab-7f721900dee4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:21:06.264028 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a8ff3c9-69cf-4879-81ab-7f721900dee4 admin admin] PATH_INFO: `/v3/projects/7b6773482882475a948589dc69981673/users/a3158b19be4f4405ac13624163daed37/roles/425d4b481c7846e0bc07e7e805d5b15f` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:21:06.335852 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b7e7ebd-3d60-4b65-b17e-9f67b75b12e7 admin admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:21:06.347642 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c814821-7936-4173-ab1c-98ff575790e7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdffbc851086441ebd01022d1e4c2609', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b9d94f6a68f7470d9494f112175620dc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:21:06.347894 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a8ff3c9-69cf-4879-81ab-7f721900dee4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7b6773482882475a948589dc69981673, user_id=a3158b19be4f4405ac13624163daed37, role_id=425d4b481c7846e0bc07e7e805d5b15f)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:21:06.348287 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c814821-7936-4173-ab1c-98ff575790e7 service neutron] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:21:06.348594 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c814821-7936-4173-ab1c-98ff575790e7 service neutron] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:21:06.348872 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c814821-7936-4173-ab1c-98ff575790e7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:21:06.357557 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a8ff3c9-69cf-4879-81ab-7f721900dee4 admin admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:21:06.372750 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c814821-7936-4173-ab1c-98ff575790e7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:21:06.374559 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c814821-7936-4173-ab1c-98ff575790e7 service neutron] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:21:06.393422 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 863/1732] 10.222.0.18 () {60 vars in 1483 bytes} [Tue Sep 13 03:21:06 2022] PUT /identity/v3/projects/7b6773482882475a948589dc69981673/users/a3158b19be4f4405ac13624163daed37/roles/425d4b481c7846e0bc07e7e805d5b15f => generated 0 bytes in 203 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 03:21:06.394572 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a8ff3c9-69cf-4879-81ab-7f721900dee4 admin admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:21:06.399730 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 870/1733] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Sep 13 03:21:06 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 307 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 03:21:06.569220 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a8ff3c9-69cf-4879-81ab-7f721900dee4 admin admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:21:06.580702 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95117ded-ba2c-460f-9f56-9a59e2d54057 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76c5181fc7be45279ac00017bb4640b3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:21:06.581743 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95117ded-ba2c-460f-9f56-9a59e2d54057 admin admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:21:06.582166 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95117ded-ba2c-460f-9f56-9a59e2d54057 admin admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:21:06.583795 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95117ded-ba2c-460f-9f56-9a59e2d54057 admin admin] PATH_INFO: `/v3/roles` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:21:06.584621 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95117ded-ba2c-460f-9f56-9a59e2d54057 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:21:06.587068 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95117ded-ba2c-460f-9f56-9a59e2d54057 admin admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:21:06.611327 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 864/1734] 10.222.0.18 () {58 vars in 1122 bytes} [Tue Sep 13 03:21:06 2022] GET /identity/v3/roles => generated 1420 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 13 03:21:06.616536 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c814821-7936-4173-ab1c-98ff575790e7 service neutron] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:21:06.633668 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c814821-7936-4173-ab1c-98ff575790e7 service neutron] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:21:06.658902 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55973b3e-5733-414d-b10a-d707ba1cd7b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76c5181fc7be45279ac00017bb4640b3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:21:06.670182 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55973b3e-5733-414d-b10a-d707ba1cd7b6 admin admin] REQUEST_METHOD: `PUT` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:21:06.672595 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55973b3e-5733-414d-b10a-d707ba1cd7b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:21:06.675299 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55973b3e-5733-414d-b10a-d707ba1cd7b6 admin admin] PATH_INFO: `/v3/projects/7b6773482882475a948589dc69981673/users/a3158b19be4f4405ac13624163daed37/roles/425d4b481c7846e0bc07e7e805d5b15f` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:21:06.691117 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55973b3e-5733-414d-b10a-d707ba1cd7b6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7b6773482882475a948589dc69981673, user_id=a3158b19be4f4405ac13624163daed37, role_id=425d4b481c7846e0bc07e7e805d5b15f)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:21:06.705677 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55973b3e-5733-414d-b10a-d707ba1cd7b6 admin admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:21:06.751359 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 871/1735] 10.222.0.18 () {60 vars in 1483 bytes} [Tue Sep 13 03:21:06 2022] PUT /identity/v3/projects/7b6773482882475a948589dc69981673/users/a3158b19be4f4405ac13624163daed37/roles/425d4b481c7846e0bc07e7e805d5b15f => generated 0 bytes in 135 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 03:21:06.763554 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc840fa0-7cf2-4445-90ff-ae54fb15d1f8 None None] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:21:06.763668 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc840fa0-7cf2-4445-90ff-ae54fb15d1f8 None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:21:06.763976 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc840fa0-7cf2-4445-90ff-ae54fb15d1f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:21:06.845525 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.auth.core [None req-fc840fa0-7cf2-4445-90ff-ae54fb15d1f8 None None] MFA Rules not processed for user `a3158b19be4f4405ac13624163daed37`. Rule list: `[]` (Enabled: `True`). {{(pid=81400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 03:21:06.846754 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-fc840fa0-7cf2-4445-90ff-ae54fb15d1f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:21:06.979021 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 865/1736] 10.222.0.18 () {58 vars in 958 bytes} [Tue Sep 13 03:21:06 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 217 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 03:21:06.988854 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55973b3e-5733-414d-b10a-d707ba1cd7b6 admin admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:21:06.991313 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-55973b3e-5733-414d-b10a-d707ba1cd7b6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:21:07.050214 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:21:07.107391 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d90ab2-2a98-40a3-aa9d-e0585cc33e9d None None] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:21:07.108453 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d90ab2-2a98-40a3-aa9d-e0585cc33e9d None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:21:07.110015 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d90ab2-2a98-40a3-aa9d-e0585cc33e9d None None] PATH_INFO: `/` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:21:07.110591 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 866/1737] 10.222.0.18 () {54 vars in 912 bytes} [Tue Sep 13 03:21:07 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 03:21:07.120924 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ebd9dd7-1432-4896-9ef2-e3870f70dcbd None None] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:21:07.121812 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ebd9dd7-1432-4896-9ef2-e3870f70dcbd None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:21:07.122226 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ebd9dd7-1432-4896-9ef2-e3870f70dcbd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:21:07.155125 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55973b3e-5733-414d-b10a-d707ba1cd7b6 admin admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:21:07.159250 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.auth.core [None req-3ebd9dd7-1432-4896-9ef2-e3870f70dcbd None None] MFA Rules not processed for user `cdffbc851086441ebd01022d1e4c2609`. Rule list: `[]` (Enabled: `True`). {{(pid=81400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 03:21:07.164171 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-3ebd9dd7-1432-4896-9ef2-e3870f70dcbd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:21:07.164869 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32d9a868-011e-4883-a830-93f556551c7d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '924ee3f608d34a18b0ddc4695f102a56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b9d94f6a68f7470d9494f112175620dc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:21:07.165713 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d9a868-011e-4883-a830-93f556551c7d service cinder] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:21:07.166257 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d9a868-011e-4883-a830-93f556551c7d service cinder] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:21:07.166695 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d9a868-011e-4883-a830-93f556551c7d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:21:07.184670 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:21:07.213706 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32d9a868-011e-4883-a830-93f556551c7d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:21:07.220836 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32d9a868-011e-4883-a830-93f556551c7d service cinder] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:21:07.262812 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 872/1738] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Sep 13 03:21:06 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 274 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 03:21:07.270978 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0483df1-fbaf-4ef1-9e80-1c45e9be67c8 None None] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:21:07.271267 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0483df1-fbaf-4ef1-9e80-1c45e9be67c8 None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:21:07.271669 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0483df1-fbaf-4ef1-9e80-1c45e9be67c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:21:07.362953 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.auth.core [None req-a0483df1-fbaf-4ef1-9e80-1c45e9be67c8 None None] MFA Rules not processed for user `616dd34b9fde4ed6b42ec22421ebb504`. Rule list: `[]` (Enabled: `True`). {{(pid=81399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 03:21:07.363957 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 867/1739] 10.222.0.18 () {58 vars in 1009 bytes} [Tue Sep 13 03:21:07 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 244 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 13 03:21:07.364448 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-a0483df1-fbaf-4ef1-9e80-1c45e9be67c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:21:07.404303 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ebd9dd7-1432-4896-9ef2-e3870f70dcbd None None] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:21:07.416302 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ebd9dd7-1432-4896-9ef2-e3870f70dcbd None None] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:21:07.436264 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-166b0158-c607-48a1-b4ee-e1b992c39b62 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdffbc851086441ebd01022d1e4c2609', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b9d94f6a68f7470d9494f112175620dc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:21:07.437087 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-166b0158-c607-48a1-b4ee-e1b992c39b62 service neutron] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:21:07.437525 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-166b0158-c607-48a1-b4ee-e1b992c39b62 service neutron] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:21:07.438061 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-166b0158-c607-48a1-b4ee-e1b992c39b62 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:21:07.463027 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-166b0158-c607-48a1-b4ee-e1b992c39b62 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:21:07.464368 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-166b0158-c607-48a1-b4ee-e1b992c39b62 service neutron] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:21:07.503632 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 868/1740] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Sep 13 03:21:07 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 13 03:21:07.523375 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 873/1741] 10.222.0.18 () {58 vars in 958 bytes} [Tue Sep 13 03:21:07 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 253 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 03:21:07.533270 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f828e8-5e8c-4134-ad70-384994a16257 None None] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:21:07.533484 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f828e8-5e8c-4134-ad70-384994a16257 None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:21:07.533764 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f828e8-5e8c-4134-ad70-384994a16257 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:21:07.588555 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.auth.core [None req-63f828e8-5e8c-4134-ad70-384994a16257 None None] MFA Rules not processed for user `616dd34b9fde4ed6b42ec22421ebb504`. Rule list: `[]` (Enabled: `True`). {{(pid=81400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 03:21:07.590021 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-63f828e8-5e8c-4134-ad70-384994a16257 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:21:07.602073 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 869/1742] 10.222.0.18 () {58 vars in 958 bytes} [Tue Sep 13 03:21:07 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 70 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 03:21:07.667249 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0483df1-fbaf-4ef1-9e80-1c45e9be67c8 None None] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:21:07.668844 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-a0483df1-fbaf-4ef1-9e80-1c45e9be67c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:21:07.865913 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63f828e8-5e8c-4134-ad70-384994a16257 None None] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:21:07.867251 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-63f828e8-5e8c-4134-ad70-384994a16257 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:21:07.965965 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0483df1-fbaf-4ef1-9e80-1c45e9be67c8 None None] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:21:07.993615 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70bba294-4a44-4efc-84f0-4b6e31b28b4a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7377f5bee4a49959182e1bd34bc97a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b9d94f6a68f7470d9494f112175620dc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:21:07.994821 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70bba294-4a44-4efc-84f0-4b6e31b28b4a service nova] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:21:07.994821 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70bba294-4a44-4efc-84f0-4b6e31b28b4a service nova] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:21:07.994821 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70bba294-4a44-4efc-84f0-4b6e31b28b4a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:21:08.029757 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70bba294-4a44-4efc-84f0-4b6e31b28b4a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:21:08.031162 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70bba294-4a44-4efc-84f0-4b6e31b28b4a service nova] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:21:08.103263 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 874/1743] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Sep 13 03:21:07 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 437 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 03:21:08.105917 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70bba294-4a44-4efc-84f0-4b6e31b28b4a service nova] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:21:08.111455 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-70bba294-4a44-4efc-84f0-4b6e31b28b4a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:21:08.142514 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63f828e8-5e8c-4134-ad70-384994a16257 None None] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:21:08.146928 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83afc636-224d-4176-a2fe-8773e090665e service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b99f1b2374d4d2f8db78306316180d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b9d94f6a68f7470d9494f112175620dc', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:21:08.147562 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83afc636-224d-4176-a2fe-8773e090665e service placement] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:21:08.147845 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83afc636-224d-4176-a2fe-8773e090665e service placement] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:21:08.148120 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83afc636-224d-4176-a2fe-8773e090665e service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:21:08.151554 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-83afc636-224d-4176-a2fe-8773e090665e service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:21:08.184364 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83afc636-224d-4176-a2fe-8773e090665e service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:21:08.185998 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83afc636-224d-4176-a2fe-8773e090665e service placement] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:21:08.239878 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 870/1744] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Sep 13 03:21:07 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 374 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 03:21:08.273268 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:21:08.399725 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70bba294-4a44-4efc-84f0-4b6e31b28b4a service nova] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:21:08.416936 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2536ffa0-fc72-469b-9ae6-047b74c73ef6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76c5181fc7be45279ac00017bb4640b3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:21:08.417601 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2536ffa0-fc72-469b-9ae6-047b74c73ef6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:21:08.417880 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2536ffa0-fc72-469b-9ae6-047b74c73ef6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:21:08.418149 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2536ffa0-fc72-469b-9ae6-047b74c73ef6 admin admin] PATH_INFO: `/v3/projects/abfd705df8b14fb08d55dcdf98c6d7cc` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:21:08.419518 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2536ffa0-fc72-469b-9ae6-047b74c73ef6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=abfd705df8b14fb08d55dcdf98c6d7cc)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:21:08.421715 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2536ffa0-fc72-469b-9ae6-047b74c73ef6 admin admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:21:08.440916 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83afc636-224d-4176-a2fe-8773e090665e service placement] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:21:08.456631 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83afc636-224d-4176-a2fe-8773e090665e service placement] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:21:08.479218 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70e681d3-0b0e-47cc-9307-badabac4969e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76c5181fc7be45279ac00017bb4640b3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:21:08.479874 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e681d3-0b0e-47cc-9307-badabac4969e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:21:08.480284 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e681d3-0b0e-47cc-9307-badabac4969e admin admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:21:08.480435 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e681d3-0b0e-47cc-9307-badabac4969e admin admin] PATH_INFO: `/v3/users/b640f179ec9247398eb7f26376dde7d4` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:21:08.485521 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70e681d3-0b0e-47cc-9307-badabac4969e admin admin] RBAC: Authorizing `identity:delete_user(user_id=b640f179ec9247398eb7f26376dde7d4)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:21:08.487520 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70e681d3-0b0e-47cc-9307-badabac4969e admin admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:21:08.539721 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-2536ffa0-fc72-469b-9ae6-047b74c73ef6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'abfd705df8b14fb08d55dcdf98c6d7cc', 'request_id': 'req-2536ffa0-fc72-469b-9ae6-047b74c73ef6'} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:21:08.552311 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-2536ffa0-fc72-469b-9ae6-047b74c73ef6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'abfd705df8b14fb08d55dcdf98c6d7cc', 'request_id': 'req-2536ffa0-fc72-469b-9ae6-047b74c73ef6'} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:21:08.560788 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-2536ffa0-fc72-469b-9ae6-047b74c73ef6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'abfd705df8b14fb08d55dcdf98c6d7cc', 'request_id': 'req-2536ffa0-fc72-469b-9ae6-047b74c73ef6'} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:21:08.592692 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-2536ffa0-fc72-469b-9ae6-047b74c73ef6 admin admin] The token cache is being invalidate because project abfd705df8b14fb08d55dcdf98c6d7cc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81399) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 03:21:08.593088 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-2536ffa0-fc72-469b-9ae6-047b74c73ef6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:21:08.599113 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-2536ffa0-fc72-469b-9ae6-047b74c73ef6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:21:08.604846 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 875/1745] 10.222.0.18 () {58 vars in 1233 bytes} [Tue Sep 13 03:21:08 2022] DELETE /identity/v3/projects/abfd705df8b14fb08d55dcdf98c6d7cc => generated 0 bytes in 497 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 03:21:08.606096 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2536ffa0-fc72-469b-9ae6-047b74c73ef6 admin admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:21:08.608156 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-2536ffa0-fc72-469b-9ae6-047b74c73ef6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:21:08.615415 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-70e681d3-0b0e-47cc-9307-badabac4969e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b640f179ec9247398eb7f26376dde7d4'} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:21:08.633389 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-70e681d3-0b0e-47cc-9307-badabac4969e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b640f179ec9247398eb7f26376dde7d4'} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:21:08.634162 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-70e681d3-0b0e-47cc-9307-badabac4969e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b640f179ec9247398eb7f26376dde7d4'} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:21:08.662397 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-70e681d3-0b0e-47cc-9307-badabac4969e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b640f179ec9247398eb7f26376dde7d4'} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:21:08.676723 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-70e681d3-0b0e-47cc-9307-badabac4969e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b640f179ec9247398eb7f26376dde7d4'} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:21:08.691722 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 871/1746] 10.222.0.18 () {58 vars in 1224 bytes} [Tue Sep 13 03:21:08 2022] DELETE /identity/v3/users/b640f179ec9247398eb7f26376dde7d4 => generated 0 bytes in 251 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 03:21:08.695110 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-738e33e2-128f-4386-b608-015f6ebe9dc9 None None] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:21:08.695463 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-738e33e2-128f-4386-b608-015f6ebe9dc9 None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:21:08.695743 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-738e33e2-128f-4386-b608-015f6ebe9dc9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:21:08.727841 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:21:08.782542 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.auth.core [None req-738e33e2-128f-4386-b608-015f6ebe9dc9 None None] MFA Rules not processed for user `b85258f01e1f436585eb6b9a4f3813b8`. Rule list: `[]` (Enabled: `True`). {{(pid=81400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 03:21:08.783680 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-738e33e2-128f-4386-b608-015f6ebe9dc9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:21:08.835483 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2536ffa0-fc72-469b-9ae6-047b74c73ef6 admin admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:21:08.957172 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 872/1747] 10.222.0.18 () {58 vars in 958 bytes} [Tue Sep 13 03:21:08 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 265 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 03:21:08.959952 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e1856a5-1041-4a9b-a893-cd9e0f966ee8 None None] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:21:08.961837 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e1856a5-1041-4a9b-a893-cd9e0f966ee8 None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:21:08.962642 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e1856a5-1041-4a9b-a893-cd9e0f966ee8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:21:09.013877 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c178595a-71bb-4dbf-8fd4-b21bd2e4a8f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76c5181fc7be45279ac00017bb4640b3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:21:09.014963 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c178595a-71bb-4dbf-8fd4-b21bd2e4a8f2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:21:09.015435 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c178595a-71bb-4dbf-8fd4-b21bd2e4a8f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:21:09.015779 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c178595a-71bb-4dbf-8fd4-b21bd2e4a8f2 admin admin] PATH_INFO: `/v3/users/e0d9caad4d6d4f72ad4807230776d96e` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:21:09.017297 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c178595a-71bb-4dbf-8fd4-b21bd2e4a8f2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e0d9caad4d6d4f72ad4807230776d96e)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:21:09.020861 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c178595a-71bb-4dbf-8fd4-b21bd2e4a8f2 admin admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:21:09.039541 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.auth.core [None req-0e1856a5-1041-4a9b-a893-cd9e0f966ee8 None None] MFA Rules not processed for user `49f280ab086b4610855c4ffa378517d9`. Rule list: `[]` (Enabled: `True`). {{(pid=81400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 03:21:09.041007 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:21:09.048181 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-0e1856a5-1041-4a9b-a893-cd9e0f966ee8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:21:09.059986 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 873/1748] 10.222.0.18 () {58 vars in 958 bytes} [Tue Sep 13 03:21:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 03:21:09.062028 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ed50b1-5d63-4bbb-95fc-fa7fdd9862d2 None None] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:21:09.062534 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ed50b1-5d63-4bbb-95fc-fa7fdd9862d2 None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:21:09.063116 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ed50b1-5d63-4bbb-95fc-fa7fdd9862d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:21:09.102519 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.auth.core [None req-75ed50b1-5d63-4bbb-95fc-fa7fdd9862d2 None None] MFA Rules not processed for user `b85258f01e1f436585eb6b9a4f3813b8`. Rule list: `[]` (Enabled: `True`). {{(pid=81400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 03:21:09.104715 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-75ed50b1-5d63-4bbb-95fc-fa7fdd9862d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:21:09.176114 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-c178595a-71bb-4dbf-8fd4-b21bd2e4a8f2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e0d9caad4d6d4f72ad4807230776d96e'} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:21:09.182372 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 874/1749] 10.222.0.18 () {58 vars in 958 bytes} [Tue Sep 13 03:21:09 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 03:21:09.183505 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75ed50b1-5d63-4bbb-95fc-fa7fdd9862d2 None None] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:21:09.185587 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-75ed50b1-5d63-4bbb-95fc-fa7fdd9862d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:21:09.211054 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-c178595a-71bb-4dbf-8fd4-b21bd2e4a8f2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e0d9caad4d6d4f72ad4807230776d96e'} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:21:09.272341 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-c178595a-71bb-4dbf-8fd4-b21bd2e4a8f2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e0d9caad4d6d4f72ad4807230776d96e'} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:21:09.273911 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-c178595a-71bb-4dbf-8fd4-b21bd2e4a8f2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e0d9caad4d6d4f72ad4807230776d96e'} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:21:09.291335 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-c178595a-71bb-4dbf-8fd4-b21bd2e4a8f2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e0d9caad4d6d4f72ad4807230776d96e'} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:21:09.313237 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 876/1750] 10.222.0.18 () {58 vars in 1224 bytes} [Tue Sep 13 03:21:08 2022] DELETE /identity/v3/users/e0d9caad4d6d4f72ad4807230776d96e => generated 0 bytes in 707 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 03:21:09.314166 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c178595a-71bb-4dbf-8fd4-b21bd2e4a8f2 admin admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:21:09.327601 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-c178595a-71bb-4dbf-8fd4-b21bd2e4a8f2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:21:09.376385 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75ed50b1-5d63-4bbb-95fc-fa7fdd9862d2 None None] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:21:09.657909 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3f7b818-ebc9-41f2-a68f-953d326d683b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdffbc851086441ebd01022d1e4c2609', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b9d94f6a68f7470d9494f112175620dc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:21:09.663929 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f7b818-ebc9-41f2-a68f-953d326d683b service neutron] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:21:09.664556 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f7b818-ebc9-41f2-a68f-953d326d683b service neutron] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:21:09.666732 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f7b818-ebc9-41f2-a68f-953d326d683b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:21:09.668559 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c178595a-71bb-4dbf-8fd4-b21bd2e4a8f2 admin admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:21:09.671012 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-c3f7b818-ebc9-41f2-a68f-953d326d683b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:21:09.704417 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fd27262-a925-4cab-b3ed-fdd6374e9d42 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7377f5bee4a49959182e1bd34bc97a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b9d94f6a68f7470d9494f112175620dc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:21:09.710077 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd27262-a925-4cab-b3ed-fdd6374e9d42 service nova] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:21:09.711679 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd27262-a925-4cab-b3ed-fdd6374e9d42 service nova] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:21:09.712125 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd27262-a925-4cab-b3ed-fdd6374e9d42 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:21:09.724597 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-7fd27262-a925-4cab-b3ed-fdd6374e9d42 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:21:09.871551 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:21:09.908110 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3f7b818-ebc9-41f2-a68f-953d326d683b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:21:09.909808 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3f7b818-ebc9-41f2-a68f-953d326d683b service neutron] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:21:09.961724 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 875/1751] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Sep 13 03:21:09 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 779 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 03:21:09.970449 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e7a31a-16ba-4695-a8a8-1d3128348cb1 None None] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:21:09.972248 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e7a31a-16ba-4695-a8a8-1d3128348cb1 None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:21:09.974467 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e7a31a-16ba-4695-a8a8-1d3128348cb1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:21:10.030076 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fd27262-a925-4cab-b3ed-fdd6374e9d42 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:21:10.035442 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fd27262-a925-4cab-b3ed-fdd6374e9d42 service nova] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:21:10.053359 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 877/1752] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Sep 13 03:21:09 2022] GET /identity/v3/auth/tokens => generated 3270 bytes in 740 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 03:21:10.078296 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.auth.core [None req-82e7a31a-16ba-4695-a8a8-1d3128348cb1 None None] MFA Rules not processed for user `4f76a7c6fe9347b0bea5f701bccb6150`. Rule list: `[]` (Enabled: `True`). {{(pid=81400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 03:21:10.079786 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-82e7a31a-16ba-4695-a8a8-1d3128348cb1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:21:10.392235 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 876/1753] 10.222.0.18 () {58 vars in 958 bytes} [Tue Sep 13 03:21:09 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 426 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 03:21:10.865017 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fd27262-a925-4cab-b3ed-fdd6374e9d42 service nova] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:21:10.866847 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-7fd27262-a925-4cab-b3ed-fdd6374e9d42 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:21:10.905889 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fd27262-a925-4cab-b3ed-fdd6374e9d42 service nova] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:21:10.924218 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4c3e772-d119-4332-9c20-f21ee249c6a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76c5181fc7be45279ac00017bb4640b3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:21:10.924896 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c3e772-d119-4332-9c20-f21ee249c6a0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:21:10.925178 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c3e772-d119-4332-9c20-f21ee249c6a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:21:10.925460 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c3e772-d119-4332-9c20-f21ee249c6a0 admin admin] PATH_INFO: `/v3/projects/49ef5d64ceef40229724e226a98bf998` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:21:10.933503 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4c3e772-d119-4332-9c20-f21ee249c6a0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=49ef5d64ceef40229724e226a98bf998)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:21:10.943976 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4c3e772-d119-4332-9c20-f21ee249c6a0 admin admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:21:11.041782 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:21:11.047782 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82e7a31a-16ba-4695-a8a8-1d3128348cb1 None None] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:21:11.049873 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-82e7a31a-16ba-4695-a8a8-1d3128348cb1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:21:11.092938 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82e7a31a-16ba-4695-a8a8-1d3128348cb1 None None] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:21:11.119673 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-b4c3e772-d119-4332-9c20-f21ee249c6a0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '49ef5d64ceef40229724e226a98bf998', 'request_id': 'req-b4c3e772-d119-4332-9c20-f21ee249c6a0'} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:21:11.164398 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-b4c3e772-d119-4332-9c20-f21ee249c6a0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '49ef5d64ceef40229724e226a98bf998', 'request_id': 'req-b4c3e772-d119-4332-9c20-f21ee249c6a0'} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:21:11.214833 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-b4c3e772-d119-4332-9c20-f21ee249c6a0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '49ef5d64ceef40229724e226a98bf998', 'request_id': 'req-b4c3e772-d119-4332-9c20-f21ee249c6a0'} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:21:11.269655 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c36174c-79d9-46dc-b979-5a64c79abdef service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7377f5bee4a49959182e1bd34bc97a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b9d94f6a68f7470d9494f112175620dc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:21:11.271894 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c36174c-79d9-46dc-b979-5a64c79abdef service nova] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:21:11.272399 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c36174c-79d9-46dc-b979-5a64c79abdef service nova] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:21:11.272874 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c36174c-79d9-46dc-b979-5a64c79abdef service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:21:11.296657 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-b4c3e772-d119-4332-9c20-f21ee249c6a0 admin admin] The token cache is being invalidate because project 49ef5d64ceef40229724e226a98bf998 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81399) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 03:21:11.296956 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-b4c3e772-d119-4332-9c20-f21ee249c6a0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:21:11.303491 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-b4c3e772-d119-4332-9c20-f21ee249c6a0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:21:11.309812 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 878/1754] 10.222.0.18 () {58 vars in 1233 bytes} [Tue Sep 13 03:21:10 2022] DELETE /identity/v3/projects/49ef5d64ceef40229724e226a98bf998 => generated 0 bytes in 447 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 03:21:11.314057 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4c3e772-d119-4332-9c20-f21ee249c6a0 admin admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:21:11.332602 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-b4c3e772-d119-4332-9c20-f21ee249c6a0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:21:11.414875 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c36174c-79d9-46dc-b979-5a64c79abdef service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:21:11.418571 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c36174c-79d9-46dc-b979-5a64c79abdef service nova] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:21:11.451129 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 877/1755] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Sep 13 03:21:11 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 404 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 03:21:11.455009 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca86f843-e674-4057-a32d-4f45f28d085a None None] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:21:11.455365 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca86f843-e674-4057-a32d-4f45f28d085a None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:21:11.455644 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca86f843-e674-4057-a32d-4f45f28d085a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:21:11.520982 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.auth.core [None req-ca86f843-e674-4057-a32d-4f45f28d085a None None] MFA Rules not processed for user `49f280ab086b4610855c4ffa378517d9`. Rule list: `[]` (Enabled: `True`). {{(pid=81400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 03:21:11.525808 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-ca86f843-e674-4057-a32d-4f45f28d085a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:21:11.554957 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:21:11.567954 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4c3e772-d119-4332-9c20-f21ee249c6a0 admin admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:21:11.584245 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14443c42-5670-436f-ab35-303693d08865 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76c5181fc7be45279ac00017bb4640b3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:21:11.584935 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14443c42-5670-436f-ab35-303693d08865 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:21:11.585220 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14443c42-5670-436f-ab35-303693d08865 admin admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:21:11.585676 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14443c42-5670-436f-ab35-303693d08865 admin admin] PATH_INFO: `/v3/projects/11c5a74585df4cd195f06c9b2aadc3f9` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:21:11.588672 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14443c42-5670-436f-ab35-303693d08865 admin admin] RBAC: Authorizing `identity:delete_project(project_id=11c5a74585df4cd195f06c9b2aadc3f9)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:21:11.590638 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14443c42-5670-436f-ab35-303693d08865 admin admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:21:11.700874 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-14443c42-5670-436f-ab35-303693d08865 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '11c5a74585df4cd195f06c9b2aadc3f9', 'request_id': 'req-14443c42-5670-436f-ab35-303693d08865'} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:21:11.717196 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-14443c42-5670-436f-ab35-303693d08865 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '11c5a74585df4cd195f06c9b2aadc3f9', 'request_id': 'req-14443c42-5670-436f-ab35-303693d08865'} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:21:11.736062 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-14443c42-5670-436f-ab35-303693d08865 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '11c5a74585df4cd195f06c9b2aadc3f9', 'request_id': 'req-14443c42-5670-436f-ab35-303693d08865'} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:21:11.758428 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-14443c42-5670-436f-ab35-303693d08865 admin admin] The token cache is being invalidate because project 11c5a74585df4cd195f06c9b2aadc3f9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81399) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 03:21:11.758844 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-14443c42-5670-436f-ab35-303693d08865 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:21:11.759707 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-14443c42-5670-436f-ab35-303693d08865 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:21:11.762292 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 879/1756] 10.222.0.18 () {58 vars in 1233 bytes} [Tue Sep 13 03:21:11 2022] DELETE /identity/v3/projects/11c5a74585df4cd195f06c9b2aadc3f9 => generated 0 bytes in 449 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 03:21:11.766217 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14443c42-5670-436f-ab35-303693d08865 admin admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:21:11.770505 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-14443c42-5670-436f-ab35-303693d08865 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:21:11.774242 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 878/1757] 10.222.0.18 () {58 vars in 958 bytes} [Tue Sep 13 03:21:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 322 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 03:21:11.802345 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98dc4748-7a59-4af7-a6cf-dbab15c24494 None None] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:21:11.802644 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98dc4748-7a59-4af7-a6cf-dbab15c24494 None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:21:11.802906 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98dc4748-7a59-4af7-a6cf-dbab15c24494 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:21:11.845427 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.auth.core [None req-98dc4748-7a59-4af7-a6cf-dbab15c24494 None None] MFA Rules not processed for user `49f280ab086b4610855c4ffa378517d9`. Rule list: `[]` (Enabled: `True`). {{(pid=81400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 03:21:11.846815 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-98dc4748-7a59-4af7-a6cf-dbab15c24494 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:21:11.913469 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=85,keystone:DELETE=13,keystone:INSERT=2 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:21:11.954741 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14443c42-5670-436f-ab35-303693d08865 admin admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:21:11.967555 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f5e4731-39fe-465e-8a80-8927f16d50f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76c5181fc7be45279ac00017bb4640b3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:21:11.968234 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f5e4731-39fe-465e-8a80-8927f16d50f0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:21:11.968587 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f5e4731-39fe-465e-8a80-8927f16d50f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:21:11.968872 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f5e4731-39fe-465e-8a80-8927f16d50f0 admin admin] PATH_INFO: `/v3/users/a3158b19be4f4405ac13624163daed37` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:21:11.973336 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f5e4731-39fe-465e-8a80-8927f16d50f0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a3158b19be4f4405ac13624163daed37)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:21:11.976205 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f5e4731-39fe-465e-8a80-8927f16d50f0 admin admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:21:12.115060 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 879/1758] 10.222.0.18 () {58 vars in 958 bytes} [Tue Sep 13 03:21:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 313 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 03:21:12.116784 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9da85171-b9e9-4be0-b472-d2425359fc66 None None] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:21:12.117084 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9da85171-b9e9-4be0-b472-d2425359fc66 None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:21:12.117193 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-4f5e4731-39fe-465e-8a80-8927f16d50f0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a3158b19be4f4405ac13624163daed37'} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:21:12.117588 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9da85171-b9e9-4be0-b472-d2425359fc66 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:21:12.132300 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-4f5e4731-39fe-465e-8a80-8927f16d50f0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a3158b19be4f4405ac13624163daed37'} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:21:12.165459 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-4f5e4731-39fe-465e-8a80-8927f16d50f0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a3158b19be4f4405ac13624163daed37'} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:21:12.169435 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-4f5e4731-39fe-465e-8a80-8927f16d50f0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a3158b19be4f4405ac13624163daed37'} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:21:12.181122 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.auth.core [None req-9da85171-b9e9-4be0-b472-d2425359fc66 None None] MFA Rules not processed for user `b85258f01e1f436585eb6b9a4f3813b8`. Rule list: `[]` (Enabled: `True`). {{(pid=81400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 03:21:12.184264 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-9da85171-b9e9-4be0-b472-d2425359fc66 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:21:12.212652 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-4f5e4731-39fe-465e-8a80-8927f16d50f0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a3158b19be4f4405ac13624163daed37'} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:21:12.260361 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 880/1759] 10.222.0.18 () {58 vars in 1224 bytes} [Tue Sep 13 03:21:11 2022] DELETE /identity/v3/users/a3158b19be4f4405ac13624163daed37 => generated 0 bytes in 495 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 13 03:21:12.267631 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f5e4731-39fe-465e-8a80-8927f16d50f0 admin admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:21:12.277231 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-4f5e4731-39fe-465e-8a80-8927f16d50f0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:21:12.359890 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 880/1760] 10.222.0.18 () {58 vars in 958 bytes} [Tue Sep 13 03:21:12 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 246 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 03:21:12.545984 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f5e4731-39fe-465e-8a80-8927f16d50f0 admin admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:21:12.571387 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9be98697-65d5-4952-b316-07af11a43f96 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76c5181fc7be45279ac00017bb4640b3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:21:12.572243 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be98697-65d5-4952-b316-07af11a43f96 admin admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:21:12.572685 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be98697-65d5-4952-b316-07af11a43f96 admin admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:21:12.573017 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be98697-65d5-4952-b316-07af11a43f96 admin admin] PATH_INFO: `/v3/domains` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:21:12.573900 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9be98697-65d5-4952-b316-07af11a43f96 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 03:21:12.574431 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9be98697-65d5-4952-b316-07af11a43f96 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:21:12.577420 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9be98697-65d5-4952-b316-07af11a43f96 admin admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:21:12.592056 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 881/1761] 10.222.0.18 () {58 vars in 1153 bytes} [Tue Sep 13 03:21:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 331 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 03:21:12.596504 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9da85171-b9e9-4be0-b472-d2425359fc66 None None] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:21:12.610564 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9da85171-b9e9-4be0-b472-d2425359fc66 None None] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:21:12.626882 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b411bc84-a539-4169-bf82-900dda5b39b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76c5181fc7be45279ac00017bb4640b3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:21:12.632313 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b411bc84-a539-4169-bf82-900dda5b39b6 admin admin] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:21:12.632313 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b411bc84-a539-4169-bf82-900dda5b39b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:21:12.632532 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b411bc84-a539-4169-bf82-900dda5b39b6 admin admin] PATH_INFO: `/v3/projects` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:21:12.633010 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b411bc84-a539-4169-bf82-900dda5b39b6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:21:12.640015 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b411bc84-a539-4169-bf82-900dda5b39b6 admin admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:21:12.763619 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 881/1762] 10.222.0.18 () {60 vars in 1153 bytes} [Tue Sep 13 03:21:12 2022] POST /identity/v3/projects => generated 379 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 03:21:12.768378 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9be98697-65d5-4952-b316-07af11a43f96 admin admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:21:12.789342 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9be98697-65d5-4952-b316-07af11a43f96 admin admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:21:12.975250 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:21:12.998792 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cb57dc2-7ed0-4534-ade7-c3603090b67d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76c5181fc7be45279ac00017bb4640b3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:21:12.999592 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cb57dc2-7ed0-4534-ade7-c3603090b67d admin admin] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:21:12.999881 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cb57dc2-7ed0-4534-ade7-c3603090b67d admin admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:21:13.000160 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cb57dc2-7ed0-4534-ade7-c3603090b67d admin admin] PATH_INFO: `/v3/users` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:21:13.000810 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cb57dc2-7ed0-4534-ade7-c3603090b67d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:21:13.006227 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cb57dc2-7ed0-4534-ade7-c3603090b67d admin admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:21:13.064307 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 882/1763] 10.222.0.18 () {60 vars in 1144 bytes} [Tue Sep 13 03:21:12 2022] POST /identity/v3/users => generated 290 bytes in 297 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 03:21:13.068379 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b411bc84-a539-4169-bf82-900dda5b39b6 admin admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:21:13.083059 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b411bc84-a539-4169-bf82-900dda5b39b6 admin admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:21:13.101746 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35cfeda6-b245-4062-a553-7ae38e9d9adb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76c5181fc7be45279ac00017bb4640b3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:21:13.102380 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35cfeda6-b245-4062-a553-7ae38e9d9adb admin admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:21:13.102671 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35cfeda6-b245-4062-a553-7ae38e9d9adb admin admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:21:13.102941 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35cfeda6-b245-4062-a553-7ae38e9d9adb admin admin] PATH_INFO: `/v3/roles` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:21:13.103589 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35cfeda6-b245-4062-a553-7ae38e9d9adb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:21:13.105180 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35cfeda6-b245-4062-a553-7ae38e9d9adb admin admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:21:13.130293 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 882/1764] 10.222.0.18 () {58 vars in 1122 bytes} [Tue Sep 13 03:21:13 2022] GET /identity/v3/roles => generated 1420 bytes in 62 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 13 03:21:13.134898 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cb57dc2-7ed0-4534-ade7-c3603090b67d admin admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:21:13.146354 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cb57dc2-7ed0-4534-ade7-c3603090b67d admin admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:21:13.164669 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dcce1a5-5024-4495-88e2-3ecc1849c76c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76c5181fc7be45279ac00017bb4640b3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:21:13.165599 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dcce1a5-5024-4495-88e2-3ecc1849c76c admin admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:21:13.165989 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dcce1a5-5024-4495-88e2-3ecc1849c76c admin admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:21:13.166367 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dcce1a5-5024-4495-88e2-3ecc1849c76c admin admin] PATH_INFO: `/v3/roles` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:21:13.167309 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dcce1a5-5024-4495-88e2-3ecc1849c76c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:21:13.169221 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dcce1a5-5024-4495-88e2-3ecc1849c76c admin admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:21:13.173648 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35cfeda6-b245-4062-a553-7ae38e9d9adb admin admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:21:13.175893 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-35cfeda6-b245-4062-a553-7ae38e9d9adb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:21:13.209123 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35cfeda6-b245-4062-a553-7ae38e9d9adb admin admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:21:13.224440 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 883/1765] 10.222.0.18 () {58 vars in 1122 bytes} [Tue Sep 13 03:21:13 2022] GET /identity/v3/roles => generated 1420 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 03:21:13.225804 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dcce1a5-5024-4495-88e2-3ecc1849c76c admin admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:21:13.228155 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f519c266-40a1-4d4d-b31e-9b4ab6731edd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76c5181fc7be45279ac00017bb4640b3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:21:13.232267 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f519c266-40a1-4d4d-b31e-9b4ab6731edd admin admin] REQUEST_METHOD: `DELETE` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:21:13.233811 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f519c266-40a1-4d4d-b31e-9b4ab6731edd admin admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:21:13.235059 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f519c266-40a1-4d4d-b31e-9b4ab6731edd admin admin] PATH_INFO: `/v3/projects/7b6773482882475a948589dc69981673` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:21:13.241396 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f519c266-40a1-4d4d-b31e-9b4ab6731edd admin admin] RBAC: Authorizing `identity:delete_project(project_id=7b6773482882475a948589dc69981673)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:21:13.244932 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f519c266-40a1-4d4d-b31e-9b4ab6731edd admin admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:21:13.267037 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dcce1a5-5024-4495-88e2-3ecc1849c76c admin admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:21:13.281678 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-323e655f-0161-4edb-aa4c-03665a0adf6e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76c5181fc7be45279ac00017bb4640b3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:21:13.282797 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-323e655f-0161-4edb-aa4c-03665a0adf6e admin admin] REQUEST_METHOD: `PUT` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:21:13.283941 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-323e655f-0161-4edb-aa4c-03665a0adf6e admin admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:21:13.284532 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-323e655f-0161-4edb-aa4c-03665a0adf6e admin admin] PATH_INFO: `/v3/projects/7c86a7b399d24c1ab23c826eaddee1d4/users/76d7cc0174ce4e0ab993d6b1f1c466a6/roles/1eb7a5b5203945c29e6d22d653f47847` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:21:13.310840 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:21:13.320787 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-323e655f-0161-4edb-aa4c-03665a0adf6e admin admin] RBAC: Authorizing `identity:create_grant(project_id=7c86a7b399d24c1ab23c826eaddee1d4, user_id=76d7cc0174ce4e0ab993d6b1f1c466a6, role_id=1eb7a5b5203945c29e6d22d653f47847)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:21:13.333232 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-323e655f-0161-4edb-aa4c-03665a0adf6e admin admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:21:13.378301 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-f519c266-40a1-4d4d-b31e-9b4ab6731edd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7b6773482882475a948589dc69981673', 'request_id': 'req-f519c266-40a1-4d4d-b31e-9b4ab6731edd'} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:21:13.379821 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-f519c266-40a1-4d4d-b31e-9b4ab6731edd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7b6773482882475a948589dc69981673', 'request_id': 'req-f519c266-40a1-4d4d-b31e-9b4ab6731edd'} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:21:13.382806 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 884/1766] 10.222.0.18 () {60 vars in 1483 bytes} [Tue Sep 13 03:21:13 2022] PUT /identity/v3/projects/7c86a7b399d24c1ab23c826eaddee1d4/users/76d7cc0174ce4e0ab993d6b1f1c466a6/roles/1eb7a5b5203945c29e6d22d653f47847 => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 03:21:13.385375 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05281fae-3284-4985-90e9-639393871d4c None None] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:21:13.387132 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05281fae-3284-4985-90e9-639393871d4c None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:21:13.387441 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05281fae-3284-4985-90e9-639393871d4c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:21:13.411719 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-f519c266-40a1-4d4d-b31e-9b4ab6731edd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7b6773482882475a948589dc69981673', 'request_id': 'req-f519c266-40a1-4d4d-b31e-9b4ab6731edd'} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:21:13.455707 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.auth.core [None req-05281fae-3284-4985-90e9-639393871d4c None None] MFA Rules not processed for user `76d7cc0174ce4e0ab993d6b1f1c466a6`. Rule list: `[]` (Enabled: `True`). {{(pid=81399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 03:21:13.462274 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-05281fae-3284-4985-90e9-639393871d4c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:21:13.468276 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-f519c266-40a1-4d4d-b31e-9b4ab6731edd admin admin] The token cache is being invalidate because project 7b6773482882475a948589dc69981673 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81400) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 03:21:13.468691 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-f519c266-40a1-4d4d-b31e-9b4ab6731edd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:21:13.479150 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-f519c266-40a1-4d4d-b31e-9b4ab6731edd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:21:13.482142 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 883/1767] 10.222.0.18 () {58 vars in 1233 bytes} [Tue Sep 13 03:21:13 2022] DELETE /identity/v3/projects/7b6773482882475a948589dc69981673 => generated 0 bytes in 309 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 13 03:21:13.486796 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58bdcdae-7f08-4472-815b-bbfa6e0d17cb None None] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:21:13.487086 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58bdcdae-7f08-4472-815b-bbfa6e0d17cb None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:21:13.487363 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58bdcdae-7f08-4472-815b-bbfa6e0d17cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:21:13.539626 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.auth.core [None req-58bdcdae-7f08-4472-815b-bbfa6e0d17cb None None] MFA Rules not processed for user `49f280ab086b4610855c4ffa378517d9`. Rule list: `[]` (Enabled: `True`). {{(pid=81400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 03:21:13.540856 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-58bdcdae-7f08-4472-815b-bbfa6e0d17cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:21:13.659644 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 885/1768] 10.222.0.18 () {58 vars in 958 bytes} [Tue Sep 13 03:21:13 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 275 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 03:21:13.668872 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db51eb45-df19-4107-b13a-d7424ba98668 None None] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:21:13.669178 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db51eb45-df19-4107-b13a-d7424ba98668 None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:21:13.669478 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db51eb45-df19-4107-b13a-d7424ba98668 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:21:13.685015 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 884/1769] 10.222.0.18 () {58 vars in 958 bytes} [Tue Sep 13 03:21:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 199 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 13 03:21:13.687305 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58bdcdae-7f08-4472-815b-bbfa6e0d17cb None None] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:21:13.693083 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-58bdcdae-7f08-4472-815b-bbfa6e0d17cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:21:13.733677 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58bdcdae-7f08-4472-815b-bbfa6e0d17cb None None] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:21:13.746437 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deb2322d-5bb2-4532-9db1-1df53d9d5078 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76c5181fc7be45279ac00017bb4640b3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:21:13.747732 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb2322d-5bb2-4532-9db1-1df53d9d5078 admin admin] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:21:13.747732 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb2322d-5bb2-4532-9db1-1df53d9d5078 admin admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:21:13.747953 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb2322d-5bb2-4532-9db1-1df53d9d5078 admin admin] PATH_INFO: `/v3/projects` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:21:13.748838 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deb2322d-5bb2-4532-9db1-1df53d9d5078 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:21:13.751419 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deb2322d-5bb2-4532-9db1-1df53d9d5078 admin admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:21:13.810957 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 885/1770] 10.222.0.18 () {60 vars in 1153 bytes} [Tue Sep 13 03:21:13 2022] POST /identity/v3/projects => generated 383 bytes in 125 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 03:21:13.812882 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac9e82d-5c6e-40c5-b691-f3e2d3685f2b None None] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:21:13.813299 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac9e82d-5c6e-40c5-b691-f3e2d3685f2b None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:21:13.813859 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac9e82d-5c6e-40c5-b691-f3e2d3685f2b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:21:13.878622 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.auth.core [None req-db51eb45-df19-4107-b13a-d7424ba98668 None None] MFA Rules not processed for user `4c2d20fb54a54ecc99b47f237be4917c`. Rule list: `[]` (Enabled: `True`). {{(pid=81399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 03:21:13.880142 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-db51eb45-df19-4107-b13a-d7424ba98668 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:21:13.889605 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.auth.core [None req-9ac9e82d-5c6e-40c5-b691-f3e2d3685f2b None None] MFA Rules not processed for user `49f280ab086b4610855c4ffa378517d9`. Rule list: `[]` (Enabled: `True`). {{(pid=81400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 03:21:13.893409 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-9ac9e82d-5c6e-40c5-b691-f3e2d3685f2b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:21:14.038379 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 886/1771] 10.222.0.18 () {58 vars in 1009 bytes} [Tue Sep 13 03:21:13 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 379 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 03:21:14.039880 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db51eb45-df19-4107-b13a-d7424ba98668 None None] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:21:14.045379 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 886/1772] 10.222.0.18 () {58 vars in 958 bytes} [Tue Sep 13 03:21:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 234 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 03:21:14.051192 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ac9e82d-5c6e-40c5-b691-f3e2d3685f2b None None] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:21:14.062131 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db51eb45-df19-4107-b13a-d7424ba98668 None None] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:21:14.087541 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cac9d463-d751-4797-9aa7-5e03196e1aba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76c5181fc7be45279ac00017bb4640b3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:21:14.088181 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac9d463-d751-4797-9aa7-5e03196e1aba admin admin] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:21:14.088465 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac9d463-d751-4797-9aa7-5e03196e1aba admin admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:21:14.088739 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac9d463-d751-4797-9aa7-5e03196e1aba admin admin] PATH_INFO: `/v3/users` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:21:14.099163 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cac9d463-d751-4797-9aa7-5e03196e1aba admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:21:14.099163 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cac9d463-d751-4797-9aa7-5e03196e1aba admin admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:21:14.111007 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ac9e82d-5c6e-40c5-b691-f3e2d3685f2b None None] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:21:14.116961 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dc30c5c-b041-46f0-a48d-008f90ede2ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76c5181fc7be45279ac00017bb4640b3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:21:14.117717 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc30c5c-b041-46f0-a48d-008f90ede2ed admin admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:21:14.119421 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc30c5c-b041-46f0-a48d-008f90ede2ed admin admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:21:14.119961 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc30c5c-b041-46f0-a48d-008f90ede2ed admin admin] PATH_INFO: `/v3/domains` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:21:14.121434 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dc30c5c-b041-46f0-a48d-008f90ede2ed admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 03:21:14.123042 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dc30c5c-b041-46f0-a48d-008f90ede2ed admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:21:14.128868 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dc30c5c-b041-46f0-a48d-008f90ede2ed admin admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:21:14.132866 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:21:14.155508 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 887/1773] 10.222.0.18 () {60 vars in 1144 bytes} [Tue Sep 13 03:21:14 2022] POST /identity/v3/users => generated 298 bytes in 116 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 03:21:14.159697 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cac9d463-d751-4797-9aa7-5e03196e1aba admin admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:21:14.172679 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 887/1774] 10.222.0.18 () {58 vars in 1153 bytes} [Tue Sep 13 03:21:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 123 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 03:21:14.179662 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cac9d463-d751-4797-9aa7-5e03196e1aba admin admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:21:14.181181 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dc30c5c-b041-46f0-a48d-008f90ede2ed admin admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:21:14.185868 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbd873cf-6b7a-4040-969f-1636fe9d4b78 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76c5181fc7be45279ac00017bb4640b3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:21:14.186953 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd873cf-6b7a-4040-969f-1636fe9d4b78 admin admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:21:14.187235 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd873cf-6b7a-4040-969f-1636fe9d4b78 admin admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:21:14.187514 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd873cf-6b7a-4040-969f-1636fe9d4b78 admin admin] PATH_INFO: `/v3/roles` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:21:14.188278 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbd873cf-6b7a-4040-969f-1636fe9d4b78 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:21:14.190268 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbd873cf-6b7a-4040-969f-1636fe9d4b78 admin admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:21:14.218513 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 888/1775] 10.222.0.18 () {58 vars in 1122 bytes} [Tue Sep 13 03:21:14 2022] GET /identity/v3/roles => generated 1420 bytes in 59 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 13 03:21:14.222630 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbd873cf-6b7a-4040-969f-1636fe9d4b78 admin admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:21:14.224784 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dc30c5c-b041-46f0-a48d-008f90ede2ed admin admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:21:14.242593 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-837c92ef-c534-4ee5-a9c8-f8ed2808cc6b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76c5181fc7be45279ac00017bb4640b3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:21:14.243636 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-837c92ef-c534-4ee5-a9c8-f8ed2808cc6b admin admin] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:21:14.244177 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-837c92ef-c534-4ee5-a9c8-f8ed2808cc6b admin admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:21:14.244979 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-837c92ef-c534-4ee5-a9c8-f8ed2808cc6b admin admin] PATH_INFO: `/v3/projects` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:21:14.247480 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-837c92ef-c534-4ee5-a9c8-f8ed2808cc6b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:21:14.253196 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-837c92ef-c534-4ee5-a9c8-f8ed2808cc6b admin admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:21:14.273279 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbd873cf-6b7a-4040-969f-1636fe9d4b78 admin admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:21:14.306547 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:DELETE=7,keystone:SELECT=91,keystone:INSERT=2 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:21:14.311269 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46549359-73e5-4bb5-98c9-fab66c6a86ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76c5181fc7be45279ac00017bb4640b3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:21:14.313956 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46549359-73e5-4bb5-98c9-fab66c6a86ae admin admin] REQUEST_METHOD: `PUT` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:21:14.315553 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46549359-73e5-4bb5-98c9-fab66c6a86ae admin admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:21:14.316054 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46549359-73e5-4bb5-98c9-fab66c6a86ae admin admin] PATH_INFO: `/v3/projects/30a882ea0e7b46039e44abe7e9d957b1/users/7cbbcd0d1cba4052baf9c7d58370d5f4/roles/425d4b481c7846e0bc07e7e805d5b15f` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:21:14.357071 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46549359-73e5-4bb5-98c9-fab66c6a86ae admin admin] RBAC: Authorizing `identity:create_grant(project_id=30a882ea0e7b46039e44abe7e9d957b1, user_id=7cbbcd0d1cba4052baf9c7d58370d5f4, role_id=425d4b481c7846e0bc07e7e805d5b15f)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:21:14.362693 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 888/1776] 10.222.0.18 () {60 vars in 1153 bytes} [Tue Sep 13 03:21:14 2022] POST /identity/v3/projects => generated 381 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 03:21:14.362870 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46549359-73e5-4bb5-98c9-fab66c6a86ae admin admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:21:14.365766 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-837c92ef-c534-4ee5-a9c8-f8ed2808cc6b admin admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:21:14.372078 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-837c92ef-c534-4ee5-a9c8-f8ed2808cc6b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:21:14.404097 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 889/1777] 10.222.0.18 () {60 vars in 1483 bytes} [Tue Sep 13 03:21:14 2022] PUT /identity/v3/projects/30a882ea0e7b46039e44abe7e9d957b1/users/7cbbcd0d1cba4052baf9c7d58370d5f4/roles/425d4b481c7846e0bc07e7e805d5b15f => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 03:21:14.405805 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46549359-73e5-4bb5-98c9-fab66c6a86ae admin admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:21:14.552391 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46549359-73e5-4bb5-98c9-fab66c6a86ae admin admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:21:14.558595 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beb01c30-f606-458e-a6f6-6d15b1881807 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76c5181fc7be45279ac00017bb4640b3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:21:14.568476 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beb01c30-f606-458e-a6f6-6d15b1881807 admin admin] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:21:14.569342 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beb01c30-f606-458e-a6f6-6d15b1881807 admin admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:21:14.569613 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beb01c30-f606-458e-a6f6-6d15b1881807 admin admin] PATH_INFO: `/v3/users` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:21:14.570363 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beb01c30-f606-458e-a6f6-6d15b1881807 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:21:14.573834 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beb01c30-f606-458e-a6f6-6d15b1881807 admin admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:21:14.609189 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 890/1778] 10.222.0.18 () {60 vars in 1144 bytes} [Tue Sep 13 03:21:14 2022] POST /identity/v3/users => generated 291 bytes in 204 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 03:21:14.615563 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beb01c30-f606-458e-a6f6-6d15b1881807 admin admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:21:14.633160 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beb01c30-f606-458e-a6f6-6d15b1881807 admin admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:21:14.640074 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fa50d38-9029-4511-91e7-6ea4189c5f2d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76c5181fc7be45279ac00017bb4640b3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:21:14.640821 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa50d38-9029-4511-91e7-6ea4189c5f2d admin admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:21:14.641212 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa50d38-9029-4511-91e7-6ea4189c5f2d admin admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:21:14.641598 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa50d38-9029-4511-91e7-6ea4189c5f2d admin admin] PATH_INFO: `/v3/roles` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:21:14.642418 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fa50d38-9029-4511-91e7-6ea4189c5f2d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:21:14.646301 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fa50d38-9029-4511-91e7-6ea4189c5f2d admin admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:21:14.668031 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 891/1779] 10.222.0.18 () {58 vars in 1122 bytes} [Tue Sep 13 03:21:14 2022] GET /identity/v3/roles => generated 1420 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 03:21:14.668297 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fa50d38-9029-4511-91e7-6ea4189c5f2d admin admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:21:14.713046 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fa50d38-9029-4511-91e7-6ea4189c5f2d admin admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:21:14.729983 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e3483be-2d9f-454f-9f69-212168dfcfac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76c5181fc7be45279ac00017bb4640b3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:21:14.734084 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e3483be-2d9f-454f-9f69-212168dfcfac admin admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:21:14.734543 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e3483be-2d9f-454f-9f69-212168dfcfac admin admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:21:14.734867 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e3483be-2d9f-454f-9f69-212168dfcfac admin admin] PATH_INFO: `/v3/roles` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:21:14.739078 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e3483be-2d9f-454f-9f69-212168dfcfac admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:21:14.739468 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-837c92ef-c534-4ee5-a9c8-f8ed2808cc6b admin admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:21:14.744743 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e3483be-2d9f-454f-9f69-212168dfcfac admin admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:21:14.788955 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 892/1780] 10.222.0.18 () {58 vars in 1122 bytes} [Tue Sep 13 03:21:14 2022] GET /identity/v3/roles => generated 1420 bytes in 118 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 03:21:14.794103 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e3483be-2d9f-454f-9f69-212168dfcfac admin admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:21:14.816525 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e3483be-2d9f-454f-9f69-212168dfcfac admin admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:21:14.826474 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-656273ab-14ab-40ac-8986-cb8070c115e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76c5181fc7be45279ac00017bb4640b3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:21:14.827430 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-656273ab-14ab-40ac-8986-cb8070c115e5 admin admin] REQUEST_METHOD: `PUT` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:21:14.827880 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-656273ab-14ab-40ac-8986-cb8070c115e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:21:14.828307 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-656273ab-14ab-40ac-8986-cb8070c115e5 admin admin] PATH_INFO: `/v3/projects/30a882ea0e7b46039e44abe7e9d957b1/users/7cbbcd0d1cba4052baf9c7d58370d5f4/roles/425d4b481c7846e0bc07e7e805d5b15f` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:21:14.830432 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-656273ab-14ab-40ac-8986-cb8070c115e5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=30a882ea0e7b46039e44abe7e9d957b1, user_id=7cbbcd0d1cba4052baf9c7d58370d5f4, role_id=425d4b481c7846e0bc07e7e805d5b15f)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:21:14.835099 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-656273ab-14ab-40ac-8986-cb8070c115e5 admin admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:21:14.848871 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 893/1781] 10.222.0.18 () {60 vars in 1483 bytes} [Tue Sep 13 03:21:14 2022] PUT /identity/v3/projects/30a882ea0e7b46039e44abe7e9d957b1/users/7cbbcd0d1cba4052baf9c7d58370d5f4/roles/425d4b481c7846e0bc07e7e805d5b15f => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 03:21:14.850080 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-656273ab-14ab-40ac-8986-cb8070c115e5 admin admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:21:14.892823 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4ce3ee9-f514-4767-a170-a5581ac4f0e2 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '893b0c0d85964b7dbc9705c0a091682c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b9d94f6a68f7470d9494f112175620dc', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:21:14.893583 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ce3ee9-f514-4767-a170-a5581ac4f0e2 service swift] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:21:14.893987 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ce3ee9-f514-4767-a170-a5581ac4f0e2 service swift] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:21:14.894357 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ce3ee9-f514-4767-a170-a5581ac4f0e2 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:21:15.027765 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4ce3ee9-f514-4767-a170-a5581ac4f0e2 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:21:15.029508 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4ce3ee9-f514-4767-a170-a5581ac4f0e2 service swift] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:21:15.072934 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 889/1782] 10.222.0.18 () {60 vars in 1384 bytes} [Tue Sep 13 03:21:14 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 710 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 13 03:21:15.077237 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07fdbd7a-52b5-41ff-9fcf-6c2c24341fa9 None None] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:21:15.077570 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07fdbd7a-52b5-41ff-9fcf-6c2c24341fa9 None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:21:15.077927 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07fdbd7a-52b5-41ff-9fcf-6c2c24341fa9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:21:15.152802 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.auth.core [None req-07fdbd7a-52b5-41ff-9fcf-6c2c24341fa9 None None] MFA Rules not processed for user `7cbbcd0d1cba4052baf9c7d58370d5f4`. Rule list: `[]` (Enabled: `True`). {{(pid=81400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 03:21:15.154289 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-07fdbd7a-52b5-41ff-9fcf-6c2c24341fa9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:21:15.168020 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:INSERT=1,keystone:SELECT=99 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:21:15.230302 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-656273ab-14ab-40ac-8986-cb8070c115e5 admin admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:21:15.261009 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96c4ae59-69e9-4c1f-a727-a407de552a4d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76c5181fc7be45279ac00017bb4640b3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:21:15.263038 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c4ae59-69e9-4c1f-a727-a407de552a4d admin admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:21:15.263038 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c4ae59-69e9-4c1f-a727-a407de552a4d admin admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:21:15.263038 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c4ae59-69e9-4c1f-a727-a407de552a4d admin admin] PATH_INFO: `/v3/roles` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:21:15.263038 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96c4ae59-69e9-4c1f-a727-a407de552a4d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:21:15.264719 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96c4ae59-69e9-4c1f-a727-a407de552a4d admin admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:21:15.310713 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 894/1783] 10.222.0.18 () {58 vars in 1122 bytes} [Tue Sep 13 03:21:14 2022] GET /identity/v3/roles => generated 1420 bytes in 461 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 03:21:15.315331 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96c4ae59-69e9-4c1f-a727-a407de552a4d admin admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:21:15.336119 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96c4ae59-69e9-4c1f-a727-a407de552a4d admin admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:21:15.346230 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99e6f299-c2c9-4f7a-bffd-edbea1eb1a86 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76c5181fc7be45279ac00017bb4640b3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:21:15.350459 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e6f299-c2c9-4f7a-bffd-edbea1eb1a86 admin admin] REQUEST_METHOD: `PUT` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:21:15.354822 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e6f299-c2c9-4f7a-bffd-edbea1eb1a86 admin admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:21:15.357188 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e6f299-c2c9-4f7a-bffd-edbea1eb1a86 admin admin] PATH_INFO: `/v3/projects/56995cc802ac4e9ba69eb5c7486703a8/users/a501ad47cf9a42bdbbdaacda2f0a1563/roles/1eb7a5b5203945c29e6d22d653f47847` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:21:15.378472 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:INSERT=6,keystone:SELECT=94 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:21:15.418984 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 890/1784] 10.222.0.18 () {58 vars in 958 bytes} [Tue Sep 13 03:21:15 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 345 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 03:21:15.435519 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07fdbd7a-52b5-41ff-9fcf-6c2c24341fa9 None None] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:21:15.436855 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-07fdbd7a-52b5-41ff-9fcf-6c2c24341fa9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:21:15.477534 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99e6f299-c2c9-4f7a-bffd-edbea1eb1a86 admin admin] RBAC: Authorizing `identity:create_grant(project_id=56995cc802ac4e9ba69eb5c7486703a8, user_id=a501ad47cf9a42bdbbdaacda2f0a1563, role_id=1eb7a5b5203945c29e6d22d653f47847)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:21:15.493445 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99e6f299-c2c9-4f7a-bffd-edbea1eb1a86 admin admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:21:15.552928 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 895/1785] 10.222.0.18 () {60 vars in 1483 bytes} [Tue Sep 13 03:21:15 2022] PUT /identity/v3/projects/56995cc802ac4e9ba69eb5c7486703a8/users/a501ad47cf9a42bdbbdaacda2f0a1563/roles/1eb7a5b5203945c29e6d22d653f47847 => generated 0 bytes in 241 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 03:21:15.564475 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba8919ba-b5b3-4788-951f-c7a47423a413 None None] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:21:15.564801 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba8919ba-b5b3-4788-951f-c7a47423a413 None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:21:15.565025 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba8919ba-b5b3-4788-951f-c7a47423a413 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:21:15.650300 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.auth.core [None req-ba8919ba-b5b3-4788-951f-c7a47423a413 None None] MFA Rules not processed for user `a501ad47cf9a42bdbbdaacda2f0a1563`. Rule list: `[]` (Enabled: `True`). {{(pid=81399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 03:21:15.652811 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-ba8919ba-b5b3-4788-951f-c7a47423a413 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:21:15.662793 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07fdbd7a-52b5-41ff-9fcf-6c2c24341fa9 None None] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:21:15.919424 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94eb035e-3e40-4bd1-ba0a-a1ed5525a869 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '924ee3f608d34a18b0ddc4695f102a56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b9d94f6a68f7470d9494f112175620dc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:21:15.920322 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94eb035e-3e40-4bd1-ba0a-a1ed5525a869 service cinder] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:21:15.920610 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94eb035e-3e40-4bd1-ba0a-a1ed5525a869 service cinder] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:21:15.920867 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94eb035e-3e40-4bd1-ba0a-a1ed5525a869 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:21:16.018309 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 896/1786] 10.222.0.18 () {58 vars in 958 bytes} [Tue Sep 13 03:21:15 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 454 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 03:21:16.026108 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba8919ba-b5b3-4788-951f-c7a47423a413 None None] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:21:16.178683 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94eb035e-3e40-4bd1-ba0a-a1ed5525a869 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:21:16.197131 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94eb035e-3e40-4bd1-ba0a-a1ed5525a869 service cinder] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:21:16.216771 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 891/1787] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Sep 13 03:21:15 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 782 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 03:21:16.279019 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba8919ba-b5b3-4788-951f-c7a47423a413 None None] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:21:16.285687 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca6c1fea-beee-4b52-992a-3d3c633e2060 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76c5181fc7be45279ac00017bb4640b3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:21:16.286563 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca6c1fea-beee-4b52-992a-3d3c633e2060 admin admin] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:21:16.286872 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca6c1fea-beee-4b52-992a-3d3c633e2060 admin admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:21:16.287142 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca6c1fea-beee-4b52-992a-3d3c633e2060 admin admin] PATH_INFO: `/v3/projects` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:21:16.288681 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca6c1fea-beee-4b52-992a-3d3c633e2060 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:21:16.292859 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca6c1fea-beee-4b52-992a-3d3c633e2060 admin admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:21:16.384221 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 897/1788] 10.222.0.18 () {60 vars in 1153 bytes} [Tue Sep 13 03:21:16 2022] POST /identity/v3/projects => generated 383 bytes in 359 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 03:21:16.389774 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94eb035e-3e40-4bd1-ba0a-a1ed5525a869 service cinder] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:21:16.414120 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94eb035e-3e40-4bd1-ba0a-a1ed5525a869 service cinder] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:21:16.431088 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:21:16.480921 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca6c1fea-beee-4b52-992a-3d3c633e2060 admin admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:21:16.520572 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9600b952-7c61-45b5-91e8-d9f224f0d4aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76c5181fc7be45279ac00017bb4640b3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:21:16.521690 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9600b952-7c61-45b5-91e8-d9f224f0d4aa admin admin] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:21:16.522203 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9600b952-7c61-45b5-91e8-d9f224f0d4aa admin admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:21:16.522584 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9600b952-7c61-45b5-91e8-d9f224f0d4aa admin admin] PATH_INFO: `/v3/users` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:21:16.523467 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9600b952-7c61-45b5-91e8-d9f224f0d4aa admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:21:16.526246 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9600b952-7c61-45b5-91e8-d9f224f0d4aa admin admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:21:16.570391 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 892/1789] 10.222.0.18 () {60 vars in 1144 bytes} [Tue Sep 13 03:21:16 2022] POST /identity/v3/users => generated 298 bytes in 182 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 03:21:16.586505 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9600b952-7c61-45b5-91e8-d9f224f0d4aa admin admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:21:16.631645 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9600b952-7c61-45b5-91e8-d9f224f0d4aa admin admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:21:16.657114 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c9f1cb3-ded7-4e85-a6f6-5601e8fd0870 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76c5181fc7be45279ac00017bb4640b3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:21:16.664800 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c9f1cb3-ded7-4e85-a6f6-5601e8fd0870 admin admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:21:16.668772 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c9f1cb3-ded7-4e85-a6f6-5601e8fd0870 admin admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:21:16.669273 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c9f1cb3-ded7-4e85-a6f6-5601e8fd0870 admin admin] PATH_INFO: `/v3/roles` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:21:16.671483 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c9f1cb3-ded7-4e85-a6f6-5601e8fd0870 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:21:16.677094 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c9f1cb3-ded7-4e85-a6f6-5601e8fd0870 admin admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:21:16.712140 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca6c1fea-beee-4b52-992a-3d3c633e2060 admin admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:21:16.735850 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8654a5a-dbbb-4915-9896-5ec4f729ee9d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '924ee3f608d34a18b0ddc4695f102a56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b9d94f6a68f7470d9494f112175620dc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:21:16.741441 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8654a5a-dbbb-4915-9896-5ec4f729ee9d service cinder] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:21:16.741813 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8654a5a-dbbb-4915-9896-5ec4f729ee9d service cinder] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:21:16.748082 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8654a5a-dbbb-4915-9896-5ec4f729ee9d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:21:16.751678 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 893/1790] 10.222.0.18 () {58 vars in 1122 bytes} [Tue Sep 13 03:21:16 2022] GET /identity/v3/roles => generated 1420 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 03:21:16.753016 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c9f1cb3-ded7-4e85-a6f6-5601e8fd0870 admin admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:21:16.780831 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c9f1cb3-ded7-4e85-a6f6-5601e8fd0870 admin admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:21:16.800408 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3522f2f8-a676-4137-aa60-a2796d09ca88 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76c5181fc7be45279ac00017bb4640b3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:21:16.801106 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3522f2f8-a676-4137-aa60-a2796d09ca88 admin admin] REQUEST_METHOD: `PUT` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:21:16.801509 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3522f2f8-a676-4137-aa60-a2796d09ca88 admin admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:21:16.802358 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3522f2f8-a676-4137-aa60-a2796d09ca88 admin admin] PATH_INFO: `/v3/projects/512eb3b2096c484093cd191e5da2928f/users/bed20513890a4a669901ee5802cdfcfa/roles/425d4b481c7846e0bc07e7e805d5b15f` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:21:16.854838 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3522f2f8-a676-4137-aa60-a2796d09ca88 admin admin] RBAC: Authorizing `identity:create_grant(project_id=512eb3b2096c484093cd191e5da2928f, user_id=bed20513890a4a669901ee5802cdfcfa, role_id=425d4b481c7846e0bc07e7e805d5b15f)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:21:16.860152 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:21:16.863122 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3522f2f8-a676-4137-aa60-a2796d09ca88 admin admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:21:16.897383 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 894/1791] 10.222.0.18 () {60 vars in 1483 bytes} [Tue Sep 13 03:21:16 2022] PUT /identity/v3/projects/512eb3b2096c484093cd191e5da2928f/users/bed20513890a4a669901ee5802cdfcfa/roles/425d4b481c7846e0bc07e7e805d5b15f => generated 0 bytes in 145 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 03:21:16.903810 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3522f2f8-a676-4137-aa60-a2796d09ca88 admin admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:21:17.055486 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8654a5a-dbbb-4915-9896-5ec4f729ee9d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:21:17.062094 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8654a5a-dbbb-4915-9896-5ec4f729ee9d service cinder] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:21:17.105878 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 898/1792] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Sep 13 03:21:16 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 625 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 03:21:17.141098 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3522f2f8-a676-4137-aa60-a2796d09ca88 admin admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:21:17.172567 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eec5ed58-da01-4e79-a8b0-f6718b9d7b0c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76c5181fc7be45279ac00017bb4640b3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:21:17.173481 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eec5ed58-da01-4e79-a8b0-f6718b9d7b0c admin admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:21:17.174136 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eec5ed58-da01-4e79-a8b0-f6718b9d7b0c admin admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:21:17.174876 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eec5ed58-da01-4e79-a8b0-f6718b9d7b0c admin admin] PATH_INFO: `/v3/roles` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:21:17.177241 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eec5ed58-da01-4e79-a8b0-f6718b9d7b0c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:21:17.181361 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eec5ed58-da01-4e79-a8b0-f6718b9d7b0c admin admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:21:17.207126 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 895/1793] 10.222.0.18 () {58 vars in 1122 bytes} [Tue Sep 13 03:21:16 2022] GET /identity/v3/roles => generated 1420 bytes in 304 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 03:21:17.212845 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8654a5a-dbbb-4915-9896-5ec4f729ee9d service cinder] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:21:17.245183 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8654a5a-dbbb-4915-9896-5ec4f729ee9d service cinder] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:21:17.274112 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc096aca-2ff8-4662-87a2-5301a3be2ea7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76c5181fc7be45279ac00017bb4640b3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:21:17.280249 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc096aca-2ff8-4662-87a2-5301a3be2ea7 admin admin] REQUEST_METHOD: `PUT` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:21:17.281367 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc096aca-2ff8-4662-87a2-5301a3be2ea7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:21:17.281825 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc096aca-2ff8-4662-87a2-5301a3be2ea7 admin admin] PATH_INFO: `/v3/projects/512eb3b2096c484093cd191e5da2928f/users/bed20513890a4a669901ee5802cdfcfa/roles/425d4b481c7846e0bc07e7e805d5b15f` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:21:17.296724 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc096aca-2ff8-4662-87a2-5301a3be2ea7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=512eb3b2096c484093cd191e5da2928f, user_id=bed20513890a4a669901ee5802cdfcfa, role_id=425d4b481c7846e0bc07e7e805d5b15f)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:21:17.300258 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc096aca-2ff8-4662-87a2-5301a3be2ea7 admin admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:21:17.359735 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 899/1794] 10.222.0.18 () {60 vars in 1483 bytes} [Tue Sep 13 03:21:17 2022] PUT /identity/v3/projects/512eb3b2096c484093cd191e5da2928f/users/bed20513890a4a669901ee5802cdfcfa/roles/425d4b481c7846e0bc07e7e805d5b15f => generated 0 bytes in 147 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 03:21:17.372484 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2053349c-4726-40a0-a64c-e2d7ff428077 None None] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:21:17.373223 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2053349c-4726-40a0-a64c-e2d7ff428077 None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:21:17.373879 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2053349c-4726-40a0-a64c-e2d7ff428077 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:21:17.508635 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.auth.core [None req-2053349c-4726-40a0-a64c-e2d7ff428077 None None] MFA Rules not processed for user `bed20513890a4a669901ee5802cdfcfa`. Rule list: `[]` (Enabled: `True`). {{(pid=81400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 03:21:17.510305 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-2053349c-4726-40a0-a64c-e2d7ff428077 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:21:17.676338 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:21:17.760818 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 896/1795] 10.222.0.18 () {58 vars in 958 bytes} [Tue Sep 13 03:21:17 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 391 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 03:21:18.191241 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc096aca-2ff8-4662-87a2-5301a3be2ea7 admin admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:21:18.342171 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc096aca-2ff8-4662-87a2-5301a3be2ea7 admin admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:21:18.361033 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92e9ac74-7d5c-47c8-9dfe-3d62ddc29473 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '924ee3f608d34a18b0ddc4695f102a56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b9d94f6a68f7470d9494f112175620dc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:21:18.362573 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92e9ac74-7d5c-47c8-9dfe-3d62ddc29473 service cinder] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:21:18.363142 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92e9ac74-7d5c-47c8-9dfe-3d62ddc29473 service cinder] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:21:18.366932 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92e9ac74-7d5c-47c8-9dfe-3d62ddc29473 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:21:18.420365 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92e9ac74-7d5c-47c8-9dfe-3d62ddc29473 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:21:18.425340 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92e9ac74-7d5c-47c8-9dfe-3d62ddc29473 service cinder] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:21:18.480394 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 900/1796] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Sep 13 03:21:18 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 288 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 03:21:18.822369 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2053349c-4726-40a0-a64c-e2d7ff428077 None None] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:21:18.834332 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2053349c-4726-40a0-a64c-e2d7ff428077 None None] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:21:18.846926 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2ea4ec8-5b38-47e7-a642-c002541ca11d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '924ee3f608d34a18b0ddc4695f102a56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b9d94f6a68f7470d9494f112175620dc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:21:18.847642 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2ea4ec8-5b38-47e7-a642-c002541ca11d service cinder] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:21:18.847929 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2ea4ec8-5b38-47e7-a642-c002541ca11d service cinder] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:21:18.848127 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2ea4ec8-5b38-47e7-a642-c002541ca11d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:21:18.973095 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2ea4ec8-5b38-47e7-a642-c002541ca11d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:21:18.975310 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2ea4ec8-5b38-47e7-a642-c002541ca11d service cinder] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:21:19.000380 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 897/1797] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Sep 13 03:21:18 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 178 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 03:21:19.901683 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92e9ac74-7d5c-47c8-9dfe-3d62ddc29473 service cinder] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:21:20.082093 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92e9ac74-7d5c-47c8-9dfe-3d62ddc29473 service cinder] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:21:20.089143 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-430a5564-2b8c-43ec-ba7c-84758fdba28d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76c5181fc7be45279ac00017bb4640b3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:21:20.090035 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-430a5564-2b8c-43ec-ba7c-84758fdba28d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:21:20.090576 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-430a5564-2b8c-43ec-ba7c-84758fdba28d admin admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:21:20.091076 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-430a5564-2b8c-43ec-ba7c-84758fdba28d admin admin] PATH_INFO: `/v3/users/a501ad47cf9a42bdbbdaacda2f0a1563` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:21:20.093193 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-430a5564-2b8c-43ec-ba7c-84758fdba28d admin admin] RBAC: Authorizing `identity:delete_user(user_id=a501ad47cf9a42bdbbdaacda2f0a1563)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:21:20.095703 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-430a5564-2b8c-43ec-ba7c-84758fdba28d admin admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:21:20.186913 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=92,keystone:INSERT=1,keystone:DELETE=7 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:21:20.194285 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-430a5564-2b8c-43ec-ba7c-84758fdba28d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a501ad47cf9a42bdbbdaacda2f0a1563'} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:21:20.213117 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-430a5564-2b8c-43ec-ba7c-84758fdba28d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a501ad47cf9a42bdbbdaacda2f0a1563'} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:21:20.233884 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-430a5564-2b8c-43ec-ba7c-84758fdba28d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a501ad47cf9a42bdbbdaacda2f0a1563'} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:21:20.235039 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-430a5564-2b8c-43ec-ba7c-84758fdba28d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a501ad47cf9a42bdbbdaacda2f0a1563'} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:21:20.245977 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-430a5564-2b8c-43ec-ba7c-84758fdba28d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a501ad47cf9a42bdbbdaacda2f0a1563'} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:21:20.258746 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 901/1798] 10.222.0.18 () {58 vars in 1224 bytes} [Tue Sep 13 03:21:19 2022] DELETE /identity/v3/users/a501ad47cf9a42bdbbdaacda2f0a1563 => generated 0 bytes in 355 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 03:21:20.260418 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-609a59e8-f452-4198-85a1-527f20fe7388 None None] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:21:20.260979 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-609a59e8-f452-4198-85a1-527f20fe7388 None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:21:20.260979 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-609a59e8-f452-4198-85a1-527f20fe7388 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:21:20.286977 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.auth.core [None req-609a59e8-f452-4198-85a1-527f20fe7388 None None] MFA Rules not processed for user `49f280ab086b4610855c4ffa378517d9`. Rule list: `[]` (Enabled: `True`). {{(pid=81400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 03:21:20.289508 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-609a59e8-f452-4198-85a1-527f20fe7388 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:21:20.383524 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 898/1799] 10.222.0.18 () {58 vars in 958 bytes} [Tue Sep 13 03:21:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 03:21:20.390687 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-430a5564-2b8c-43ec-ba7c-84758fdba28d admin admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:21:20.392381 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-430a5564-2b8c-43ec-ba7c-84758fdba28d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:21:20.433910 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d1f020-328f-4df0-84dd-04617b26058a None None] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:21:20.434180 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d1f020-328f-4df0-84dd-04617b26058a None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:21:20.434452 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d1f020-328f-4df0-84dd-04617b26058a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:21:20.459255 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.auth.core [None req-44d1f020-328f-4df0-84dd-04617b26058a None None] MFA Rules not processed for user `4f76a7c6fe9347b0bea5f701bccb6150`. Rule list: `[]` (Enabled: `True`). {{(pid=81400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 03:21:20.461088 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-44d1f020-328f-4df0-84dd-04617b26058a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:21:20.477644 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-430a5564-2b8c-43ec-ba7c-84758fdba28d admin admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:21:20.483455 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3e7b7f2-d24a-4b38-b594-b739d7a53915 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdffbc851086441ebd01022d1e4c2609', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b9d94f6a68f7470d9494f112175620dc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:21:20.484743 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3e7b7f2-d24a-4b38-b594-b739d7a53915 service neutron] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:21:20.484743 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3e7b7f2-d24a-4b38-b594-b739d7a53915 service neutron] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:21:20.484743 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3e7b7f2-d24a-4b38-b594-b739d7a53915 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:21:20.495304 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3e7b7f2-d24a-4b38-b594-b739d7a53915 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:21:20.497733 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3e7b7f2-d24a-4b38-b594-b739d7a53915 service neutron] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:21:20.513230 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 902/1800] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Sep 13 03:21:20 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 123 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 03:21:20.558021 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 899/1801] 10.222.0.18 () {58 vars in 958 bytes} [Tue Sep 13 03:21:20 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 03:21:21.459951 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3e7b7f2-d24a-4b38-b594-b739d7a53915 service neutron] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:21:21.461954 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-a3e7b7f2-d24a-4b38-b594-b739d7a53915 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:21:21.492394 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3e7b7f2-d24a-4b38-b594-b739d7a53915 service neutron] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:21:21.512602 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be716ebe-7bb9-4cea-ada4-06f593b6a159 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76c5181fc7be45279ac00017bb4640b3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:21:21.515016 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be716ebe-7bb9-4cea-ada4-06f593b6a159 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:21:21.515467 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be716ebe-7bb9-4cea-ada4-06f593b6a159 admin admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:21:21.515896 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be716ebe-7bb9-4cea-ada4-06f593b6a159 admin admin] PATH_INFO: `/v3/projects/56995cc802ac4e9ba69eb5c7486703a8` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:21:21.519612 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be716ebe-7bb9-4cea-ada4-06f593b6a159 admin admin] RBAC: Authorizing `identity:delete_project(project_id=56995cc802ac4e9ba69eb5c7486703a8)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:21:21.522214 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be716ebe-7bb9-4cea-ada4-06f593b6a159 admin admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:21:21.621105 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-be716ebe-7bb9-4cea-ada4-06f593b6a159 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '56995cc802ac4e9ba69eb5c7486703a8', 'request_id': 'req-be716ebe-7bb9-4cea-ada4-06f593b6a159'} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:21:21.628997 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-be716ebe-7bb9-4cea-ada4-06f593b6a159 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '56995cc802ac4e9ba69eb5c7486703a8', 'request_id': 'req-be716ebe-7bb9-4cea-ada4-06f593b6a159'} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:21:21.637410 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-be716ebe-7bb9-4cea-ada4-06f593b6a159 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '56995cc802ac4e9ba69eb5c7486703a8', 'request_id': 'req-be716ebe-7bb9-4cea-ada4-06f593b6a159'} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:21:21.664108 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-be716ebe-7bb9-4cea-ada4-06f593b6a159 admin admin] The token cache is being invalidate because project 56995cc802ac4e9ba69eb5c7486703a8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81399) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 03:21:21.664401 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-be716ebe-7bb9-4cea-ada4-06f593b6a159 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:21:21.665081 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-be716ebe-7bb9-4cea-ada4-06f593b6a159 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:21:21.666644 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 903/1802] 10.222.0.18 () {58 vars in 1233 bytes} [Tue Sep 13 03:21:21 2022] DELETE /identity/v3/projects/56995cc802ac4e9ba69eb5c7486703a8 => generated 0 bytes in 207 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 03:21:21.671025 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44d1f020-328f-4df0-84dd-04617b26058a None None] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:21:21.684440 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44d1f020-328f-4df0-84dd-04617b26058a None None] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:21:21.778248 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:21:21.836890 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eefeed49-aac4-42cd-85f4-1ff9e84f5648 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76c5181fc7be45279ac00017bb4640b3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:21:21.837396 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eefeed49-aac4-42cd-85f4-1ff9e84f5648 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:21:21.837678 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eefeed49-aac4-42cd-85f4-1ff9e84f5648 admin admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:21:21.837952 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eefeed49-aac4-42cd-85f4-1ff9e84f5648 admin admin] PATH_INFO: `/v3/users/bed20513890a4a669901ee5802cdfcfa` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:21:21.840294 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eefeed49-aac4-42cd-85f4-1ff9e84f5648 admin admin] RBAC: Authorizing `identity:delete_user(user_id=bed20513890a4a669901ee5802cdfcfa)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:21:21.841990 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eefeed49-aac4-42cd-85f4-1ff9e84f5648 admin admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:21:21.929848 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-eefeed49-aac4-42cd-85f4-1ff9e84f5648 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bed20513890a4a669901ee5802cdfcfa'} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:21:21.944209 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-eefeed49-aac4-42cd-85f4-1ff9e84f5648 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bed20513890a4a669901ee5802cdfcfa'} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:21:21.944913 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-eefeed49-aac4-42cd-85f4-1ff9e84f5648 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bed20513890a4a669901ee5802cdfcfa'} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:21:21.962288 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-eefeed49-aac4-42cd-85f4-1ff9e84f5648 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bed20513890a4a669901ee5802cdfcfa'} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:21:21.973995 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-eefeed49-aac4-42cd-85f4-1ff9e84f5648 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bed20513890a4a669901ee5802cdfcfa'} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:21:21.980751 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 900/1803] 10.222.0.18 () {58 vars in 1224 bytes} [Tue Sep 13 03:21:21 2022] DELETE /identity/v3/users/bed20513890a4a669901ee5802cdfcfa => generated 0 bytes in 310 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 03:21:22.319953 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be716ebe-7bb9-4cea-ada4-06f593b6a159 admin admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:21:22.321407 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-be716ebe-7bb9-4cea-ada4-06f593b6a159 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:21:22.341578 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be716ebe-7bb9-4cea-ada4-06f593b6a159 admin admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:21:22.396858 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:21:22.411331 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-377dc6b8-badd-4d20-b9d2-1dff7bba2e55 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76c5181fc7be45279ac00017bb4640b3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:21:22.412020 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-377dc6b8-badd-4d20-b9d2-1dff7bba2e55 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:21:22.412298 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-377dc6b8-badd-4d20-b9d2-1dff7bba2e55 admin admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:21:22.412565 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-377dc6b8-badd-4d20-b9d2-1dff7bba2e55 admin admin] PATH_INFO: `/v3/projects/512eb3b2096c484093cd191e5da2928f` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:21:22.413704 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-377dc6b8-badd-4d20-b9d2-1dff7bba2e55 admin admin] RBAC: Authorizing `identity:delete_project(project_id=512eb3b2096c484093cd191e5da2928f)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:21:22.415674 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-377dc6b8-badd-4d20-b9d2-1dff7bba2e55 admin admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:21:22.470091 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-377dc6b8-badd-4d20-b9d2-1dff7bba2e55 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '512eb3b2096c484093cd191e5da2928f', 'request_id': 'req-377dc6b8-badd-4d20-b9d2-1dff7bba2e55'} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:21:22.475961 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-377dc6b8-badd-4d20-b9d2-1dff7bba2e55 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '512eb3b2096c484093cd191e5da2928f', 'request_id': 'req-377dc6b8-badd-4d20-b9d2-1dff7bba2e55'} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:21:22.483318 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-377dc6b8-badd-4d20-b9d2-1dff7bba2e55 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '512eb3b2096c484093cd191e5da2928f', 'request_id': 'req-377dc6b8-badd-4d20-b9d2-1dff7bba2e55'} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:21:22.501680 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-377dc6b8-badd-4d20-b9d2-1dff7bba2e55 admin admin] The token cache is being invalidate because project 512eb3b2096c484093cd191e5da2928f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81399) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 03:21:22.502002 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-377dc6b8-badd-4d20-b9d2-1dff7bba2e55 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:21:22.502737 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-377dc6b8-badd-4d20-b9d2-1dff7bba2e55 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:21:22.504421 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 904/1804] 10.222.0.18 () {58 vars in 1233 bytes} [Tue Sep 13 03:21:22 2022] DELETE /identity/v3/projects/512eb3b2096c484093cd191e5da2928f => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 03:21:22.509659 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee3be84-aee1-4eb3-908d-8b5abff2814d None None] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:21:22.509975 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee3be84-aee1-4eb3-908d-8b5abff2814d None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:21:22.510253 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee3be84-aee1-4eb3-908d-8b5abff2814d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:21:22.530089 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.auth.core [None req-7ee3be84-aee1-4eb3-908d-8b5abff2814d None None] MFA Rules not processed for user `49f280ab086b4610855c4ffa378517d9`. Rule list: `[]` (Enabled: `True`). {{(pid=81400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 03:21:22.531368 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-7ee3be84-aee1-4eb3-908d-8b5abff2814d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:21:22.598757 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 901/1805] 10.222.0.18 () {58 vars in 958 bytes} [Tue Sep 13 03:21:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 03:21:22.607168 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-558f00d0-cfc8-4a82-a2a7-a2e9eebdec02 None None] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:21:22.607654 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-558f00d0-cfc8-4a82-a2a7-a2e9eebdec02 None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:21:22.608063 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-558f00d0-cfc8-4a82-a2a7-a2e9eebdec02 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:21:22.644641 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.auth.core [None req-558f00d0-cfc8-4a82-a2a7-a2e9eebdec02 None None] MFA Rules not processed for user `49f280ab086b4610855c4ffa378517d9`. Rule list: `[]` (Enabled: `True`). {{(pid=81399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 03:21:22.646172 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-558f00d0-cfc8-4a82-a2a7-a2e9eebdec02 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:21:22.658402 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 905/1806] 10.222.0.18 () {58 vars in 958 bytes} [Tue Sep 13 03:21:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 55 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 03:21:22.665846 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ee3be84-aee1-4eb3-908d-8b5abff2814d None None] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:21:22.678945 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ee3be84-aee1-4eb3-908d-8b5abff2814d None None] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:21:22.684337 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35b748cc-aaee-4786-bd69-670eac97ea02 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76c5181fc7be45279ac00017bb4640b3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:21:22.684962 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b748cc-aaee-4786-bd69-670eac97ea02 admin admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:21:22.685384 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b748cc-aaee-4786-bd69-670eac97ea02 admin admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:21:22.685622 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b748cc-aaee-4786-bd69-670eac97ea02 admin admin] PATH_INFO: `/v3/domains` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:21:22.686413 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35b748cc-aaee-4786-bd69-670eac97ea02 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 03:21:22.686781 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35b748cc-aaee-4786-bd69-670eac97ea02 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:21:22.688546 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35b748cc-aaee-4786-bd69-670eac97ea02 admin admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:21:22.702907 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 902/1807] 10.222.0.18 () {58 vars in 1153 bytes} [Tue Sep 13 03:21:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 03:21:22.707866 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-558f00d0-cfc8-4a82-a2a7-a2e9eebdec02 None None] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:21:22.717596 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-558f00d0-cfc8-4a82-a2a7-a2e9eebdec02 None None] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:21:22.723028 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ddf2df0-3592-4bbf-a473-9c78caa386a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76c5181fc7be45279ac00017bb4640b3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:21:22.724216 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ddf2df0-3592-4bbf-a473-9c78caa386a7 admin admin] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:21:22.724540 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ddf2df0-3592-4bbf-a473-9c78caa386a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:21:22.724814 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ddf2df0-3592-4bbf-a473-9c78caa386a7 admin admin] PATH_INFO: `/v3/projects` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:21:22.725449 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ddf2df0-3592-4bbf-a473-9c78caa386a7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:21:22.727533 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ddf2df0-3592-4bbf-a473-9c78caa386a7 admin admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:21:22.765015 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 906/1808] 10.222.0.18 () {60 vars in 1153 bytes} [Tue Sep 13 03:21:22 2022] POST /identity/v3/projects => generated 383 bytes in 57 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 03:21:22.768452 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35b748cc-aaee-4786-bd69-670eac97ea02 admin admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:21:22.780305 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35b748cc-aaee-4786-bd69-670eac97ea02 admin admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:21:22.834140 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:21:22.875329 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c768350d-517f-4e7f-bec0-e0d3ebee45f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76c5181fc7be45279ac00017bb4640b3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:21:22.875954 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c768350d-517f-4e7f-bec0-e0d3ebee45f1 admin admin] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:21:22.876231 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c768350d-517f-4e7f-bec0-e0d3ebee45f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:21:22.876500 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c768350d-517f-4e7f-bec0-e0d3ebee45f1 admin admin] PATH_INFO: `/v3/users` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:21:22.877298 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c768350d-517f-4e7f-bec0-e0d3ebee45f1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:21:22.879606 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c768350d-517f-4e7f-bec0-e0d3ebee45f1 admin admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:21:22.906555 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 903/1809] 10.222.0.18 () {60 vars in 1144 bytes} [Tue Sep 13 03:21:22 2022] POST /identity/v3/users => generated 292 bytes in 138 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 13 03:21:22.910292 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ddf2df0-3592-4bbf-a473-9c78caa386a7 admin admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:21:22.940693 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ddf2df0-3592-4bbf-a473-9c78caa386a7 admin admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:21:22.960314 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67df70e3-3a6a-459f-bf0a-2b4f606d5166 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76c5181fc7be45279ac00017bb4640b3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:21:22.960953 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67df70e3-3a6a-459f-bf0a-2b4f606d5166 admin admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:21:22.961235 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67df70e3-3a6a-459f-bf0a-2b4f606d5166 admin admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:21:22.961574 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67df70e3-3a6a-459f-bf0a-2b4f606d5166 admin admin] PATH_INFO: `/v3/roles` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:21:22.962166 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67df70e3-3a6a-459f-bf0a-2b4f606d5166 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:21:22.963965 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67df70e3-3a6a-459f-bf0a-2b4f606d5166 admin admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:21:23.000754 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c768350d-517f-4e7f-bec0-e0d3ebee45f1 admin admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:21:23.002461 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 907/1810] 10.222.0.18 () {58 vars in 1122 bytes} [Tue Sep 13 03:21:22 2022] GET /identity/v3/roles => generated 1420 bytes in 92 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 03:21:23.013100 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c768350d-517f-4e7f-bec0-e0d3ebee45f1 admin admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:21:23.031088 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4687957a-70f1-489a-a3c0-2cad769fd378 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76c5181fc7be45279ac00017bb4640b3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:21:23.032100 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4687957a-70f1-489a-a3c0-2cad769fd378 admin admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:21:23.033546 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4687957a-70f1-489a-a3c0-2cad769fd378 admin admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:21:23.034015 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4687957a-70f1-489a-a3c0-2cad769fd378 admin admin] PATH_INFO: `/v3/roles` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:21:23.034872 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4687957a-70f1-489a-a3c0-2cad769fd378 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:21:23.041379 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4687957a-70f1-489a-a3c0-2cad769fd378 admin admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:21:23.067469 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 904/1811] 10.222.0.18 () {58 vars in 1122 bytes} [Tue Sep 13 03:21:22 2022] GET /identity/v3/roles => generated 1420 bytes in 67 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 13 03:21:23.075963 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67df70e3-3a6a-459f-bf0a-2b4f606d5166 admin admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:21:23.089049 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67df70e3-3a6a-459f-bf0a-2b4f606d5166 admin admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:21:23.101732 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01a267aa-8ded-4917-a9a7-e244b271e919 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76c5181fc7be45279ac00017bb4640b3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:21:23.101732 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01a267aa-8ded-4917-a9a7-e244b271e919 admin admin] REQUEST_METHOD: `PUT` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:21:23.101732 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01a267aa-8ded-4917-a9a7-e244b271e919 admin admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:21:23.102064 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01a267aa-8ded-4917-a9a7-e244b271e919 admin admin] PATH_INFO: `/v3/projects/2d6cc509ae8f4182ab0ba2b7a26710c4/users/552a8a35649d4566af72ff08f489a988/roles/1eb7a5b5203945c29e6d22d653f47847` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:21:23.132116 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01a267aa-8ded-4917-a9a7-e244b271e919 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2d6cc509ae8f4182ab0ba2b7a26710c4, user_id=552a8a35649d4566af72ff08f489a988, role_id=1eb7a5b5203945c29e6d22d653f47847)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:21:23.134856 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01a267aa-8ded-4917-a9a7-e244b271e919 admin admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:21:23.167128 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 908/1812] 10.222.0.18 () {60 vars in 1483 bytes} [Tue Sep 13 03:21:23 2022] PUT /identity/v3/projects/2d6cc509ae8f4182ab0ba2b7a26710c4/users/552a8a35649d4566af72ff08f489a988/roles/1eb7a5b5203945c29e6d22d653f47847 => generated 0 bytes in 85 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 03:21:23.171786 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c803b628-912e-4e93-96de-82d1ca8f2872 None None] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:21:23.172266 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c803b628-912e-4e93-96de-82d1ca8f2872 None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:21:23.172561 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c803b628-912e-4e93-96de-82d1ca8f2872 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:21:23.231703 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.auth.core [None req-c803b628-912e-4e93-96de-82d1ca8f2872 None None] MFA Rules not processed for user `552a8a35649d4566af72ff08f489a988`. Rule list: `[]` (Enabled: `True`). {{(pid=81400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 03:21:23.232919 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-c803b628-912e-4e93-96de-82d1ca8f2872 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:21:23.317915 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 905/1813] 10.222.0.18 () {58 vars in 958 bytes} [Tue Sep 13 03:21:23 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 13 03:21:23.329316 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01a267aa-8ded-4917-a9a7-e244b271e919 admin admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:21:23.331062 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-01a267aa-8ded-4917-a9a7-e244b271e919 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:21:23.414104 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01a267aa-8ded-4917-a9a7-e244b271e919 admin admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:21:23.418751 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-685586a6-4d26-4652-b6c9-a053dfd48340 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '924ee3f608d34a18b0ddc4695f102a56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b9d94f6a68f7470d9494f112175620dc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:21:23.420084 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-685586a6-4d26-4652-b6c9-a053dfd48340 service cinder] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:21:23.420084 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-685586a6-4d26-4652-b6c9-a053dfd48340 service cinder] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:21:23.420084 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-685586a6-4d26-4652-b6c9-a053dfd48340 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:21:23.431111 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-685586a6-4d26-4652-b6c9-a053dfd48340 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:21:23.432592 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-685586a6-4d26-4652-b6c9-a053dfd48340 service cinder] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:21:23.447263 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 909/1814] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Sep 13 03:21:23 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 118 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 13 03:21:23.468389 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c803b628-912e-4e93-96de-82d1ca8f2872 None None] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:21:23.557922 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c803b628-912e-4e93-96de-82d1ca8f2872 None None] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:21:23.562846 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97ed4e17-69c0-4525-930d-ff83a14bfc5e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76c5181fc7be45279ac00017bb4640b3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:21:23.563631 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ed4e17-69c0-4525-930d-ff83a14bfc5e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:21:23.563963 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ed4e17-69c0-4525-930d-ff83a14bfc5e admin admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:21:23.564258 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ed4e17-69c0-4525-930d-ff83a14bfc5e admin admin] PATH_INFO: `/v3/users/552a8a35649d4566af72ff08f489a988` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:21:23.565767 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97ed4e17-69c0-4525-930d-ff83a14bfc5e admin admin] RBAC: Authorizing `identity:delete_user(user_id=552a8a35649d4566af72ff08f489a988)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:21:23.568324 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97ed4e17-69c0-4525-930d-ff83a14bfc5e admin admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:21:23.656018 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=90,keystone:INSERT=3,keystone:DELETE=7 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:21:23.658172 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-97ed4e17-69c0-4525-930d-ff83a14bfc5e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '552a8a35649d4566af72ff08f489a988'} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:21:23.669273 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-97ed4e17-69c0-4525-930d-ff83a14bfc5e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '552a8a35649d4566af72ff08f489a988'} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:21:23.670396 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-97ed4e17-69c0-4525-930d-ff83a14bfc5e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '552a8a35649d4566af72ff08f489a988'} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:21:23.689580 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-97ed4e17-69c0-4525-930d-ff83a14bfc5e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '552a8a35649d4566af72ff08f489a988'} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:21:23.701959 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-97ed4e17-69c0-4525-930d-ff83a14bfc5e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '552a8a35649d4566af72ff08f489a988'} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:21:23.719607 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 906/1815] 10.222.0.18 () {58 vars in 1224 bytes} [Tue Sep 13 03:21:23 2022] DELETE /identity/v3/users/552a8a35649d4566af72ff08f489a988 => generated 0 bytes in 252 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 03:21:23.724296 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8833cace-cac4-4654-a39f-8074d7f0c46a None None] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:21:23.724841 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8833cace-cac4-4654-a39f-8074d7f0c46a None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:21:23.724841 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8833cace-cac4-4654-a39f-8074d7f0c46a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:21:23.752604 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.auth.core [None req-8833cace-cac4-4654-a39f-8074d7f0c46a None None] MFA Rules not processed for user `49f280ab086b4610855c4ffa378517d9`. Rule list: `[]` (Enabled: `True`). {{(pid=81399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 03:21:23.755636 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-8833cace-cac4-4654-a39f-8074d7f0c46a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:21:23.771704 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:21:23.835233 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 910/1816] 10.222.0.18 () {58 vars in 958 bytes} [Tue Sep 13 03:21:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 13 03:21:23.845043 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97ed4e17-69c0-4525-930d-ff83a14bfc5e admin admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:21:23.846451 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-97ed4e17-69c0-4525-930d-ff83a14bfc5e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:21:23.918885 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97ed4e17-69c0-4525-930d-ff83a14bfc5e admin admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:21:23.922871 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c011dc28-3111-45d2-a107-9fa66a80f914 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdffbc851086441ebd01022d1e4c2609', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b9d94f6a68f7470d9494f112175620dc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:21:23.923418 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c011dc28-3111-45d2-a107-9fa66a80f914 service neutron] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:21:23.923691 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c011dc28-3111-45d2-a107-9fa66a80f914 service neutron] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:21:23.923955 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c011dc28-3111-45d2-a107-9fa66a80f914 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:21:23.933926 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c011dc28-3111-45d2-a107-9fa66a80f914 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:21:23.935423 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c011dc28-3111-45d2-a107-9fa66a80f914 service neutron] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:21:23.947021 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 907/1817] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Sep 13 03:21:23 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 13 03:21:24.529075 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8833cace-cac4-4654-a39f-8074d7f0c46a None None] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:21:24.530562 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-8833cace-cac4-4654-a39f-8074d7f0c46a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:21:24.579790 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8833cace-cac4-4654-a39f-8074d7f0c46a None None] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:21:24.596979 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5a77051-0d20-4aed-9116-d0c608a2b9bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76c5181fc7be45279ac00017bb4640b3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:21:24.597634 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5a77051-0d20-4aed-9116-d0c608a2b9bf admin admin] REQUEST_METHOD: `DELETE` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:21:24.597910 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5a77051-0d20-4aed-9116-d0c608a2b9bf admin admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:21:24.598231 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5a77051-0d20-4aed-9116-d0c608a2b9bf admin admin] PATH_INFO: `/v3/projects/2d6cc509ae8f4182ab0ba2b7a26710c4` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:21:24.599815 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5a77051-0d20-4aed-9116-d0c608a2b9bf admin admin] RBAC: Authorizing `identity:delete_project(project_id=2d6cc509ae8f4182ab0ba2b7a26710c4)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:21:24.601808 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5a77051-0d20-4aed-9116-d0c608a2b9bf admin admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:21:24.695848 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-c5a77051-0d20-4aed-9116-d0c608a2b9bf admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2d6cc509ae8f4182ab0ba2b7a26710c4', 'request_id': 'req-c5a77051-0d20-4aed-9116-d0c608a2b9bf'} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:21:24.705747 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-c5a77051-0d20-4aed-9116-d0c608a2b9bf admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2d6cc509ae8f4182ab0ba2b7a26710c4', 'request_id': 'req-c5a77051-0d20-4aed-9116-d0c608a2b9bf'} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:21:24.718841 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-c5a77051-0d20-4aed-9116-d0c608a2b9bf admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2d6cc509ae8f4182ab0ba2b7a26710c4', 'request_id': 'req-c5a77051-0d20-4aed-9116-d0c608a2b9bf'} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:21:24.743988 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-c5a77051-0d20-4aed-9116-d0c608a2b9bf admin admin] The token cache is being invalidate because project 2d6cc509ae8f4182ab0ba2b7a26710c4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81399) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 03:21:24.744632 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-c5a77051-0d20-4aed-9116-d0c608a2b9bf admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:21:24.745536 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-c5a77051-0d20-4aed-9116-d0c608a2b9bf admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:21:24.752314 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1151274a-89a6-4ef9-b9a1-7accf8a814fd None None] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:21:24.753015 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1151274a-89a6-4ef9-b9a1-7accf8a814fd None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:21:24.753015 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1151274a-89a6-4ef9-b9a1-7accf8a814fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:21:24.756254 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 911/1818] 10.222.0.18 () {58 vars in 1233 bytes} [Tue Sep 13 03:21:24 2022] DELETE /identity/v3/projects/2d6cc509ae8f4182ab0ba2b7a26710c4 => generated 0 bytes in 219 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 03:21:24.774115 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.auth.core [None req-1151274a-89a6-4ef9-b9a1-7accf8a814fd None None] MFA Rules not processed for user `49f280ab086b4610855c4ffa378517d9`. Rule list: `[]` (Enabled: `True`). {{(pid=81400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 03:21:24.775345 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-1151274a-89a6-4ef9-b9a1-7accf8a814fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:21:24.840076 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 908/1819] 10.222.0.18 () {58 vars in 958 bytes} [Tue Sep 13 03:21:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 03:21:24.847821 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c23174fd-dffc-4464-b0bf-50a2db7e1a8e None None] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:21:24.848167 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c23174fd-dffc-4464-b0bf-50a2db7e1a8e None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:21:24.848448 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c23174fd-dffc-4464-b0bf-50a2db7e1a8e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:21:24.869044 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.auth.core [None req-c23174fd-dffc-4464-b0bf-50a2db7e1a8e None None] MFA Rules not processed for user `49f280ab086b4610855c4ffa378517d9`. Rule list: `[]` (Enabled: `True`). {{(pid=81399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 03:21:24.870377 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-c23174fd-dffc-4464-b0bf-50a2db7e1a8e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:21:24.880041 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 912/1820] 10.222.0.18 () {58 vars in 958 bytes} [Tue Sep 13 03:21:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 03:21:24.884854 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1151274a-89a6-4ef9-b9a1-7accf8a814fd None None] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:21:24.894796 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1151274a-89a6-4ef9-b9a1-7accf8a814fd None None] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:21:24.899486 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11c93d00-9ce5-4963-b9d7-5230edcacf77 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76c5181fc7be45279ac00017bb4640b3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:21:24.900094 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c93d00-9ce5-4963-b9d7-5230edcacf77 admin admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:21:24.900374 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c93d00-9ce5-4963-b9d7-5230edcacf77 admin admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:21:24.900641 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c93d00-9ce5-4963-b9d7-5230edcacf77 admin admin] PATH_INFO: `/v3/domains` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:21:24.901280 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11c93d00-9ce5-4963-b9d7-5230edcacf77 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 03:21:24.901583 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11c93d00-9ce5-4963-b9d7-5230edcacf77 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:21:24.903226 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11c93d00-9ce5-4963-b9d7-5230edcacf77 admin admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:21:24.915638 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 909/1821] 10.222.0.18 () {58 vars in 1153 bytes} [Tue Sep 13 03:21:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 03:21:24.920164 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c23174fd-dffc-4464-b0bf-50a2db7e1a8e None None] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:21:24.930438 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c23174fd-dffc-4464-b0bf-50a2db7e1a8e None None] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:21:24.935293 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a7b347f-b606-438e-bb10-8af9e2606f19 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76c5181fc7be45279ac00017bb4640b3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:21:24.935903 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a7b347f-b606-438e-bb10-8af9e2606f19 admin admin] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:21:24.936187 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a7b347f-b606-438e-bb10-8af9e2606f19 admin admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:21:24.936462 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a7b347f-b606-438e-bb10-8af9e2606f19 admin admin] PATH_INFO: `/v3/projects` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:21:24.937134 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a7b347f-b606-438e-bb10-8af9e2606f19 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:21:24.939090 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a7b347f-b606-438e-bb10-8af9e2606f19 admin admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:21:24.978010 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 913/1822] 10.222.0.18 () {60 vars in 1153 bytes} [Tue Sep 13 03:21:24 2022] POST /identity/v3/projects => generated 361 bytes in 59 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 03:21:24.982847 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11c93d00-9ce5-4963-b9d7-5230edcacf77 admin admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:21:24.993534 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11c93d00-9ce5-4963-b9d7-5230edcacf77 admin admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:21:25.049057 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=96,keystone:INSERT=1,keystone:DELETE=3 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:21:25.060980 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a39ab67-910b-4964-b341-561d0b670a09 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76c5181fc7be45279ac00017bb4640b3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:21:25.061597 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a39ab67-910b-4964-b341-561d0b670a09 admin admin] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:21:25.061873 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a39ab67-910b-4964-b341-561d0b670a09 admin admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:21:25.062144 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a39ab67-910b-4964-b341-561d0b670a09 admin admin] PATH_INFO: `/v3/users` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:21:25.062816 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a39ab67-910b-4964-b341-561d0b670a09 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:21:25.064750 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a39ab67-910b-4964-b341-561d0b670a09 admin admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:21:25.086133 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 910/1823] 10.222.0.18 () {60 vars in 1144 bytes} [Tue Sep 13 03:21:24 2022] POST /identity/v3/users => generated 281 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 13 03:21:25.090172 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a7b347f-b606-438e-bb10-8af9e2606f19 admin admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:21:25.101078 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a7b347f-b606-438e-bb10-8af9e2606f19 admin admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:21:25.105714 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99734bd5-2707-4177-9e57-26f0d7758bd9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76c5181fc7be45279ac00017bb4640b3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:21:25.106315 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99734bd5-2707-4177-9e57-26f0d7758bd9 admin admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:21:25.106599 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99734bd5-2707-4177-9e57-26f0d7758bd9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:21:25.106907 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99734bd5-2707-4177-9e57-26f0d7758bd9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:21:25.107545 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99734bd5-2707-4177-9e57-26f0d7758bd9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:21:25.109246 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99734bd5-2707-4177-9e57-26f0d7758bd9 admin admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:21:25.120293 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 914/1824] 10.222.0.18 () {58 vars in 1122 bytes} [Tue Sep 13 03:21:25 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 03:21:25.124336 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a39ab67-910b-4964-b341-561d0b670a09 admin admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:21:25.134438 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a39ab67-910b-4964-b341-561d0b670a09 admin admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:21:25.139066 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4c5fdf3-0f44-44a8-8bb3-ef8ab46eaef1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76c5181fc7be45279ac00017bb4640b3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:21:25.139664 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c5fdf3-0f44-44a8-8bb3-ef8ab46eaef1 admin admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:21:25.139942 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c5fdf3-0f44-44a8-8bb3-ef8ab46eaef1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:21:25.140213 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c5fdf3-0f44-44a8-8bb3-ef8ab46eaef1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:21:25.140845 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4c5fdf3-0f44-44a8-8bb3-ef8ab46eaef1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:21:25.142467 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4c5fdf3-0f44-44a8-8bb3-ef8ab46eaef1 admin admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:21:25.153257 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 911/1825] 10.222.0.18 () {58 vars in 1122 bytes} [Tue Sep 13 03:21:25 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 03:21:25.158473 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99734bd5-2707-4177-9e57-26f0d7758bd9 admin admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:21:25.169159 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99734bd5-2707-4177-9e57-26f0d7758bd9 admin admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:21:25.175276 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-356c017d-1976-4abf-9eec-54173f90611a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76c5181fc7be45279ac00017bb4640b3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:21:25.175933 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-356c017d-1976-4abf-9eec-54173f90611a admin admin] REQUEST_METHOD: `PUT` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:21:25.176284 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-356c017d-1976-4abf-9eec-54173f90611a admin admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:21:25.177078 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-356c017d-1976-4abf-9eec-54173f90611a admin admin] PATH_INFO: `/v3/projects/8e5f061a037a41abaec8b6e1007c6d5b/users/034264f4cf014ee78011729cbf140997/roles/1eb7a5b5203945c29e6d22d653f47847` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:21:25.197849 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-356c017d-1976-4abf-9eec-54173f90611a admin admin] RBAC: Authorizing `identity:create_grant(project_id=8e5f061a037a41abaec8b6e1007c6d5b, user_id=034264f4cf014ee78011729cbf140997, role_id=1eb7a5b5203945c29e6d22d653f47847)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:21:25.200758 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-356c017d-1976-4abf-9eec-54173f90611a admin admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:21:25.219301 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 915/1826] 10.222.0.18 () {60 vars in 1483 bytes} [Tue Sep 13 03:21:25 2022] PUT /identity/v3/projects/8e5f061a037a41abaec8b6e1007c6d5b/users/034264f4cf014ee78011729cbf140997/roles/1eb7a5b5203945c29e6d22d653f47847 => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 03:21:25.229352 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0d2e8bd-0ffb-4827-8a20-fe1ebb8d535b None None] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:21:25.229688 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0d2e8bd-0ffb-4827-8a20-fe1ebb8d535b None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:21:25.230073 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0d2e8bd-0ffb-4827-8a20-fe1ebb8d535b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:21:25.259894 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.auth.core [None req-a0d2e8bd-0ffb-4827-8a20-fe1ebb8d535b None None] MFA Rules not processed for user `034264f4cf014ee78011729cbf140997`. Rule list: `[]` (Enabled: `True`). {{(pid=81400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 03:21:25.261216 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-a0d2e8bd-0ffb-4827-8a20-fe1ebb8d535b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:21:25.329772 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 912/1827] 10.222.0.18 () {58 vars in 958 bytes} [Tue Sep 13 03:21:25 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 03:21:25.365214 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-356c017d-1976-4abf-9eec-54173f90611a admin admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:21:25.366956 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-356c017d-1976-4abf-9eec-54173f90611a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:21:25.408475 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:21:25.426414 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-356c017d-1976-4abf-9eec-54173f90611a admin admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:21:25.430084 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2bc174c-85af-490a-8952-2110ee75250d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '924ee3f608d34a18b0ddc4695f102a56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b9d94f6a68f7470d9494f112175620dc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:21:25.430717 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2bc174c-85af-490a-8952-2110ee75250d service cinder] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:21:25.431001 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2bc174c-85af-490a-8952-2110ee75250d service cinder] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:21:25.431277 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2bc174c-85af-490a-8952-2110ee75250d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:21:25.439431 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2bc174c-85af-490a-8952-2110ee75250d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:21:25.440852 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2bc174c-85af-490a-8952-2110ee75250d service cinder] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:21:25.449586 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 916/1828] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Sep 13 03:21:25 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 03:21:25.465889 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0d2e8bd-0ffb-4827-8a20-fe1ebb8d535b None None] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:21:25.529425 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0d2e8bd-0ffb-4827-8a20-fe1ebb8d535b None None] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:21:25.534025 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bca78ee-929a-487b-abdd-93302a70b53f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76c5181fc7be45279ac00017bb4640b3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:21:25.534672 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bca78ee-929a-487b-abdd-93302a70b53f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:21:25.534954 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bca78ee-929a-487b-abdd-93302a70b53f admin admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:21:25.535226 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bca78ee-929a-487b-abdd-93302a70b53f admin admin] PATH_INFO: `/v3/users/034264f4cf014ee78011729cbf140997` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:21:25.536957 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bca78ee-929a-487b-abdd-93302a70b53f admin admin] RBAC: Authorizing `identity:delete_user(user_id=034264f4cf014ee78011729cbf140997)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:21:25.538914 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bca78ee-929a-487b-abdd-93302a70b53f admin admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:21:25.604055 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-1bca78ee-929a-487b-abdd-93302a70b53f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '034264f4cf014ee78011729cbf140997'} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:21:25.613707 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-1bca78ee-929a-487b-abdd-93302a70b53f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '034264f4cf014ee78011729cbf140997'} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:21:25.614481 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-1bca78ee-929a-487b-abdd-93302a70b53f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '034264f4cf014ee78011729cbf140997'} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:21:25.626828 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=87,keystone:INSERT=4,keystone:DELETE=9 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:21:25.639367 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-1bca78ee-929a-487b-abdd-93302a70b53f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '034264f4cf014ee78011729cbf140997'} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:21:25.653527 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-1bca78ee-929a-487b-abdd-93302a70b53f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '034264f4cf014ee78011729cbf140997'} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:21:25.663994 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 913/1829] 10.222.0.18 () {58 vars in 1224 bytes} [Tue Sep 13 03:21:25 2022] DELETE /identity/v3/users/034264f4cf014ee78011729cbf140997 => generated 0 bytes in 199 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 03:21:25.669227 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08fd2553-4e46-47f8-9531-101d1b5d32e6 None None] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:21:25.669506 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08fd2553-4e46-47f8-9531-101d1b5d32e6 None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:21:25.670038 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08fd2553-4e46-47f8-9531-101d1b5d32e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:21:25.699265 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.auth.core [None req-08fd2553-4e46-47f8-9531-101d1b5d32e6 None None] MFA Rules not processed for user `49f280ab086b4610855c4ffa378517d9`. Rule list: `[]` (Enabled: `True`). {{(pid=81399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 03:21:25.700486 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-08fd2553-4e46-47f8-9531-101d1b5d32e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:21:25.769090 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 917/1830] 10.222.0.18 () {58 vars in 958 bytes} [Tue Sep 13 03:21:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 03:21:25.780699 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bca78ee-929a-487b-abdd-93302a70b53f admin admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:21:25.782253 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-1bca78ee-929a-487b-abdd-93302a70b53f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:21:25.856028 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bca78ee-929a-487b-abdd-93302a70b53f admin admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:21:25.860008 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c403398c-a087-4fe3-8507-1fabd3f3ff19 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdffbc851086441ebd01022d1e4c2609', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b9d94f6a68f7470d9494f112175620dc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:21:25.860711 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c403398c-a087-4fe3-8507-1fabd3f3ff19 service neutron] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:21:25.861053 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c403398c-a087-4fe3-8507-1fabd3f3ff19 service neutron] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:21:25.861330 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c403398c-a087-4fe3-8507-1fabd3f3ff19 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:21:25.873360 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c403398c-a087-4fe3-8507-1fabd3f3ff19 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:21:25.874993 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c403398c-a087-4fe3-8507-1fabd3f3ff19 service neutron] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:21:25.885788 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 914/1831] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Sep 13 03:21:25 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 03:21:26.248415 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08fd2553-4e46-47f8-9531-101d1b5d32e6 None None] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:21:26.252099 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-08fd2553-4e46-47f8-9531-101d1b5d32e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:21:26.270442 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08fd2553-4e46-47f8-9531-101d1b5d32e6 None None] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:21:26.283465 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06706321-d7eb-4b66-9f3d-30ebb1a1e9a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76c5181fc7be45279ac00017bb4640b3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:21:26.284212 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06706321-d7eb-4b66-9f3d-30ebb1a1e9a4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:21:26.284600 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06706321-d7eb-4b66-9f3d-30ebb1a1e9a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:21:26.284941 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06706321-d7eb-4b66-9f3d-30ebb1a1e9a4 admin admin] PATH_INFO: `/v3/projects/8e5f061a037a41abaec8b6e1007c6d5b` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:21:26.286898 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06706321-d7eb-4b66-9f3d-30ebb1a1e9a4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8e5f061a037a41abaec8b6e1007c6d5b)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:21:26.289615 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06706321-d7eb-4b66-9f3d-30ebb1a1e9a4 admin admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:21:26.396854 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-06706321-d7eb-4b66-9f3d-30ebb1a1e9a4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8e5f061a037a41abaec8b6e1007c6d5b', 'request_id': 'req-06706321-d7eb-4b66-9f3d-30ebb1a1e9a4'} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:21:26.423744 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-06706321-d7eb-4b66-9f3d-30ebb1a1e9a4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8e5f061a037a41abaec8b6e1007c6d5b', 'request_id': 'req-06706321-d7eb-4b66-9f3d-30ebb1a1e9a4'} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:21:26.437547 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-06706321-d7eb-4b66-9f3d-30ebb1a1e9a4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8e5f061a037a41abaec8b6e1007c6d5b', 'request_id': 'req-06706321-d7eb-4b66-9f3d-30ebb1a1e9a4'} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:21:26.463392 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-06706321-d7eb-4b66-9f3d-30ebb1a1e9a4 admin admin] The token cache is being invalidate because project 8e5f061a037a41abaec8b6e1007c6d5b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81399) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 03:21:26.463760 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-06706321-d7eb-4b66-9f3d-30ebb1a1e9a4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:21:26.464620 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-06706321-d7eb-4b66-9f3d-30ebb1a1e9a4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:21:26.469691 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 918/1832] 10.222.0.18 () {58 vars in 1233 bytes} [Tue Sep 13 03:21:26 2022] DELETE /identity/v3/projects/8e5f061a037a41abaec8b6e1007c6d5b => generated 0 bytes in 219 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 03:21:26.471431 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a56f29b1-5b4f-4ec4-8914-f4a9c84b5786 None None] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:21:26.471712 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a56f29b1-5b4f-4ec4-8914-f4a9c84b5786 None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:21:26.471988 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a56f29b1-5b4f-4ec4-8914-f4a9c84b5786 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:21:26.504797 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.auth.core [None req-a56f29b1-5b4f-4ec4-8914-f4a9c84b5786 None None] MFA Rules not processed for user `49f280ab086b4610855c4ffa378517d9`. Rule list: `[]` (Enabled: `True`). {{(pid=81400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 03:21:26.506470 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-a56f29b1-5b4f-4ec4-8914-f4a9c84b5786 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:21:26.602878 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 915/1833] 10.222.0.18 () {58 vars in 958 bytes} [Tue Sep 13 03:21:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 13 03:21:26.605221 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a42865-d804-499b-a296-e867af9a298d None None] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:21:26.605630 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a42865-d804-499b-a296-e867af9a298d None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:21:26.605932 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a42865-d804-499b-a296-e867af9a298d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:21:26.628611 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.auth.core [None req-10a42865-d804-499b-a296-e867af9a298d None None] MFA Rules not processed for user `49f280ab086b4610855c4ffa378517d9`. Rule list: `[]` (Enabled: `True`). {{(pid=81399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 03:21:26.629861 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-10a42865-d804-499b-a296-e867af9a298d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:21:26.638437 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 919/1834] 10.222.0.18 () {58 vars in 958 bytes} [Tue Sep 13 03:21:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 13 03:21:26.642684 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a56f29b1-5b4f-4ec4-8914-f4a9c84b5786 None None] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:21:26.653496 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a56f29b1-5b4f-4ec4-8914-f4a9c84b5786 None None] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:21:26.659503 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c828c3e6-fe86-4eeb-a522-a6f026052499 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76c5181fc7be45279ac00017bb4640b3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:21:26.662226 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c828c3e6-fe86-4eeb-a522-a6f026052499 admin admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:21:26.662226 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c828c3e6-fe86-4eeb-a522-a6f026052499 admin admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:21:26.662226 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c828c3e6-fe86-4eeb-a522-a6f026052499 admin admin] PATH_INFO: `/v3/domains` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:21:26.662226 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c828c3e6-fe86-4eeb-a522-a6f026052499 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 03:21:26.662226 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c828c3e6-fe86-4eeb-a522-a6f026052499 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:21:26.663909 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c828c3e6-fe86-4eeb-a522-a6f026052499 admin admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:21:26.678469 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 916/1835] 10.222.0.18 () {58 vars in 1153 bytes} [Tue Sep 13 03:21:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 03:21:26.682696 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10a42865-d804-499b-a296-e867af9a298d None None] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:21:26.692803 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10a42865-d804-499b-a296-e867af9a298d None None] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:21:26.697944 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f08b2a0-ddfe-4b89-b657-819eef1d9036 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76c5181fc7be45279ac00017bb4640b3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:21:26.698548 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f08b2a0-ddfe-4b89-b657-819eef1d9036 admin admin] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:21:26.698866 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f08b2a0-ddfe-4b89-b657-819eef1d9036 admin admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:21:26.699162 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f08b2a0-ddfe-4b89-b657-819eef1d9036 admin admin] PATH_INFO: `/v3/projects` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:21:26.699807 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f08b2a0-ddfe-4b89-b657-819eef1d9036 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:21:26.701764 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f08b2a0-ddfe-4b89-b657-819eef1d9036 admin admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:21:26.730858 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 920/1836] 10.222.0.18 () {60 vars in 1153 bytes} [Tue Sep 13 03:21:26 2022] POST /identity/v3/projects => generated 381 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 13 03:21:26.734855 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c828c3e6-fe86-4eeb-a522-a6f026052499 admin admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:21:26.746586 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c828c3e6-fe86-4eeb-a522-a6f026052499 admin admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:21:26.805922 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f2053d2-5c89-4ac3-ac4a-910af5a60a4d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76c5181fc7be45279ac00017bb4640b3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:21:26.806546 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f2053d2-5c89-4ac3-ac4a-910af5a60a4d admin admin] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:21:26.806753 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f2053d2-5c89-4ac3-ac4a-910af5a60a4d admin admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:21:26.807020 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f2053d2-5c89-4ac3-ac4a-910af5a60a4d admin admin] PATH_INFO: `/v3/users` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:21:26.807657 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f2053d2-5c89-4ac3-ac4a-910af5a60a4d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:21:26.809558 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f2053d2-5c89-4ac3-ac4a-910af5a60a4d admin admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:21:26.825397 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=96,keystone:DELETE=1,keystone:INSERT=3 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:21:26.831023 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 917/1837] 10.222.0.18 () {60 vars in 1144 bytes} [Tue Sep 13 03:21:26 2022] POST /identity/v3/users => generated 291 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 03:21:26.835694 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f08b2a0-ddfe-4b89-b657-819eef1d9036 admin admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:21:26.846348 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f08b2a0-ddfe-4b89-b657-819eef1d9036 admin admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:21:26.851516 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dafb9589-ab15-4c95-ac3a-becf2c9958c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76c5181fc7be45279ac00017bb4640b3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:21:26.852168 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dafb9589-ab15-4c95-ac3a-becf2c9958c1 admin admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:21:26.852376 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dafb9589-ab15-4c95-ac3a-becf2c9958c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:21:26.852649 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dafb9589-ab15-4c95-ac3a-becf2c9958c1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:21:26.853373 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dafb9589-ab15-4c95-ac3a-becf2c9958c1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:21:26.854896 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dafb9589-ab15-4c95-ac3a-becf2c9958c1 admin admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:21:26.864938 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 921/1838] 10.222.0.18 () {58 vars in 1122 bytes} [Tue Sep 13 03:21:26 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 03:21:26.869082 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f2053d2-5c89-4ac3-ac4a-910af5a60a4d admin admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:21:26.879312 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f2053d2-5c89-4ac3-ac4a-910af5a60a4d admin admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:21:26.884472 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00cfb7c5-6034-4792-84bc-fb5fcdc1cf1f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76c5181fc7be45279ac00017bb4640b3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:21:26.885073 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00cfb7c5-6034-4792-84bc-fb5fcdc1cf1f admin admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:21:26.885437 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00cfb7c5-6034-4792-84bc-fb5fcdc1cf1f admin admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:21:26.885713 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00cfb7c5-6034-4792-84bc-fb5fcdc1cf1f admin admin] PATH_INFO: `/v3/roles` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:21:26.886381 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00cfb7c5-6034-4792-84bc-fb5fcdc1cf1f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:21:26.888028 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00cfb7c5-6034-4792-84bc-fb5fcdc1cf1f admin admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:21:26.899433 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 918/1839] 10.222.0.18 () {58 vars in 1122 bytes} [Tue Sep 13 03:21:26 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 03:21:26.903554 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dafb9589-ab15-4c95-ac3a-becf2c9958c1 admin admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:21:26.914686 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dafb9589-ab15-4c95-ac3a-becf2c9958c1 admin admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:21:26.919922 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0610ec6-44af-436f-af0d-a4276fd83a41 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76c5181fc7be45279ac00017bb4640b3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:21:26.920580 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0610ec6-44af-436f-af0d-a4276fd83a41 admin admin] REQUEST_METHOD: `PUT` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:21:26.920944 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0610ec6-44af-436f-af0d-a4276fd83a41 admin admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:21:26.921255 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0610ec6-44af-436f-af0d-a4276fd83a41 admin admin] PATH_INFO: `/v3/projects/b3ef3a6638114e1f9c838b00285f43f6/users/466f990dcac84f4faec701b28616273e/roles/1eb7a5b5203945c29e6d22d653f47847` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:21:26.936375 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0610ec6-44af-436f-af0d-a4276fd83a41 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b3ef3a6638114e1f9c838b00285f43f6, user_id=466f990dcac84f4faec701b28616273e, role_id=1eb7a5b5203945c29e6d22d653f47847)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:21:26.939127 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0610ec6-44af-436f-af0d-a4276fd83a41 admin admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:21:26.947986 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:21:26.952939 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 922/1840] 10.222.0.18 () {60 vars in 1483 bytes} [Tue Sep 13 03:21:26 2022] PUT /identity/v3/projects/b3ef3a6638114e1f9c838b00285f43f6/users/466f990dcac84f4faec701b28616273e/roles/1eb7a5b5203945c29e6d22d653f47847 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 03:21:26.959038 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4912f643-d2bc-4d98-828a-0e796c1723d2 None None] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:21:26.959448 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4912f643-d2bc-4d98-828a-0e796c1723d2 None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:21:26.959728 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4912f643-d2bc-4d98-828a-0e796c1723d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:21:26.980517 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.auth.core [None req-4912f643-d2bc-4d98-828a-0e796c1723d2 None None] MFA Rules not processed for user `49f280ab086b4610855c4ffa378517d9`. Rule list: `[]` (Enabled: `True`). {{(pid=81400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 03:21:26.981738 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-4912f643-d2bc-4d98-828a-0e796c1723d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:21:27.041060 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 919/1841] 10.222.0.18 () {58 vars in 958 bytes} [Tue Sep 13 03:21:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 03:21:27.050881 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0610ec6-44af-436f-af0d-a4276fd83a41 admin admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:21:27.052653 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-e0610ec6-44af-436f-af0d-a4276fd83a41 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:21:27.112499 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0610ec6-44af-436f-af0d-a4276fd83a41 admin admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:21:27.116152 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c576c4d7-9ba7-483f-95fc-13a0d688a216 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdffbc851086441ebd01022d1e4c2609', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b9d94f6a68f7470d9494f112175620dc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:21:27.116793 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c576c4d7-9ba7-483f-95fc-13a0d688a216 service neutron] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:21:27.117080 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c576c4d7-9ba7-483f-95fc-13a0d688a216 service neutron] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:21:27.117353 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c576c4d7-9ba7-483f-95fc-13a0d688a216 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:21:27.128223 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c576c4d7-9ba7-483f-95fc-13a0d688a216 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:21:27.130900 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c576c4d7-9ba7-483f-95fc-13a0d688a216 service neutron] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:21:27.141093 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 923/1842] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Sep 13 03:21:27 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 13 03:21:27.989169 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44402bce-7dc3-4cb3-95c8-28a0cb9c8377 None None] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:21:27.989683 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44402bce-7dc3-4cb3-95c8-28a0cb9c8377 None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:21:27.989808 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44402bce-7dc3-4cb3-95c8-28a0cb9c8377 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:21:28.049810 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.auth.core [None req-44402bce-7dc3-4cb3-95c8-28a0cb9c8377 None None] MFA Rules not processed for user `466f990dcac84f4faec701b28616273e`. Rule list: `[]` (Enabled: `True`). {{(pid=81400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 03:21:28.051345 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-44402bce-7dc3-4cb3-95c8-28a0cb9c8377 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:21:28.253972 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 920/1843] 10.222.0.18 () {58 vars in 958 bytes} [Tue Sep 13 03:21:27 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 266 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 03:21:28.264448 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c576c4d7-9ba7-483f-95fc-13a0d688a216 service neutron] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:21:28.266402 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-c576c4d7-9ba7-483f-95fc-13a0d688a216 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:21:28.358730 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c576c4d7-9ba7-483f-95fc-13a0d688a216 service neutron] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:21:28.362448 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06cd0d7f-309d-44cc-8c78-f48c568f6754 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '924ee3f608d34a18b0ddc4695f102a56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b9d94f6a68f7470d9494f112175620dc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:21:28.363092 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06cd0d7f-309d-44cc-8c78-f48c568f6754 service cinder] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:21:28.363381 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06cd0d7f-309d-44cc-8c78-f48c568f6754 service cinder] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:21:28.364191 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06cd0d7f-309d-44cc-8c78-f48c568f6754 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:21:28.374765 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06cd0d7f-309d-44cc-8c78-f48c568f6754 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:21:28.376255 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06cd0d7f-309d-44cc-8c78-f48c568f6754 service cinder] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:21:28.385773 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 924/1844] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Sep 13 03:21:28 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 122 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 03:21:31.099570 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44402bce-7dc3-4cb3-95c8-28a0cb9c8377 None None] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:21:31.101286 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-44402bce-7dc3-4cb3-95c8-28a0cb9c8377 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:21:31.120705 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44402bce-7dc3-4cb3-95c8-28a0cb9c8377 None None] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:21:31.130595 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-141d1b37-ccc3-46f8-814d-b712ddf40bfb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76c5181fc7be45279ac00017bb4640b3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:21:31.131357 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-141d1b37-ccc3-46f8-814d-b712ddf40bfb admin admin] REQUEST_METHOD: `DELETE` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:21:31.131702 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-141d1b37-ccc3-46f8-814d-b712ddf40bfb admin admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:21:31.132036 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-141d1b37-ccc3-46f8-814d-b712ddf40bfb admin admin] PATH_INFO: `/v3/users/76d7cc0174ce4e0ab993d6b1f1c466a6` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:21:31.133290 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-141d1b37-ccc3-46f8-814d-b712ddf40bfb admin admin] RBAC: Authorizing `identity:delete_user(user_id=76d7cc0174ce4e0ab993d6b1f1c466a6)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:21:31.135339 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-141d1b37-ccc3-46f8-814d-b712ddf40bfb admin admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:21:31.209631 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-141d1b37-ccc3-46f8-814d-b712ddf40bfb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '76d7cc0174ce4e0ab993d6b1f1c466a6'} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:21:31.227237 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-141d1b37-ccc3-46f8-814d-b712ddf40bfb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '76d7cc0174ce4e0ab993d6b1f1c466a6'} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:21:31.228060 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-141d1b37-ccc3-46f8-814d-b712ddf40bfb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '76d7cc0174ce4e0ab993d6b1f1c466a6'} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:21:31.248426 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-141d1b37-ccc3-46f8-814d-b712ddf40bfb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '76d7cc0174ce4e0ab993d6b1f1c466a6'} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:21:31.251584 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:21:31.260050 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-141d1b37-ccc3-46f8-814d-b712ddf40bfb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '76d7cc0174ce4e0ab993d6b1f1c466a6'} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:21:31.270698 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 921/1845] 10.222.0.18 () {58 vars in 1224 bytes} [Tue Sep 13 03:21:31 2022] DELETE /identity/v3/users/76d7cc0174ce4e0ab993d6b1f1c466a6 => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 03:21:31.277026 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-440ccf8f-a2c1-4e11-8f21-6be8ea4d5280 None None] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:21:31.278197 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-440ccf8f-a2c1-4e11-8f21-6be8ea4d5280 None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:21:31.278604 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-440ccf8f-a2c1-4e11-8f21-6be8ea4d5280 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:21:31.300225 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.auth.core [None req-440ccf8f-a2c1-4e11-8f21-6be8ea4d5280 None None] MFA Rules not processed for user `49f280ab086b4610855c4ffa378517d9`. Rule list: `[]` (Enabled: `True`). {{(pid=81399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 03:21:31.301431 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-440ccf8f-a2c1-4e11-8f21-6be8ea4d5280 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:21:31.369835 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 925/1846] 10.222.0.18 () {58 vars in 958 bytes} [Tue Sep 13 03:21:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 03:21:31.380341 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-141d1b37-ccc3-46f8-814d-b712ddf40bfb admin admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:21:31.381915 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-141d1b37-ccc3-46f8-814d-b712ddf40bfb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:21:31.444512 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-141d1b37-ccc3-46f8-814d-b712ddf40bfb admin admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:21:31.448343 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc1a4c19-5e63-4426-b2e1-2b28d4294de8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdffbc851086441ebd01022d1e4c2609', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b9d94f6a68f7470d9494f112175620dc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:21:31.448959 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc1a4c19-5e63-4426-b2e1-2b28d4294de8 service neutron] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:21:31.449256 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc1a4c19-5e63-4426-b2e1-2b28d4294de8 service neutron] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:21:31.449520 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc1a4c19-5e63-4426-b2e1-2b28d4294de8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:21:31.459253 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc1a4c19-5e63-4426-b2e1-2b28d4294de8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:21:31.460684 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc1a4c19-5e63-4426-b2e1-2b28d4294de8 service neutron] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:21:31.472149 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 922/1847] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Sep 13 03:21:31 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 13 03:21:31.759673 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-440ccf8f-a2c1-4e11-8f21-6be8ea4d5280 None None] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:21:31.761354 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-440ccf8f-a2c1-4e11-8f21-6be8ea4d5280 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:21:31.774431 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-440ccf8f-a2c1-4e11-8f21-6be8ea4d5280 None None] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:21:31.779288 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b1bed0b-35b8-46a4-b0b8-99c38b44aa84 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76c5181fc7be45279ac00017bb4640b3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:21:31.779942 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b1bed0b-35b8-46a4-b0b8-99c38b44aa84 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:21:31.780209 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b1bed0b-35b8-46a4-b0b8-99c38b44aa84 admin admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:21:31.780476 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b1bed0b-35b8-46a4-b0b8-99c38b44aa84 admin admin] PATH_INFO: `/v3/projects/7c86a7b399d24c1ab23c826eaddee1d4` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:21:31.781849 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b1bed0b-35b8-46a4-b0b8-99c38b44aa84 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7c86a7b399d24c1ab23c826eaddee1d4)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:21:31.783552 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b1bed0b-35b8-46a4-b0b8-99c38b44aa84 admin admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:21:31.836473 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:21:31.845583 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-0b1bed0b-35b8-46a4-b0b8-99c38b44aa84 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7c86a7b399d24c1ab23c826eaddee1d4', 'request_id': 'req-0b1bed0b-35b8-46a4-b0b8-99c38b44aa84'} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:21:31.852375 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-0b1bed0b-35b8-46a4-b0b8-99c38b44aa84 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7c86a7b399d24c1ab23c826eaddee1d4', 'request_id': 'req-0b1bed0b-35b8-46a4-b0b8-99c38b44aa84'} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:21:31.860332 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-0b1bed0b-35b8-46a4-b0b8-99c38b44aa84 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7c86a7b399d24c1ab23c826eaddee1d4', 'request_id': 'req-0b1bed0b-35b8-46a4-b0b8-99c38b44aa84'} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:21:31.877992 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-0b1bed0b-35b8-46a4-b0b8-99c38b44aa84 admin admin] The token cache is being invalidate because project 7c86a7b399d24c1ab23c826eaddee1d4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81399) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 03:21:31.878546 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-0b1bed0b-35b8-46a4-b0b8-99c38b44aa84 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:21:31.883156 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-0b1bed0b-35b8-46a4-b0b8-99c38b44aa84 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:21:31.884833 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 926/1848] 10.222.0.18 () {58 vars in 1233 bytes} [Tue Sep 13 03:21:31 2022] DELETE /identity/v3/projects/7c86a7b399d24c1ab23c826eaddee1d4 => generated 0 bytes in 126 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 03:21:31.891493 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b1bed0b-35b8-46a4-b0b8-99c38b44aa84 admin admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:21:31.892166 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca6a1507-9ccc-4ddf-bb10-1d95b738eee7 None None] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:21:31.892450 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca6a1507-9ccc-4ddf-bb10-1d95b738eee7 None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:21:31.892727 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca6a1507-9ccc-4ddf-bb10-1d95b738eee7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:21:31.892849 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-0b1bed0b-35b8-46a4-b0b8-99c38b44aa84 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:21:31.917785 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.auth.core [None req-ca6a1507-9ccc-4ddf-bb10-1d95b738eee7 None None] MFA Rules not processed for user `466f990dcac84f4faec701b28616273e`. Rule list: `[]` (Enabled: `True`). {{(pid=81400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 03:21:31.919027 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-ca6a1507-9ccc-4ddf-bb10-1d95b738eee7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:21:31.967606 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b1bed0b-35b8-46a4-b0b8-99c38b44aa84 admin admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:21:31.972356 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29b24a41-cad2-42c3-9019-56cf7e4ac7bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76c5181fc7be45279ac00017bb4640b3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:21:31.973018 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b24a41-cad2-42c3-9019-56cf7e4ac7bd admin admin] REQUEST_METHOD: `DELETE` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:21:31.973448 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b24a41-cad2-42c3-9019-56cf7e4ac7bd admin admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:21:31.973721 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b24a41-cad2-42c3-9019-56cf7e4ac7bd admin admin] PATH_INFO: `/v3/users/7cbbcd0d1cba4052baf9c7d58370d5f4` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:21:31.974930 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29b24a41-cad2-42c3-9019-56cf7e4ac7bd admin admin] RBAC: Authorizing `identity:delete_user(user_id=7cbbcd0d1cba4052baf9c7d58370d5f4)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:21:31.976877 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29b24a41-cad2-42c3-9019-56cf7e4ac7bd admin admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:21:31.989526 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 923/1849] 10.222.0.18 () {58 vars in 958 bytes} [Tue Sep 13 03:21:31 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 03:21:32.043664 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-29b24a41-cad2-42c3-9019-56cf7e4ac7bd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7cbbcd0d1cba4052baf9c7d58370d5f4'} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:21:32.052022 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-29b24a41-cad2-42c3-9019-56cf7e4ac7bd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7cbbcd0d1cba4052baf9c7d58370d5f4'} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:21:32.064732 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-29b24a41-cad2-42c3-9019-56cf7e4ac7bd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7cbbcd0d1cba4052baf9c7d58370d5f4'} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:21:32.065669 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-29b24a41-cad2-42c3-9019-56cf7e4ac7bd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7cbbcd0d1cba4052baf9c7d58370d5f4'} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:21:32.073422 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-29b24a41-cad2-42c3-9019-56cf7e4ac7bd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7cbbcd0d1cba4052baf9c7d58370d5f4'} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:21:32.079548 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 927/1850] 10.222.0.18 () {58 vars in 1224 bytes} [Tue Sep 13 03:21:31 2022] DELETE /identity/v3/users/7cbbcd0d1cba4052baf9c7d58370d5f4 => generated 0 bytes in 188 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 03:21:32.447422 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca6a1507-9ccc-4ddf-bb10-1d95b738eee7 None None] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:21:32.469301 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca6a1507-9ccc-4ddf-bb10-1d95b738eee7 None None] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:21:32.533151 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bafda7c-475b-4171-8418-7cc32d951120 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76c5181fc7be45279ac00017bb4640b3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:21:32.533866 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bafda7c-475b-4171-8418-7cc32d951120 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:21:32.534224 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bafda7c-475b-4171-8418-7cc32d951120 admin admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:21:32.534572 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bafda7c-475b-4171-8418-7cc32d951120 admin admin] PATH_INFO: `/v3/projects/30a882ea0e7b46039e44abe7e9d957b1` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:21:32.536351 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bafda7c-475b-4171-8418-7cc32d951120 admin admin] RBAC: Authorizing `identity:delete_project(project_id=30a882ea0e7b46039e44abe7e9d957b1)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:21:32.538509 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bafda7c-475b-4171-8418-7cc32d951120 admin admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:21:32.584301 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:21:32.593575 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-4bafda7c-475b-4171-8418-7cc32d951120 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '30a882ea0e7b46039e44abe7e9d957b1', 'request_id': 'req-4bafda7c-475b-4171-8418-7cc32d951120'} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:21:32.601318 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-4bafda7c-475b-4171-8418-7cc32d951120 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '30a882ea0e7b46039e44abe7e9d957b1', 'request_id': 'req-4bafda7c-475b-4171-8418-7cc32d951120'} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:21:32.609186 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-4bafda7c-475b-4171-8418-7cc32d951120 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '30a882ea0e7b46039e44abe7e9d957b1', 'request_id': 'req-4bafda7c-475b-4171-8418-7cc32d951120'} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:21:32.625032 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-4bafda7c-475b-4171-8418-7cc32d951120 admin admin] The token cache is being invalidate because project 30a882ea0e7b46039e44abe7e9d957b1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81400) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 03:21:32.625320 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-4bafda7c-475b-4171-8418-7cc32d951120 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:21:32.625958 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-4bafda7c-475b-4171-8418-7cc32d951120 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:21:32.627897 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 924/1851] 10.222.0.18 () {58 vars in 1233 bytes} [Tue Sep 13 03:21:32 2022] DELETE /identity/v3/projects/30a882ea0e7b46039e44abe7e9d957b1 => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 03:21:32.632453 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae8e7a8b-83ab-44d4-a582-257096572cc1 None None] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:21:32.637841 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae8e7a8b-83ab-44d4-a582-257096572cc1 None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:21:32.638380 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae8e7a8b-83ab-44d4-a582-257096572cc1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:21:32.667111 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.auth.core [None req-ae8e7a8b-83ab-44d4-a582-257096572cc1 None None] MFA Rules not processed for user `49f280ab086b4610855c4ffa378517d9`. Rule list: `[]` (Enabled: `True`). {{(pid=81399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 03:21:32.668474 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-ae8e7a8b-83ab-44d4-a582-257096572cc1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:21:32.724632 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:21:32.754445 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 928/1852] 10.222.0.18 () {58 vars in 958 bytes} [Tue Sep 13 03:21:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 03:21:32.762605 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-303ab116-312f-4ce4-a506-14785e7ddb5b None None] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:21:32.762911 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-303ab116-312f-4ce4-a506-14785e7ddb5b None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:21:32.763183 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-303ab116-312f-4ce4-a506-14785e7ddb5b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:21:32.809469 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.auth.core [None req-303ab116-312f-4ce4-a506-14785e7ddb5b None None] MFA Rules not processed for user `49f280ab086b4610855c4ffa378517d9`. Rule list: `[]` (Enabled: `True`). {{(pid=81400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 03:21:32.815677 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-303ab116-312f-4ce4-a506-14785e7ddb5b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:21:32.849726 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 925/1853] 10.222.0.18 () {58 vars in 958 bytes} [Tue Sep 13 03:21:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 74 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 03:21:32.849831 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae8e7a8b-83ab-44d4-a582-257096572cc1 None None] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:21:32.855244 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae8e7a8b-83ab-44d4-a582-257096572cc1 None None] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:21:32.863462 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24723c76-36d5-4c5f-b34b-97918cd03ade admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76c5181fc7be45279ac00017bb4640b3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:21:32.864517 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24723c76-36d5-4c5f-b34b-97918cd03ade admin admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:21:32.865042 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24723c76-36d5-4c5f-b34b-97918cd03ade admin admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:21:32.865397 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24723c76-36d5-4c5f-b34b-97918cd03ade admin admin] PATH_INFO: `/v3/domains` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:21:32.866199 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24723c76-36d5-4c5f-b34b-97918cd03ade admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 03:21:32.866509 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24723c76-36d5-4c5f-b34b-97918cd03ade admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:21:32.868262 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24723c76-36d5-4c5f-b34b-97918cd03ade admin admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:21:32.882525 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 929/1854] 10.222.0.18 () {58 vars in 1153 bytes} [Tue Sep 13 03:21:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 03:21:32.886879 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-303ab116-312f-4ce4-a506-14785e7ddb5b None None] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:21:32.897542 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-303ab116-312f-4ce4-a506-14785e7ddb5b None None] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:21:32.920371 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-054be609-92c9-4615-96bf-047c0d9e55d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76c5181fc7be45279ac00017bb4640b3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:21:32.921224 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-054be609-92c9-4615-96bf-047c0d9e55d7 admin admin] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:21:32.923384 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-054be609-92c9-4615-96bf-047c0d9e55d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:21:32.923784 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-054be609-92c9-4615-96bf-047c0d9e55d7 admin admin] PATH_INFO: `/v3/projects` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:21:32.924816 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-054be609-92c9-4615-96bf-047c0d9e55d7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:21:32.927226 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-054be609-92c9-4615-96bf-047c0d9e55d7 admin admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:21:32.988893 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 926/1855] 10.222.0.18 () {60 vars in 1153 bytes} [Tue Sep 13 03:21:32 2022] POST /identity/v3/projects => generated 373 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 13 03:21:32.992743 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24723c76-36d5-4c5f-b34b-97918cd03ade admin admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:21:33.002885 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24723c76-36d5-4c5f-b34b-97918cd03ade admin admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:21:33.052136 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a90ec538-fd47-4255-a274-732bb57811a8 None None] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:21:33.052136 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a90ec538-fd47-4255-a274-732bb57811a8 None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:21:33.052522 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a90ec538-fd47-4255-a274-732bb57811a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:21:33.109502 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.auth.core [None req-a90ec538-fd47-4255-a274-732bb57811a8 None None] MFA Rules not processed for user `cdffbc851086441ebd01022d1e4c2609`. Rule list: `[]` (Enabled: `True`). {{(pid=81400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 03:21:33.111444 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-a90ec538-fd47-4255-a274-732bb57811a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:21:33.131740 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09baa759-f00d-45a4-a258-3445cbd552e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76c5181fc7be45279ac00017bb4640b3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:21:33.133941 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09baa759-f00d-45a4-a258-3445cbd552e0 admin admin] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:21:33.135221 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09baa759-f00d-45a4-a258-3445cbd552e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:21:33.136042 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09baa759-f00d-45a4-a258-3445cbd552e0 admin admin] PATH_INFO: `/v3/users` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:21:33.140539 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09baa759-f00d-45a4-a258-3445cbd552e0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:21:33.143023 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09baa759-f00d-45a4-a258-3445cbd552e0 admin admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:21:33.194837 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 930/1856] 10.222.0.18 () {60 vars in 1144 bytes} [Tue Sep 13 03:21:32 2022] POST /identity/v3/users => generated 287 bytes in 203 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 03:21:33.196497 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09baa759-f00d-45a4-a258-3445cbd552e0 admin admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:21:33.207339 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09baa759-f00d-45a4-a258-3445cbd552e0 admin admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:21:33.214189 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d138f62-d877-4875-8d68-8a1c60c2a841 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76c5181fc7be45279ac00017bb4640b3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:21:33.215676 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d138f62-d877-4875-8d68-8a1c60c2a841 admin admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:21:33.215964 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d138f62-d877-4875-8d68-8a1c60c2a841 admin admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:21:33.216236 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d138f62-d877-4875-8d68-8a1c60c2a841 admin admin] PATH_INFO: `/v3/roles` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:21:33.216874 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d138f62-d877-4875-8d68-8a1c60c2a841 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:21:33.219056 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d138f62-d877-4875-8d68-8a1c60c2a841 admin admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:21:33.230760 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 931/1857] 10.222.0.18 () {58 vars in 1122 bytes} [Tue Sep 13 03:21:33 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 03:21:33.231575 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 927/1858] 10.222.0.57 () {58 vars in 1015 bytes} [Tue Sep 13 03:21:33 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 13 03:21:33.235712 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d138f62-d877-4875-8d68-8a1c60c2a841 admin admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:21:33.246569 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d138f62-d877-4875-8d68-8a1c60c2a841 admin admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:21:33.251920 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a890ff47-8a9a-45b0-a98e-094b7d115a8c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76c5181fc7be45279ac00017bb4640b3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:21:33.252729 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a890ff47-8a9a-45b0-a98e-094b7d115a8c admin admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:21:33.253124 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a890ff47-8a9a-45b0-a98e-094b7d115a8c admin admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:21:33.253380 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a890ff47-8a9a-45b0-a98e-094b7d115a8c admin admin] PATH_INFO: `/v3/roles` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:21:33.254680 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a890ff47-8a9a-45b0-a98e-094b7d115a8c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:21:33.256342 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a890ff47-8a9a-45b0-a98e-094b7d115a8c admin admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:21:33.268569 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 932/1859] 10.222.0.18 () {58 vars in 1122 bytes} [Tue Sep 13 03:21:33 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 03:21:33.275192 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a90ec538-fd47-4255-a274-732bb57811a8 None None] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:21:33.291084 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a90ec538-fd47-4255-a274-732bb57811a8 None None] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:21:33.296196 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd0a46f7-699d-4588-851f-01afc5238afb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76c5181fc7be45279ac00017bb4640b3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:21:33.296869 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd0a46f7-699d-4588-851f-01afc5238afb admin admin] REQUEST_METHOD: `PUT` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:21:33.297148 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd0a46f7-699d-4588-851f-01afc5238afb admin admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:21:33.297421 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd0a46f7-699d-4588-851f-01afc5238afb admin admin] PATH_INFO: `/v3/projects/7e521370311a419690ac1df1f0d00fd0/users/85be431e56214beca609c447f1664daa/roles/1eb7a5b5203945c29e6d22d653f47847` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:21:33.303094 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a890ff47-8a9a-45b0-a98e-094b7d115a8c admin admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:21:33.305542 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-a890ff47-8a9a-45b0-a98e-094b7d115a8c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:21:33.314888 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd0a46f7-699d-4588-851f-01afc5238afb admin admin] RBAC: Authorizing `identity:create_grant(project_id=7e521370311a419690ac1df1f0d00fd0, user_id=85be431e56214beca609c447f1664daa, role_id=1eb7a5b5203945c29e6d22d653f47847)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:21:33.318197 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd0a46f7-699d-4588-851f-01afc5238afb admin admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:21:33.320307 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a890ff47-8a9a-45b0-a98e-094b7d115a8c admin admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:21:33.325168 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-072bafb1-080d-415e-b3ee-b2c5a8fd30cf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdffbc851086441ebd01022d1e4c2609', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b9d94f6a68f7470d9494f112175620dc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:21:33.325755 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-072bafb1-080d-415e-b3ee-b2c5a8fd30cf service neutron] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:21:33.326033 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-072bafb1-080d-415e-b3ee-b2c5a8fd30cf service neutron] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:21:33.326431 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-072bafb1-080d-415e-b3ee-b2c5a8fd30cf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:21:33.335534 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-072bafb1-080d-415e-b3ee-b2c5a8fd30cf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:21:33.336735 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 928/1860] 10.222.0.18 () {60 vars in 1483 bytes} [Tue Sep 13 03:21:33 2022] PUT /identity/v3/projects/7e521370311a419690ac1df1f0d00fd0/users/85be431e56214beca609c447f1664daa/roles/1eb7a5b5203945c29e6d22d653f47847 => generated 0 bytes in 63 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 03:21:33.337838 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd0a46f7-699d-4588-851f-01afc5238afb admin admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:21:33.337963 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-072bafb1-080d-415e-b3ee-b2c5a8fd30cf service neutron] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:21:33.348575 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-fd0a46f7-699d-4588-851f-01afc5238afb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:21:33.350593 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 933/1861] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Sep 13 03:21:33 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 03:21:33.351447 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-072bafb1-080d-415e-b3ee-b2c5a8fd30cf service neutron] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:21:33.374965 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-072bafb1-080d-415e-b3ee-b2c5a8fd30cf service neutron] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:21:33.423062 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd0a46f7-699d-4588-851f-01afc5238afb admin admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:21:33.431280 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b515d3c-146a-4a50-88d0-82a7440c1c90 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdffbc851086441ebd01022d1e4c2609', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b9d94f6a68f7470d9494f112175620dc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:21:33.432310 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b515d3c-146a-4a50-88d0-82a7440c1c90 service neutron] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:21:33.432310 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b515d3c-146a-4a50-88d0-82a7440c1c90 service neutron] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:21:33.432544 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b515d3c-146a-4a50-88d0-82a7440c1c90 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:21:33.443884 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b515d3c-146a-4a50-88d0-82a7440c1c90 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:21:33.445360 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b515d3c-146a-4a50-88d0-82a7440c1c90 service neutron] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:21:33.466804 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 929/1862] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Sep 13 03:21:33 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 129 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 03:21:33.467725 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b515d3c-146a-4a50-88d0-82a7440c1c90 service neutron] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:21:33.477844 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e46af3b-81a2-433f-8408-a40d88e5259e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdffbc851086441ebd01022d1e4c2609', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b9d94f6a68f7470d9494f112175620dc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:21:33.478463 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e46af3b-81a2-433f-8408-a40d88e5259e service neutron] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:21:33.478768 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e46af3b-81a2-433f-8408-a40d88e5259e service neutron] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:21:33.479032 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e46af3b-81a2-433f-8408-a40d88e5259e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:21:33.487378 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b515d3c-146a-4a50-88d0-82a7440c1c90 service neutron] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:21:33.493976 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e46af3b-81a2-433f-8408-a40d88e5259e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:21:33.495329 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f7cda9a-be10-4959-a4b8-66a806d08c23 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdffbc851086441ebd01022d1e4c2609', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b9d94f6a68f7470d9494f112175620dc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:21:33.496157 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f7cda9a-be10-4959-a4b8-66a806d08c23 service neutron] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:21:33.497207 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e46af3b-81a2-433f-8408-a40d88e5259e service neutron] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:21:33.500304 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f7cda9a-be10-4959-a4b8-66a806d08c23 service neutron] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:21:33.500617 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f7cda9a-be10-4959-a4b8-66a806d08c23 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:21:33.517132 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f7cda9a-be10-4959-a4b8-66a806d08c23 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:21:33.518084 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 934/1863] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Sep 13 03:21:33 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 165 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 03:21:33.521386 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca907b79-70c9-47ef-b46f-37897222c1b3 None None] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:21:33.521666 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca907b79-70c9-47ef-b46f-37897222c1b3 None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:21:33.521931 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca907b79-70c9-47ef-b46f-37897222c1b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:21:33.524390 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f7cda9a-be10-4959-a4b8-66a806d08c23 service neutron] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:21:33.536075 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 930/1864] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Sep 13 03:21:33 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 68 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 03:21:33.559811 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.auth.core [None req-ca907b79-70c9-47ef-b46f-37897222c1b3 None None] MFA Rules not processed for user `85be431e56214beca609c447f1664daa`. Rule list: `[]` (Enabled: `True`). {{(pid=81399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 03:21:33.561122 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-ca907b79-70c9-47ef-b46f-37897222c1b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:21:33.592171 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:21:33.657212 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 935/1865] 10.222.0.18 () {58 vars in 958 bytes} [Tue Sep 13 03:21:33 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 03:21:33.662132 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f7cda9a-be10-4959-a4b8-66a806d08c23 service neutron] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:21:33.688417 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=93,keystone:DELETE=4,keystone:INSERT=3 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:21:33.761748 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f7cda9a-be10-4959-a4b8-66a806d08c23 service neutron] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:21:33.767199 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbf8338c-7d63-4614-982b-4cd432c33859 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76c5181fc7be45279ac00017bb4640b3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:21:33.767984 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf8338c-7d63-4614-982b-4cd432c33859 admin admin] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:21:33.768266 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf8338c-7d63-4614-982b-4cd432c33859 admin admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:21:33.768538 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf8338c-7d63-4614-982b-4cd432c33859 admin admin] PATH_INFO: `/v3/projects` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:21:33.769230 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbf8338c-7d63-4614-982b-4cd432c33859 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:21:33.771330 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbf8338c-7d63-4614-982b-4cd432c33859 admin admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:21:33.811824 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca907b79-70c9-47ef-b46f-37897222c1b3 None None] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:21:33.822187 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 931/1866] 10.222.0.18 () {60 vars in 1153 bytes} [Tue Sep 13 03:21:33 2022] POST /identity/v3/projects => generated 375 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 03:21:33.823585 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca907b79-70c9-47ef-b46f-37897222c1b3 None None] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:21:33.901046 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62d4191c-0d69-4205-9e6c-4707828a4ed0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76c5181fc7be45279ac00017bb4640b3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:21:33.902312 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d4191c-0d69-4205-9e6c-4707828a4ed0 admin admin] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:21:33.903627 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d4191c-0d69-4205-9e6c-4707828a4ed0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:21:33.904038 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d4191c-0d69-4205-9e6c-4707828a4ed0 admin admin] PATH_INFO: `/v3/users` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:21:33.904705 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62d4191c-0d69-4205-9e6c-4707828a4ed0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:21:33.907002 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62d4191c-0d69-4205-9e6c-4707828a4ed0 admin admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:21:33.942954 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbf8338c-7d63-4614-982b-4cd432c33859 admin admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:21:33.944989 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 936/1867] 10.222.0.18 () {60 vars in 1144 bytes} [Tue Sep 13 03:21:33 2022] POST /identity/v3/users => generated 294 bytes in 133 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 03:21:33.954505 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbf8338c-7d63-4614-982b-4cd432c33859 admin admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:21:33.959866 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-313439a3-370d-408f-a7ab-d4b4e92898b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76c5181fc7be45279ac00017bb4640b3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:21:33.960482 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-313439a3-370d-408f-a7ab-d4b4e92898b4 admin admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:21:33.960763 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-313439a3-370d-408f-a7ab-d4b4e92898b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:21:33.961034 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-313439a3-370d-408f-a7ab-d4b4e92898b4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:21:33.961676 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-313439a3-370d-408f-a7ab-d4b4e92898b4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:21:33.963325 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-313439a3-370d-408f-a7ab-d4b4e92898b4 admin admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:21:33.985939 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 932/1868] 10.222.0.18 () {58 vars in 1122 bytes} [Tue Sep 13 03:21:33 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 03:21:33.991239 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62d4191c-0d69-4205-9e6c-4707828a4ed0 admin admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:21:34.006360 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62d4191c-0d69-4205-9e6c-4707828a4ed0 admin admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:21:34.016565 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5082bcc2-0def-4cc5-b0a3-9b2b5f69c50e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76c5181fc7be45279ac00017bb4640b3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:21:34.017258 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5082bcc2-0def-4cc5-b0a3-9b2b5f69c50e admin admin] REQUEST_METHOD: `PUT` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:21:34.017540 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5082bcc2-0def-4cc5-b0a3-9b2b5f69c50e admin admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:21:34.017816 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5082bcc2-0def-4cc5-b0a3-9b2b5f69c50e admin admin] PATH_INFO: `/v3/projects/4c8a591ba7844589be41d82d3290a564/users/77b351bb95724e358cead30c9f565030/roles/425d4b481c7846e0bc07e7e805d5b15f` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:21:34.040194 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5082bcc2-0def-4cc5-b0a3-9b2b5f69c50e admin admin] RBAC: Authorizing `identity:create_grant(project_id=4c8a591ba7844589be41d82d3290a564, user_id=77b351bb95724e358cead30c9f565030, role_id=425d4b481c7846e0bc07e7e805d5b15f)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:21:34.044891 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5082bcc2-0def-4cc5-b0a3-9b2b5f69c50e admin admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:21:34.075807 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 937/1869] 10.222.0.18 () {60 vars in 1483 bytes} [Tue Sep 13 03:21:33 2022] PUT /identity/v3/projects/4c8a591ba7844589be41d82d3290a564/users/77b351bb95724e358cead30c9f565030/roles/425d4b481c7846e0bc07e7e805d5b15f => generated 0 bytes in 85 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 13 03:21:34.079560 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-313439a3-370d-408f-a7ab-d4b4e92898b4 admin admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:21:34.092648 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-313439a3-370d-408f-a7ab-d4b4e92898b4 admin admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:21:34.190719 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0621bb4-4e4c-435f-9144-7d1fb69044f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76c5181fc7be45279ac00017bb4640b3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:21:34.191428 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0621bb4-4e4c-435f-9144-7d1fb69044f4 admin admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:21:34.191896 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0621bb4-4e4c-435f-9144-7d1fb69044f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:21:34.192271 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0621bb4-4e4c-435f-9144-7d1fb69044f4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:21:34.193019 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0621bb4-4e4c-435f-9144-7d1fb69044f4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:21:34.194968 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0621bb4-4e4c-435f-9144-7d1fb69044f4 admin admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:21:34.214172 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 933/1870] 10.222.0.18 () {58 vars in 1122 bytes} [Tue Sep 13 03:21:34 2022] GET /identity/v3/roles => generated 1420 bytes in 133 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 13 03:21:34.215694 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5082bcc2-0def-4cc5-b0a3-9b2b5f69c50e admin admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:21:34.229445 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5082bcc2-0def-4cc5-b0a3-9b2b5f69c50e admin admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:21:34.237910 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91076bd1-6e54-4a90-bbc9-884eeb8160da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76c5181fc7be45279ac00017bb4640b3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:21:34.239902 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91076bd1-6e54-4a90-bbc9-884eeb8160da admin admin] REQUEST_METHOD: `PUT` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:21:34.240730 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91076bd1-6e54-4a90-bbc9-884eeb8160da admin admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:21:34.241123 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91076bd1-6e54-4a90-bbc9-884eeb8160da admin admin] PATH_INFO: `/v3/projects/4c8a591ba7844589be41d82d3290a564/users/77b351bb95724e358cead30c9f565030/roles/425d4b481c7846e0bc07e7e805d5b15f` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:21:34.244004 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91076bd1-6e54-4a90-bbc9-884eeb8160da admin admin] RBAC: Authorizing `identity:create_grant(project_id=4c8a591ba7844589be41d82d3290a564, user_id=77b351bb95724e358cead30c9f565030, role_id=425d4b481c7846e0bc07e7e805d5b15f)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:21:34.247578 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91076bd1-6e54-4a90-bbc9-884eeb8160da admin admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:21:34.272116 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 938/1871] 10.222.0.18 () {60 vars in 1483 bytes} [Tue Sep 13 03:21:34 2022] PUT /identity/v3/projects/4c8a591ba7844589be41d82d3290a564/users/77b351bb95724e358cead30c9f565030/roles/425d4b481c7846e0bc07e7e805d5b15f => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 03:21:34.275836 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5367954-4e0c-44a4-86ca-2d3b7ccd2659 None None] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:21:34.276253 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5367954-4e0c-44a4-86ca-2d3b7ccd2659 None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:21:34.277096 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5367954-4e0c-44a4-86ca-2d3b7ccd2659 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:21:34.326325 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.auth.core [None req-c5367954-4e0c-44a4-86ca-2d3b7ccd2659 None None] MFA Rules not processed for user `77b351bb95724e358cead30c9f565030`. Rule list: `[]` (Enabled: `True`). {{(pid=81400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 03:21:34.329257 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-c5367954-4e0c-44a4-86ca-2d3b7ccd2659 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:21:34.432391 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 934/1872] 10.222.0.18 () {58 vars in 958 bytes} [Tue Sep 13 03:21:34 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 03:21:34.446813 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91076bd1-6e54-4a90-bbc9-884eeb8160da admin admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:21:34.452469 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-91076bd1-6e54-4a90-bbc9-884eeb8160da admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:21:34.643384 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91076bd1-6e54-4a90-bbc9-884eeb8160da admin admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:21:34.653723 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f475e2d-26e3-456d-bb65-9ea6812152df service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '924ee3f608d34a18b0ddc4695f102a56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b9d94f6a68f7470d9494f112175620dc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:21:34.658819 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f475e2d-26e3-456d-bb65-9ea6812152df service cinder] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:21:34.659758 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f475e2d-26e3-456d-bb65-9ea6812152df service cinder] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:21:34.661453 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f475e2d-26e3-456d-bb65-9ea6812152df service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:21:34.733314 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:21:34.812030 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f475e2d-26e3-456d-bb65-9ea6812152df service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:21:34.816370 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f475e2d-26e3-456d-bb65-9ea6812152df service cinder] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:21:34.837591 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 939/1873] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Sep 13 03:21:34 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 391 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 03:21:35.461949 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b68c724-fa12-4609-af80-fbe29f4fde2d None None] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:21:35.462407 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b68c724-fa12-4609-af80-fbe29f4fde2d None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:21:35.462927 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b68c724-fa12-4609-af80-fbe29f4fde2d None None] PATH_INFO: `/` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:21:35.464736 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 935/1874] 10.222.0.18 () {54 vars in 921 bytes} [Tue Sep 13 03:21:35 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 03:21:35.475868 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1856b0df-c64f-4f9e-8502-684a32a43594 None None] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:21:35.476247 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1856b0df-c64f-4f9e-8502-684a32a43594 None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:21:35.476459 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1856b0df-c64f-4f9e-8502-684a32a43594 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:21:35.515790 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.auth.core [None req-1856b0df-c64f-4f9e-8502-684a32a43594 None None] MFA Rules not processed for user `c7377f5bee4a49959182e1bd34bc97a4`. Rule list: `[]` (Enabled: `True`). {{(pid=81399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 03:21:35.518471 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-1856b0df-c64f-4f9e-8502-684a32a43594 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:21:35.677043 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 940/1875] 10.222.0.18 () {58 vars in 1018 bytes} [Tue Sep 13 03:21:35 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 203 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 03:21:35.751134 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b68c724-fa12-4609-af80-fbe29f4fde2d None None] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:21:35.753098 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-0b68c724-fa12-4609-af80-fbe29f4fde2d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:21:35.776911 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b68c724-fa12-4609-af80-fbe29f4fde2d None None] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:21:35.784495 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d182c47d-1a1b-418a-bb7b-be969ce21831 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7377f5bee4a49959182e1bd34bc97a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b9d94f6a68f7470d9494f112175620dc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:21:35.785114 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d182c47d-1a1b-418a-bb7b-be969ce21831 service nova] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:21:35.785934 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d182c47d-1a1b-418a-bb7b-be969ce21831 service nova] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:21:35.786201 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d182c47d-1a1b-418a-bb7b-be969ce21831 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:21:35.797939 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d182c47d-1a1b-418a-bb7b-be969ce21831 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:21:35.800090 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d182c47d-1a1b-418a-bb7b-be969ce21831 service nova] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:21:35.807608 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:21:35.818656 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 936/1876] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Sep 13 03:21:35 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 68 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 03:21:36.083766 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4528085d-5f8c-423e-88c3-5c3718001dc5 None None] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:21:36.084911 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4528085d-5f8c-423e-88c3-5c3718001dc5 None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:21:36.085659 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4528085d-5f8c-423e-88c3-5c3718001dc5 None None] PATH_INFO: `/` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:21:36.087801 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 941/1877] 10.222.0.18 () {54 vars in 921 bytes} [Tue Sep 13 03:21:36 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 03:21:36.115243 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b1c3bb7-56db-4f13-b999-4097158363d4 None None] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:21:36.115243 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b1c3bb7-56db-4f13-b999-4097158363d4 None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:21:36.115574 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b1c3bb7-56db-4f13-b999-4097158363d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:21:36.139566 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.auth.core [None req-6b1c3bb7-56db-4f13-b999-4097158363d4 None None] MFA Rules not processed for user `c7377f5bee4a49959182e1bd34bc97a4`. Rule list: `[]` (Enabled: `True`). {{(pid=81400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 03:21:36.141042 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-6b1c3bb7-56db-4f13-b999-4097158363d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:21:36.155155 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 937/1878] 10.222.0.18 () {58 vars in 1018 bytes} [Tue Sep 13 03:21:36 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 03:21:36.306355 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4528085d-5f8c-423e-88c3-5c3718001dc5 None None] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:21:36.309038 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-4528085d-5f8c-423e-88c3-5c3718001dc5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:21:36.333328 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4528085d-5f8c-423e-88c3-5c3718001dc5 None None] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:21:36.359176 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-683c6310-405b-4443-83bd-51228acf20d9 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7377f5bee4a49959182e1bd34bc97a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b9d94f6a68f7470d9494f112175620dc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:21:36.360598 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-683c6310-405b-4443-83bd-51228acf20d9 service nova] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:21:36.362661 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-683c6310-405b-4443-83bd-51228acf20d9 service nova] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:21:36.363899 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-683c6310-405b-4443-83bd-51228acf20d9 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:21:36.381271 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-683c6310-405b-4443-83bd-51228acf20d9 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:21:36.384169 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-683c6310-405b-4443-83bd-51228acf20d9 service nova] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:21:36.405885 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 942/1879] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Sep 13 03:21:36 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 03:21:37.814931 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b1c3bb7-56db-4f13-b999-4097158363d4 None None] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:21:37.916385 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b1c3bb7-56db-4f13-b999-4097158363d4 None None] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:21:37.922372 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ce31639-6d05-4c08-a8c1-ab81931a95ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76c5181fc7be45279ac00017bb4640b3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:21:37.923070 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce31639-6d05-4c08-a8c1-ab81931a95ea admin admin] REQUEST_METHOD: `DELETE` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:21:37.923350 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce31639-6d05-4c08-a8c1-ab81931a95ea admin admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:21:37.923623 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce31639-6d05-4c08-a8c1-ab81931a95ea admin admin] PATH_INFO: `/v3/users/85be431e56214beca609c447f1664daa` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:21:37.924833 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ce31639-6d05-4c08-a8c1-ab81931a95ea admin admin] RBAC: Authorizing `identity:delete_user(user_id=85be431e56214beca609c447f1664daa)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:21:37.927061 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ce31639-6d05-4c08-a8c1-ab81931a95ea admin admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:21:37.999597 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-8ce31639-6d05-4c08-a8c1-ab81931a95ea admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '85be431e56214beca609c447f1664daa'} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:21:38.010268 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-8ce31639-6d05-4c08-a8c1-ab81931a95ea admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '85be431e56214beca609c447f1664daa'} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:21:38.011314 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-8ce31639-6d05-4c08-a8c1-ab81931a95ea admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '85be431e56214beca609c447f1664daa'} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:21:38.028652 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-8ce31639-6d05-4c08-a8c1-ab81931a95ea admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '85be431e56214beca609c447f1664daa'} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:21:38.036812 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-8ce31639-6d05-4c08-a8c1-ab81931a95ea admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '85be431e56214beca609c447f1664daa'} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:21:38.047107 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 938/1880] 10.222.0.18 () {58 vars in 1224 bytes} [Tue Sep 13 03:21:37 2022] DELETE /identity/v3/users/85be431e56214beca609c447f1664daa => generated 0 bytes in 230 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 03:21:38.049536 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d323be7-66f9-429f-a7e1-f419a0a9efd2 None None] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:21:38.050006 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d323be7-66f9-429f-a7e1-f419a0a9efd2 None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:21:38.050108 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d323be7-66f9-429f-a7e1-f419a0a9efd2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:21:38.091863 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.auth.core [None req-0d323be7-66f9-429f-a7e1-f419a0a9efd2 None None] MFA Rules not processed for user `49f280ab086b4610855c4ffa378517d9`. Rule list: `[]` (Enabled: `True`). {{(pid=81399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 03:21:38.093267 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-0d323be7-66f9-429f-a7e1-f419a0a9efd2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:21:38.198146 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 943/1881] 10.222.0.18 () {58 vars in 958 bytes} [Tue Sep 13 03:21:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 03:21:38.229008 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ce31639-6d05-4c08-a8c1-ab81931a95ea admin admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:21:38.234996 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-8ce31639-6d05-4c08-a8c1-ab81931a95ea admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:21:38.387799 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ce31639-6d05-4c08-a8c1-ab81931a95ea admin admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:21:38.392795 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8994d7b-36ce-4b53-b859-37c58529de53 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdffbc851086441ebd01022d1e4c2609', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b9d94f6a68f7470d9494f112175620dc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:21:38.394460 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8994d7b-36ce-4b53-b859-37c58529de53 service neutron] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:21:38.394812 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8994d7b-36ce-4b53-b859-37c58529de53 service neutron] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:21:38.395082 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8994d7b-36ce-4b53-b859-37c58529de53 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:21:38.408461 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8994d7b-36ce-4b53-b859-37c58529de53 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:21:38.411358 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8994d7b-36ce-4b53-b859-37c58529de53 service neutron] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:21:38.425164 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 939/1882] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Sep 13 03:21:38 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 197 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 03:21:38.961145 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d323be7-66f9-429f-a7e1-f419a0a9efd2 None None] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:21:38.962652 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-0d323be7-66f9-429f-a7e1-f419a0a9efd2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:21:38.976848 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d323be7-66f9-429f-a7e1-f419a0a9efd2 None None] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:21:38.981521 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dc5a802-50da-480b-b785-8fdfef6f44c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76c5181fc7be45279ac00017bb4640b3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:21:38.982169 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dc5a802-50da-480b-b785-8fdfef6f44c0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:21:38.983059 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dc5a802-50da-480b-b785-8fdfef6f44c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:21:38.983332 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dc5a802-50da-480b-b785-8fdfef6f44c0 admin admin] PATH_INFO: `/v3/projects/7e521370311a419690ac1df1f0d00fd0` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:21:38.984674 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dc5a802-50da-480b-b785-8fdfef6f44c0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7e521370311a419690ac1df1f0d00fd0)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:21:38.986561 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dc5a802-50da-480b-b785-8fdfef6f44c0 admin admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:21:39.029730 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:21:39.033658 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-1dc5a802-50da-480b-b785-8fdfef6f44c0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7e521370311a419690ac1df1f0d00fd0', 'request_id': 'req-1dc5a802-50da-480b-b785-8fdfef6f44c0'} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:21:39.040378 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-1dc5a802-50da-480b-b785-8fdfef6f44c0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7e521370311a419690ac1df1f0d00fd0', 'request_id': 'req-1dc5a802-50da-480b-b785-8fdfef6f44c0'} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:21:39.047084 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-1dc5a802-50da-480b-b785-8fdfef6f44c0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7e521370311a419690ac1df1f0d00fd0', 'request_id': 'req-1dc5a802-50da-480b-b785-8fdfef6f44c0'} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:21:39.059248 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-1dc5a802-50da-480b-b785-8fdfef6f44c0 admin admin] The token cache is being invalidate because project 7e521370311a419690ac1df1f0d00fd0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81399) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 03:21:39.059875 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-1dc5a802-50da-480b-b785-8fdfef6f44c0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:21:39.060140 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-1dc5a802-50da-480b-b785-8fdfef6f44c0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:21:39.061636 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 944/1883] 10.222.0.18 () {58 vars in 1233 bytes} [Tue Sep 13 03:21:38 2022] DELETE /identity/v3/projects/7e521370311a419690ac1df1f0d00fd0 => generated 0 bytes in 101 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 13 03:21:39.065629 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8994d7b-36ce-4b53-b859-37c58529de53 service neutron] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:21:39.075470 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8994d7b-36ce-4b53-b859-37c58529de53 service neutron] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:21:39.084487 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:21:39.132984 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e42391b-9134-4093-8699-8b00d416580b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76c5181fc7be45279ac00017bb4640b3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:21:39.133598 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e42391b-9134-4093-8699-8b00d416580b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:21:39.133870 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e42391b-9134-4093-8699-8b00d416580b admin admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:21:39.134134 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e42391b-9134-4093-8699-8b00d416580b admin admin] PATH_INFO: `/v3/users/77b351bb95724e358cead30c9f565030` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:21:39.135250 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e42391b-9134-4093-8699-8b00d416580b admin admin] RBAC: Authorizing `identity:delete_user(user_id=77b351bb95724e358cead30c9f565030)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:21:39.137143 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e42391b-9134-4093-8699-8b00d416580b admin admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:21:39.184748 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-6e42391b-9134-4093-8699-8b00d416580b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '77b351bb95724e358cead30c9f565030'} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:21:39.192074 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-6e42391b-9134-4093-8699-8b00d416580b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '77b351bb95724e358cead30c9f565030'} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:21:39.192702 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-6e42391b-9134-4093-8699-8b00d416580b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '77b351bb95724e358cead30c9f565030'} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:21:39.205323 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-6e42391b-9134-4093-8699-8b00d416580b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '77b351bb95724e358cead30c9f565030'} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:21:39.212735 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-6e42391b-9134-4093-8699-8b00d416580b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '77b351bb95724e358cead30c9f565030'} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:21:39.218967 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 940/1884] 10.222.0.18 () {58 vars in 1224 bytes} [Tue Sep 13 03:21:39 2022] DELETE /identity/v3/users/77b351bb95724e358cead30c9f565030 => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 13 03:21:39.580042 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dc5a802-50da-480b-b785-8fdfef6f44c0 admin admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:21:39.581438 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-1dc5a802-50da-480b-b785-8fdfef6f44c0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:21:39.596508 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dc5a802-50da-480b-b785-8fdfef6f44c0 admin admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:21:39.665221 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca6bb603-170a-492d-9698-9c596959183b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76c5181fc7be45279ac00017bb4640b3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:21:39.665854 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca6bb603-170a-492d-9698-9c596959183b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:21:39.666135 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca6bb603-170a-492d-9698-9c596959183b admin admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:21:39.666402 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca6bb603-170a-492d-9698-9c596959183b admin admin] PATH_INFO: `/v3/projects/4c8a591ba7844589be41d82d3290a564` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:21:39.667557 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca6bb603-170a-492d-9698-9c596959183b admin admin] RBAC: Authorizing `identity:delete_project(project_id=4c8a591ba7844589be41d82d3290a564)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:21:39.669453 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca6bb603-170a-492d-9698-9c596959183b admin admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:21:39.717077 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-ca6bb603-170a-492d-9698-9c596959183b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4c8a591ba7844589be41d82d3290a564', 'request_id': 'req-ca6bb603-170a-492d-9698-9c596959183b'} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:21:39.722887 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-ca6bb603-170a-492d-9698-9c596959183b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4c8a591ba7844589be41d82d3290a564', 'request_id': 'req-ca6bb603-170a-492d-9698-9c596959183b'} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:21:39.729129 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-ca6bb603-170a-492d-9698-9c596959183b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4c8a591ba7844589be41d82d3290a564', 'request_id': 'req-ca6bb603-170a-492d-9698-9c596959183b'} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:21:39.741994 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-ca6bb603-170a-492d-9698-9c596959183b admin admin] The token cache is being invalidate because project 4c8a591ba7844589be41d82d3290a564 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81399) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 03:21:39.742261 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-ca6bb603-170a-492d-9698-9c596959183b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:21:39.742922 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-ca6bb603-170a-492d-9698-9c596959183b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:21:39.744507 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 945/1885] 10.222.0.18 () {58 vars in 1233 bytes} [Tue Sep 13 03:21:39 2022] DELETE /identity/v3/projects/4c8a591ba7844589be41d82d3290a564 => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 03:21:39.749207 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b358f29e-a4e4-42db-93ae-6095fe5d366d None None] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:21:39.749494 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b358f29e-a4e4-42db-93ae-6095fe5d366d None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:21:39.749768 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b358f29e-a4e4-42db-93ae-6095fe5d366d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:21:39.769662 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.auth.core [None req-b358f29e-a4e4-42db-93ae-6095fe5d366d None None] MFA Rules not processed for user `49f280ab086b4610855c4ffa378517d9`. Rule list: `[]` (Enabled: `True`). {{(pid=81400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 03:21:39.770895 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-b358f29e-a4e4-42db-93ae-6095fe5d366d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:21:39.841875 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 941/1886] 10.222.0.18 () {58 vars in 958 bytes} [Tue Sep 13 03:21:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 03:21:39.849592 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-917fef46-389f-4dae-bf6d-d32b8d43eb7b None None] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:21:39.849883 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-917fef46-389f-4dae-bf6d-d32b8d43eb7b None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:21:39.850187 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-917fef46-389f-4dae-bf6d-d32b8d43eb7b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:21:39.871113 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.auth.core [None req-917fef46-389f-4dae-bf6d-d32b8d43eb7b None None] MFA Rules not processed for user `49f280ab086b4610855c4ffa378517d9`. Rule list: `[]` (Enabled: `True`). {{(pid=81399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 03:21:39.872329 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-917fef46-389f-4dae-bf6d-d32b8d43eb7b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:21:39.882236 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 946/1887] 10.222.0.18 () {58 vars in 958 bytes} [Tue Sep 13 03:21:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 03:21:39.886551 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b358f29e-a4e4-42db-93ae-6095fe5d366d None None] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:21:39.900924 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b358f29e-a4e4-42db-93ae-6095fe5d366d None None] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:21:39.905951 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb6230bc-cc7c-4a62-9740-95e3e2ab65d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76c5181fc7be45279ac00017bb4640b3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:21:39.906559 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb6230bc-cc7c-4a62-9740-95e3e2ab65d8 admin admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:21:39.906863 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb6230bc-cc7c-4a62-9740-95e3e2ab65d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:21:39.907130 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb6230bc-cc7c-4a62-9740-95e3e2ab65d8 admin admin] PATH_INFO: `/v3/domains` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:21:39.907768 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb6230bc-cc7c-4a62-9740-95e3e2ab65d8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 03:21:39.908064 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb6230bc-cc7c-4a62-9740-95e3e2ab65d8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:21:39.909682 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb6230bc-cc7c-4a62-9740-95e3e2ab65d8 admin admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:21:39.922740 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 942/1888] 10.222.0.18 () {58 vars in 1153 bytes} [Tue Sep 13 03:21:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 03:21:39.927431 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-917fef46-389f-4dae-bf6d-d32b8d43eb7b None None] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:21:39.937420 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-917fef46-389f-4dae-bf6d-d32b8d43eb7b None None] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:21:39.943186 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ac097e1-15de-43bc-a0de-299b6e9206e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76c5181fc7be45279ac00017bb4640b3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:21:39.943798 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac097e1-15de-43bc-a0de-299b6e9206e6 admin admin] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:21:39.944109 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac097e1-15de-43bc-a0de-299b6e9206e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:21:39.944377 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac097e1-15de-43bc-a0de-299b6e9206e6 admin admin] PATH_INFO: `/v3/projects` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:21:39.945023 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ac097e1-15de-43bc-a0de-299b6e9206e6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:21:39.946977 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ac097e1-15de-43bc-a0de-299b6e9206e6 admin admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:21:39.983926 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 947/1889] 10.222.0.18 () {60 vars in 1153 bytes} [Tue Sep 13 03:21:39 2022] POST /identity/v3/projects => generated 375 bytes in 57 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 13 03:21:39.987961 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb6230bc-cc7c-4a62-9740-95e3e2ab65d8 admin admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:21:39.998562 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb6230bc-cc7c-4a62-9740-95e3e2ab65d8 admin admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:21:40.015053 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:21:40.063238 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f20a440-d1f7-406f-813e-1f6900c03f67 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76c5181fc7be45279ac00017bb4640b3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:21:40.063845 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f20a440-d1f7-406f-813e-1f6900c03f67 admin admin] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:21:40.064117 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f20a440-d1f7-406f-813e-1f6900c03f67 admin admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:21:40.064382 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f20a440-d1f7-406f-813e-1f6900c03f67 admin admin] PATH_INFO: `/v3/users` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:21:40.065019 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f20a440-d1f7-406f-813e-1f6900c03f67 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:21:40.067021 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f20a440-d1f7-406f-813e-1f6900c03f67 admin admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:21:40.088431 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 943/1890] 10.222.0.18 () {60 vars in 1144 bytes} [Tue Sep 13 03:21:39 2022] POST /identity/v3/users => generated 288 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 13 03:21:40.092523 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ac097e1-15de-43bc-a0de-299b6e9206e6 admin admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:21:40.103094 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ac097e1-15de-43bc-a0de-299b6e9206e6 admin admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:21:40.107622 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3212a2c-1906-40e4-99e2-9773b2fc92c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76c5181fc7be45279ac00017bb4640b3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:21:40.108224 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3212a2c-1906-40e4-99e2-9773b2fc92c1 admin admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:21:40.108489 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3212a2c-1906-40e4-99e2-9773b2fc92c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:21:40.108761 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3212a2c-1906-40e4-99e2-9773b2fc92c1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:21:40.109395 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3212a2c-1906-40e4-99e2-9773b2fc92c1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:21:40.111091 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3212a2c-1906-40e4-99e2-9773b2fc92c1 admin admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:21:40.121570 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 948/1891] 10.222.0.18 () {58 vars in 1122 bytes} [Tue Sep 13 03:21:40 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 03:21:40.125334 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f20a440-d1f7-406f-813e-1f6900c03f67 admin admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:21:40.135273 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f20a440-d1f7-406f-813e-1f6900c03f67 admin admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:21:40.139897 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-473681d1-e136-4d3a-ac4a-c4a89af77d5f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76c5181fc7be45279ac00017bb4640b3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:21:40.140483 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-473681d1-e136-4d3a-ac4a-c4a89af77d5f admin admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:21:40.140757 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-473681d1-e136-4d3a-ac4a-c4a89af77d5f admin admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:21:40.141055 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-473681d1-e136-4d3a-ac4a-c4a89af77d5f admin admin] PATH_INFO: `/v3/roles` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:21:40.141685 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-473681d1-e136-4d3a-ac4a-c4a89af77d5f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:21:40.143373 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-473681d1-e136-4d3a-ac4a-c4a89af77d5f admin admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:21:40.153786 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 944/1892] 10.222.0.18 () {58 vars in 1122 bytes} [Tue Sep 13 03:21:40 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 03:21:40.157690 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3212a2c-1906-40e4-99e2-9773b2fc92c1 admin admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:21:40.168452 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3212a2c-1906-40e4-99e2-9773b2fc92c1 admin admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:21:40.173209 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8fdc51d-a08f-4215-b79c-4c40418aed67 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76c5181fc7be45279ac00017bb4640b3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:21:40.173845 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8fdc51d-a08f-4215-b79c-4c40418aed67 admin admin] REQUEST_METHOD: `PUT` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:21:40.174124 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8fdc51d-a08f-4215-b79c-4c40418aed67 admin admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:21:40.174393 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8fdc51d-a08f-4215-b79c-4c40418aed67 admin admin] PATH_INFO: `/v3/projects/e5caec8137784904a0ef679b2f01729f/users/cbde25e814c34f43a2135059a079c33f/roles/1eb7a5b5203945c29e6d22d653f47847` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:21:40.189028 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8fdc51d-a08f-4215-b79c-4c40418aed67 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e5caec8137784904a0ef679b2f01729f, user_id=cbde25e814c34f43a2135059a079c33f, role_id=1eb7a5b5203945c29e6d22d653f47847)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:21:40.191694 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8fdc51d-a08f-4215-b79c-4c40418aed67 admin admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:21:40.199022 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=85,keystone:DELETE=11,keystone:INSERT=4 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:21:40.205262 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 949/1893] 10.222.0.18 () {60 vars in 1483 bytes} [Tue Sep 13 03:21:40 2022] PUT /identity/v3/projects/e5caec8137784904a0ef679b2f01729f/users/cbde25e814c34f43a2135059a079c33f/roles/1eb7a5b5203945c29e6d22d653f47847 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 03:21:40.216337 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72fa4e2b-1ee8-4bda-9cd1-49f43816c5f9 None None] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:21:40.216643 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72fa4e2b-1ee8-4bda-9cd1-49f43816c5f9 None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:21:40.216912 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72fa4e2b-1ee8-4bda-9cd1-49f43816c5f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:21:40.246169 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.auth.core [None req-72fa4e2b-1ee8-4bda-9cd1-49f43816c5f9 None None] MFA Rules not processed for user `cbde25e814c34f43a2135059a079c33f`. Rule list: `[]` (Enabled: `True`). {{(pid=81400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 03:21:40.247398 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-72fa4e2b-1ee8-4bda-9cd1-49f43816c5f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:21:40.309720 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 945/1894] 10.222.0.18 () {58 vars in 958 bytes} [Tue Sep 13 03:21:40 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 03:21:40.314237 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8fdc51d-a08f-4215-b79c-4c40418aed67 admin admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:21:40.375742 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8fdc51d-a08f-4215-b79c-4c40418aed67 admin admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:21:40.380354 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44fe0e8e-3679-41bd-a810-a2a04bdef99b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76c5181fc7be45279ac00017bb4640b3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:21:40.380954 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44fe0e8e-3679-41bd-a810-a2a04bdef99b admin admin] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:21:40.381235 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44fe0e8e-3679-41bd-a810-a2a04bdef99b admin admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:21:40.381507 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44fe0e8e-3679-41bd-a810-a2a04bdef99b admin admin] PATH_INFO: `/v3/projects` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:21:40.382138 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44fe0e8e-3679-41bd-a810-a2a04bdef99b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:21:40.384102 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44fe0e8e-3679-41bd-a810-a2a04bdef99b admin admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:21:40.411950 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 950/1895] 10.222.0.18 () {60 vars in 1153 bytes} [Tue Sep 13 03:21:40 2022] POST /identity/v3/projects => generated 375 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 03:21:40.416107 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72fa4e2b-1ee8-4bda-9cd1-49f43816c5f9 None None] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:21:40.427926 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72fa4e2b-1ee8-4bda-9cd1-49f43816c5f9 None None] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:21:40.485881 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3baf124-cf16-4e94-bf98-a40aafd2f577 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76c5181fc7be45279ac00017bb4640b3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:21:40.486498 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3baf124-cf16-4e94-bf98-a40aafd2f577 admin admin] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:21:40.486810 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3baf124-cf16-4e94-bf98-a40aafd2f577 admin admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:21:40.487082 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3baf124-cf16-4e94-bf98-a40aafd2f577 admin admin] PATH_INFO: `/v3/users` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:21:40.487719 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3baf124-cf16-4e94-bf98-a40aafd2f577 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:21:40.489676 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3baf124-cf16-4e94-bf98-a40aafd2f577 admin admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:21:40.512972 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 946/1896] 10.222.0.18 () {60 vars in 1144 bytes} [Tue Sep 13 03:21:40 2022] POST /identity/v3/users => generated 288 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 13 03:21:40.516975 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44fe0e8e-3679-41bd-a810-a2a04bdef99b admin admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:21:40.527935 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44fe0e8e-3679-41bd-a810-a2a04bdef99b admin admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:21:40.532809 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-524b24fa-67fc-469a-9ea1-937c272635d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76c5181fc7be45279ac00017bb4640b3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:21:40.533416 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-524b24fa-67fc-469a-9ea1-937c272635d1 admin admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:21:40.533694 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-524b24fa-67fc-469a-9ea1-937c272635d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:21:40.533965 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-524b24fa-67fc-469a-9ea1-937c272635d1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:21:40.535227 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-524b24fa-67fc-469a-9ea1-937c272635d1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:21:40.537087 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-524b24fa-67fc-469a-9ea1-937c272635d1 admin admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:21:40.553203 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 951/1897] 10.222.0.18 () {58 vars in 1122 bytes} [Tue Sep 13 03:21:40 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 13 03:21:40.554586 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3baf124-cf16-4e94-bf98-a40aafd2f577 admin admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:21:40.565377 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3baf124-cf16-4e94-bf98-a40aafd2f577 admin admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:21:40.570658 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccf2849d-950d-4788-bc90-3f8a024c3c02 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76c5181fc7be45279ac00017bb4640b3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:21:40.570961 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccf2849d-950d-4788-bc90-3f8a024c3c02 admin admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:21:40.571237 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccf2849d-950d-4788-bc90-3f8a024c3c02 admin admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:21:40.571508 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccf2849d-950d-4788-bc90-3f8a024c3c02 admin admin] PATH_INFO: `/v3/roles` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:21:40.572148 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccf2849d-950d-4788-bc90-3f8a024c3c02 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:21:40.573771 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccf2849d-950d-4788-bc90-3f8a024c3c02 admin admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:21:40.589394 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 947/1898] 10.222.0.18 () {58 vars in 1122 bytes} [Tue Sep 13 03:21:40 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 03:21:40.593651 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-524b24fa-67fc-469a-9ea1-937c272635d1 admin admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:21:40.604853 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-524b24fa-67fc-469a-9ea1-937c272635d1 admin admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:21:40.609896 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28867cf4-497d-48d2-b73a-ee780fac6132 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76c5181fc7be45279ac00017bb4640b3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:21:40.610556 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28867cf4-497d-48d2-b73a-ee780fac6132 admin admin] REQUEST_METHOD: `PUT` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:21:40.610951 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28867cf4-497d-48d2-b73a-ee780fac6132 admin admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:21:40.611220 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28867cf4-497d-48d2-b73a-ee780fac6132 admin admin] PATH_INFO: `/v3/projects/a2f8da1d82034dd69e09f8ae8c48958d/users/ba68adca11f541c685d476e6d181fc4a/roles/1eb7a5b5203945c29e6d22d653f47847` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:21:40.630869 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28867cf4-497d-48d2-b73a-ee780fac6132 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a2f8da1d82034dd69e09f8ae8c48958d, user_id=ba68adca11f541c685d476e6d181fc4a, role_id=1eb7a5b5203945c29e6d22d653f47847)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:21:40.633822 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28867cf4-497d-48d2-b73a-ee780fac6132 admin admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:21:40.651532 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 952/1899] 10.222.0.18 () {60 vars in 1483 bytes} [Tue Sep 13 03:21:40 2022] PUT /identity/v3/projects/a2f8da1d82034dd69e09f8ae8c48958d/users/ba68adca11f541c685d476e6d181fc4a/roles/1eb7a5b5203945c29e6d22d653f47847 => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 13 03:21:40.662245 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54df14f5-2755-4451-af8c-006f5d6dc558 None None] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:21:40.662527 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54df14f5-2755-4451-af8c-006f5d6dc558 None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:21:40.662828 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54df14f5-2755-4451-af8c-006f5d6dc558 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:21:40.677173 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:21:40.693811 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.auth.core [None req-54df14f5-2755-4451-af8c-006f5d6dc558 None None] MFA Rules not processed for user `ba68adca11f541c685d476e6d181fc4a`. Rule list: `[]` (Enabled: `True`). {{(pid=81400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 03:21:40.695048 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-54df14f5-2755-4451-af8c-006f5d6dc558 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:21:40.761552 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 948/1900] 10.222.0.18 () {58 vars in 958 bytes} [Tue Sep 13 03:21:40 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 03:21:40.766028 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28867cf4-497d-48d2-b73a-ee780fac6132 admin admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:21:40.829496 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28867cf4-497d-48d2-b73a-ee780fac6132 admin admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:21:40.834833 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-005a4c9f-31f2-4a12-9b52-8de696c00ae7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76c5181fc7be45279ac00017bb4640b3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:21:40.835438 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-005a4c9f-31f2-4a12-9b52-8de696c00ae7 admin admin] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:21:40.835715 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-005a4c9f-31f2-4a12-9b52-8de696c00ae7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:21:40.835985 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-005a4c9f-31f2-4a12-9b52-8de696c00ae7 admin admin] PATH_INFO: `/v3/projects` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:21:40.836625 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-005a4c9f-31f2-4a12-9b52-8de696c00ae7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:21:40.838529 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-005a4c9f-31f2-4a12-9b52-8de696c00ae7 admin admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:21:40.869464 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 953/1901] 10.222.0.18 () {60 vars in 1153 bytes} [Tue Sep 13 03:21:40 2022] POST /identity/v3/projects => generated 375 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 03:21:40.870879 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54df14f5-2755-4451-af8c-006f5d6dc558 None None] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:21:40.882371 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54df14f5-2755-4451-af8c-006f5d6dc558 None None] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:21:40.937918 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e176265-4131-4c6e-8b9a-29c4d2227aed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76c5181fc7be45279ac00017bb4640b3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:21:40.938529 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e176265-4131-4c6e-8b9a-29c4d2227aed admin admin] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:21:40.938837 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e176265-4131-4c6e-8b9a-29c4d2227aed admin admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:21:40.939108 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e176265-4131-4c6e-8b9a-29c4d2227aed admin admin] PATH_INFO: `/v3/users` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:21:40.939732 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e176265-4131-4c6e-8b9a-29c4d2227aed admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:21:40.942169 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e176265-4131-4c6e-8b9a-29c4d2227aed admin admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:21:40.963701 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 949/1902] 10.222.0.18 () {60 vars in 1144 bytes} [Tue Sep 13 03:21:40 2022] POST /identity/v3/users => generated 294 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 03:21:40.967718 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-005a4c9f-31f2-4a12-9b52-8de696c00ae7 admin admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:21:40.978221 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-005a4c9f-31f2-4a12-9b52-8de696c00ae7 admin admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:21:40.982944 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92a65238-90a1-420b-b333-006984922b22 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76c5181fc7be45279ac00017bb4640b3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:21:40.984034 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a65238-90a1-420b-b333-006984922b22 admin admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:21:40.984433 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a65238-90a1-420b-b333-006984922b22 admin admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:21:40.984876 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a65238-90a1-420b-b333-006984922b22 admin admin] PATH_INFO: `/v3/roles` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:21:40.988192 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92a65238-90a1-420b-b333-006984922b22 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:21:40.988192 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92a65238-90a1-420b-b333-006984922b22 admin admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:21:40.999523 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 954/1903] 10.222.0.18 () {58 vars in 1122 bytes} [Tue Sep 13 03:21:40 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 13 03:21:41.003541 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e176265-4131-4c6e-8b9a-29c4d2227aed admin admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:21:41.014897 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e176265-4131-4c6e-8b9a-29c4d2227aed admin admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:21:41.020681 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d89218b2-69a2-4dd9-b0a8-35606e661995 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76c5181fc7be45279ac00017bb4640b3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:21:41.021585 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d89218b2-69a2-4dd9-b0a8-35606e661995 admin admin] REQUEST_METHOD: `PUT` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:21:41.021962 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d89218b2-69a2-4dd9-b0a8-35606e661995 admin admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:21:41.022327 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d89218b2-69a2-4dd9-b0a8-35606e661995 admin admin] PATH_INFO: `/v3/projects/16268c8e0db548089f1d7453d8d3dc0c/users/655c24322f564615b084db8d2d8c4327/roles/425d4b481c7846e0bc07e7e805d5b15f` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:21:41.045496 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d89218b2-69a2-4dd9-b0a8-35606e661995 admin admin] RBAC: Authorizing `identity:create_grant(project_id=16268c8e0db548089f1d7453d8d3dc0c, user_id=655c24322f564615b084db8d2d8c4327, role_id=425d4b481c7846e0bc07e7e805d5b15f)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:21:41.048380 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d89218b2-69a2-4dd9-b0a8-35606e661995 admin admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:21:41.071305 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 950/1904] 10.222.0.18 () {60 vars in 1483 bytes} [Tue Sep 13 03:21:41 2022] PUT /identity/v3/projects/16268c8e0db548089f1d7453d8d3dc0c/users/655c24322f564615b084db8d2d8c4327/roles/425d4b481c7846e0bc07e7e805d5b15f => generated 0 bytes in 68 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 13 03:21:41.075248 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92a65238-90a1-420b-b333-006984922b22 admin admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:21:41.097715 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92a65238-90a1-420b-b333-006984922b22 admin admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:21:41.125740 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:21:41.166053 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07eb1078-1c7e-4108-b886-cb0367f05d43 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76c5181fc7be45279ac00017bb4640b3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:21:41.166898 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07eb1078-1c7e-4108-b886-cb0367f05d43 admin admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:21:41.167166 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07eb1078-1c7e-4108-b886-cb0367f05d43 admin admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:21:41.167486 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07eb1078-1c7e-4108-b886-cb0367f05d43 admin admin] PATH_INFO: `/v3/roles` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:21:41.168124 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07eb1078-1c7e-4108-b886-cb0367f05d43 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:21:41.169984 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07eb1078-1c7e-4108-b886-cb0367f05d43 admin admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:21:41.180243 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 955/1905] 10.222.0.18 () {58 vars in 1122 bytes} [Tue Sep 13 03:21:41 2022] GET /identity/v3/roles => generated 1420 bytes in 106 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 03:21:41.184424 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d89218b2-69a2-4dd9-b0a8-35606e661995 admin admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:21:41.195246 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d89218b2-69a2-4dd9-b0a8-35606e661995 admin admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:21:41.199744 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-460d67af-e6b1-4ea4-9b70-fdd4ec938616 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76c5181fc7be45279ac00017bb4640b3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:21:41.200387 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-460d67af-e6b1-4ea4-9b70-fdd4ec938616 admin admin] REQUEST_METHOD: `PUT` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:21:41.200663 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-460d67af-e6b1-4ea4-9b70-fdd4ec938616 admin admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:21:41.200932 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-460d67af-e6b1-4ea4-9b70-fdd4ec938616 admin admin] PATH_INFO: `/v3/projects/16268c8e0db548089f1d7453d8d3dc0c/users/655c24322f564615b084db8d2d8c4327/roles/425d4b481c7846e0bc07e7e805d5b15f` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:21:41.202841 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-460d67af-e6b1-4ea4-9b70-fdd4ec938616 admin admin] RBAC: Authorizing `identity:create_grant(project_id=16268c8e0db548089f1d7453d8d3dc0c, user_id=655c24322f564615b084db8d2d8c4327, role_id=425d4b481c7846e0bc07e7e805d5b15f)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:21:41.205523 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-460d67af-e6b1-4ea4-9b70-fdd4ec938616 admin admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:21:41.215558 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 951/1906] 10.222.0.18 () {60 vars in 1483 bytes} [Tue Sep 13 03:21:41 2022] PUT /identity/v3/projects/16268c8e0db548089f1d7453d8d3dc0c/users/655c24322f564615b084db8d2d8c4327/roles/425d4b481c7846e0bc07e7e805d5b15f => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 03:21:41.225349 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db6e507-874b-4c65-8237-a6e43c5161f8 None None] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:21:41.225641 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db6e507-874b-4c65-8237-a6e43c5161f8 None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:21:41.225940 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db6e507-874b-4c65-8237-a6e43c5161f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:21:41.253200 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.auth.core [None req-9db6e507-874b-4c65-8237-a6e43c5161f8 None None] MFA Rules not processed for user `655c24322f564615b084db8d2d8c4327`. Rule list: `[]` (Enabled: `True`). {{(pid=81399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 03:21:41.254968 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-9db6e507-874b-4c65-8237-a6e43c5161f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:21:41.321284 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 956/1907] 10.222.0.18 () {58 vars in 958 bytes} [Tue Sep 13 03:21:41 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 13 03:21:41.333590 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-460d67af-e6b1-4ea4-9b70-fdd4ec938616 admin admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:21:41.335315 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-460d67af-e6b1-4ea4-9b70-fdd4ec938616 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:21:41.388126 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:21:41.392529 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-460d67af-e6b1-4ea4-9b70-fdd4ec938616 admin admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:21:41.396980 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90e389fc-e462-4544-81a0-a94b876a5742 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '924ee3f608d34a18b0ddc4695f102a56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b9d94f6a68f7470d9494f112175620dc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:21:41.397563 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e389fc-e462-4544-81a0-a94b876a5742 service cinder] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:21:41.397840 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e389fc-e462-4544-81a0-a94b876a5742 service cinder] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:21:41.398111 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e389fc-e462-4544-81a0-a94b876a5742 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:21:41.454245 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90e389fc-e462-4544-81a0-a94b876a5742 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:21:41.455700 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90e389fc-e462-4544-81a0-a94b876a5742 service cinder] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:21:41.464402 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 952/1908] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Sep 13 03:21:41 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 131 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 03:21:42.997305 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9db6e507-874b-4c65-8237-a6e43c5161f8 None None] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:21:43.008358 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9db6e507-874b-4c65-8237-a6e43c5161f8 None None] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:21:43.012018 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85c65178-1a91-4e04-ad60-944b7452782e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '924ee3f608d34a18b0ddc4695f102a56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b9d94f6a68f7470d9494f112175620dc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:21:43.013091 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85c65178-1a91-4e04-ad60-944b7452782e service cinder] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:21:43.013091 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85c65178-1a91-4e04-ad60-944b7452782e service cinder] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:21:43.013302 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85c65178-1a91-4e04-ad60-944b7452782e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:21:43.089266 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85c65178-1a91-4e04-ad60-944b7452782e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:21:43.090828 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85c65178-1a91-4e04-ad60-944b7452782e service cinder] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:21:43.102008 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 957/1909] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Sep 13 03:21:42 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 03:21:43.727750 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90e389fc-e462-4544-81a0-a94b876a5742 service cinder] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:21:43.736355 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90e389fc-e462-4544-81a0-a94b876a5742 service cinder] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:21:43.740042 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f42de0d2-9f4f-4d46-a67d-6cbe6cc87e24 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '924ee3f608d34a18b0ddc4695f102a56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b9d94f6a68f7470d9494f112175620dc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:21:43.740640 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f42de0d2-9f4f-4d46-a67d-6cbe6cc87e24 service cinder] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:21:43.741057 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f42de0d2-9f4f-4d46-a67d-6cbe6cc87e24 service cinder] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:21:43.741499 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f42de0d2-9f4f-4d46-a67d-6cbe6cc87e24 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:21:43.751145 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f42de0d2-9f4f-4d46-a67d-6cbe6cc87e24 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:21:43.752551 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f42de0d2-9f4f-4d46-a67d-6cbe6cc87e24 service cinder] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:21:43.763183 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 953/1910] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Sep 13 03:21:43 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 03:21:45.472697 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85c65178-1a91-4e04-ad60-944b7452782e service cinder] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:21:45.475451 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-85c65178-1a91-4e04-ad60-944b7452782e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:21:45.546245 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:21:45.573004 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85c65178-1a91-4e04-ad60-944b7452782e service cinder] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:21:45.594216 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1add7ee6-06b4-4bd0-bd31-9625d41111ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76c5181fc7be45279ac00017bb4640b3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:21:45.595015 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1add7ee6-06b4-4bd0-bd31-9625d41111ed admin admin] REQUEST_METHOD: `DELETE` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:21:45.595440 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1add7ee6-06b4-4bd0-bd31-9625d41111ed admin admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:21:45.595814 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1add7ee6-06b4-4bd0-bd31-9625d41111ed admin admin] PATH_INFO: `/v3/users/bdfaadaa24804313a6274259ffb2a18c` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:21:45.597062 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1add7ee6-06b4-4bd0-bd31-9625d41111ed admin admin] RBAC: Authorizing `identity:delete_user(user_id=bdfaadaa24804313a6274259ffb2a18c)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:21:45.599361 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1add7ee6-06b4-4bd0-bd31-9625d41111ed admin admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:21:45.663227 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-1add7ee6-06b4-4bd0-bd31-9625d41111ed admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bdfaadaa24804313a6274259ffb2a18c'} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:21:45.677893 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-1add7ee6-06b4-4bd0-bd31-9625d41111ed admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bdfaadaa24804313a6274259ffb2a18c'} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:21:45.696031 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-1add7ee6-06b4-4bd0-bd31-9625d41111ed admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bdfaadaa24804313a6274259ffb2a18c'} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:21:45.698091 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-1add7ee6-06b4-4bd0-bd31-9625d41111ed admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bdfaadaa24804313a6274259ffb2a18c'} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:21:45.712362 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-1add7ee6-06b4-4bd0-bd31-9625d41111ed admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bdfaadaa24804313a6274259ffb2a18c'} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:21:45.719888 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 958/1911] 10.222.0.18 () {58 vars in 1224 bytes} [Tue Sep 13 03:21:45 2022] DELETE /identity/v3/users/bdfaadaa24804313a6274259ffb2a18c => generated 0 bytes in 248 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 03:21:45.727232 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae276ed-07c0-4f46-a073-d711a999ab2a None None] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:21:45.728094 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae276ed-07c0-4f46-a073-d711a999ab2a None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:21:45.731240 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae276ed-07c0-4f46-a073-d711a999ab2a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:21:45.765472 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.auth.core [None req-4ae276ed-07c0-4f46-a073-d711a999ab2a None None] MFA Rules not processed for user `49f280ab086b4610855c4ffa378517d9`. Rule list: `[]` (Enabled: `True`). {{(pid=81400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 03:21:45.770937 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-4ae276ed-07c0-4f46-a073-d711a999ab2a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:21:45.876810 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 954/1912] 10.222.0.18 () {58 vars in 958 bytes} [Tue Sep 13 03:21:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 03:21:45.888774 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1add7ee6-06b4-4bd0-bd31-9625d41111ed admin admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:21:45.890509 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-1add7ee6-06b4-4bd0-bd31-9625d41111ed admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:21:45.967870 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1add7ee6-06b4-4bd0-bd31-9625d41111ed admin admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:21:45.972007 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4558aecf-0c43-431e-b6bd-46002e1d48ac service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdffbc851086441ebd01022d1e4c2609', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b9d94f6a68f7470d9494f112175620dc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:21:45.973392 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4558aecf-0c43-431e-b6bd-46002e1d48ac service neutron] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:21:45.973964 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4558aecf-0c43-431e-b6bd-46002e1d48ac service neutron] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:21:45.973964 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4558aecf-0c43-431e-b6bd-46002e1d48ac service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:21:46.005468 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4558aecf-0c43-431e-b6bd-46002e1d48ac service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:21:46.007114 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4558aecf-0c43-431e-b6bd-46002e1d48ac service neutron] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:21:46.019416 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 959/1913] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Sep 13 03:21:45 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 131 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 03:21:46.415705 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ae276ed-07c0-4f46-a073-d711a999ab2a None None] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:21:46.417429 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-4ae276ed-07c0-4f46-a073-d711a999ab2a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:21:46.444728 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ae276ed-07c0-4f46-a073-d711a999ab2a None None] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:21:46.450867 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f38f618b-71fc-49db-a102-63dc2b7bd76c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76c5181fc7be45279ac00017bb4640b3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:21:46.451644 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f38f618b-71fc-49db-a102-63dc2b7bd76c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:21:46.452272 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f38f618b-71fc-49db-a102-63dc2b7bd76c admin admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:21:46.453492 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f38f618b-71fc-49db-a102-63dc2b7bd76c admin admin] PATH_INFO: `/v3/projects/133f7e95cccf44e98b48b9ac551098d7` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:21:46.455435 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f38f618b-71fc-49db-a102-63dc2b7bd76c admin admin] RBAC: Authorizing `identity:delete_project(project_id=133f7e95cccf44e98b48b9ac551098d7)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:21:46.457978 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f38f618b-71fc-49db-a102-63dc2b7bd76c admin admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:21:46.519285 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-f38f618b-71fc-49db-a102-63dc2b7bd76c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '133f7e95cccf44e98b48b9ac551098d7', 'request_id': 'req-f38f618b-71fc-49db-a102-63dc2b7bd76c'} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:21:46.526678 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-f38f618b-71fc-49db-a102-63dc2b7bd76c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '133f7e95cccf44e98b48b9ac551098d7', 'request_id': 'req-f38f618b-71fc-49db-a102-63dc2b7bd76c'} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:21:46.536641 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-f38f618b-71fc-49db-a102-63dc2b7bd76c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '133f7e95cccf44e98b48b9ac551098d7', 'request_id': 'req-f38f618b-71fc-49db-a102-63dc2b7bd76c'} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:21:46.561928 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-f38f618b-71fc-49db-a102-63dc2b7bd76c admin admin] The token cache is being invalidate because project 133f7e95cccf44e98b48b9ac551098d7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81400) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 03:21:46.562309 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-f38f618b-71fc-49db-a102-63dc2b7bd76c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:21:46.563046 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-f38f618b-71fc-49db-a102-63dc2b7bd76c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:21:46.565268 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 955/1914] 10.222.0.18 () {58 vars in 1233 bytes} [Tue Sep 13 03:21:46 2022] DELETE /identity/v3/projects/133f7e95cccf44e98b48b9ac551098d7 => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 03:21:46.568825 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4558aecf-0c43-431e-b6bd-46002e1d48ac service neutron] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:21:46.579709 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4558aecf-0c43-431e-b6bd-46002e1d48ac service neutron] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:21:46.649691 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54f617c6-657b-49f1-89e2-3dae1ba47809 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76c5181fc7be45279ac00017bb4640b3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:21:46.650868 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54f617c6-657b-49f1-89e2-3dae1ba47809 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:21:46.651208 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54f617c6-657b-49f1-89e2-3dae1ba47809 admin admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:21:46.651543 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54f617c6-657b-49f1-89e2-3dae1ba47809 admin admin] PATH_INFO: `/v3/users/23038257634a4029856b0b8cb855235e` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:21:46.653040 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54f617c6-657b-49f1-89e2-3dae1ba47809 admin admin] RBAC: Authorizing `identity:delete_user(user_id=23038257634a4029856b0b8cb855235e)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:21:46.656019 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54f617c6-657b-49f1-89e2-3dae1ba47809 admin admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:21:46.662045 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:21:46.763990 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-54f617c6-657b-49f1-89e2-3dae1ba47809 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '23038257634a4029856b0b8cb855235e'} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:21:46.782539 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-54f617c6-657b-49f1-89e2-3dae1ba47809 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '23038257634a4029856b0b8cb855235e'} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:21:46.806241 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-54f617c6-657b-49f1-89e2-3dae1ba47809 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '23038257634a4029856b0b8cb855235e'} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:21:46.807491 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-54f617c6-657b-49f1-89e2-3dae1ba47809 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '23038257634a4029856b0b8cb855235e'} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:21:46.816142 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-54f617c6-657b-49f1-89e2-3dae1ba47809 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '23038257634a4029856b0b8cb855235e'} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:21:46.824566 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 960/1915] 10.222.0.18 () {58 vars in 1224 bytes} [Tue Sep 13 03:21:46 2022] DELETE /identity/v3/users/23038257634a4029856b0b8cb855235e => generated 0 bytes in 256 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 13 03:21:47.213080 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f38f618b-71fc-49db-a102-63dc2b7bd76c admin admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:21:47.214643 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-f38f618b-71fc-49db-a102-63dc2b7bd76c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:21:47.228377 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f38f618b-71fc-49db-a102-63dc2b7bd76c admin admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:21:47.240008 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:21:47.291852 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd701d2e-0039-4808-95d2-fc20b26ccbb9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76c5181fc7be45279ac00017bb4640b3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:21:47.292481 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd701d2e-0039-4808-95d2-fc20b26ccbb9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:21:47.292757 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd701d2e-0039-4808-95d2-fc20b26ccbb9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:21:47.293023 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd701d2e-0039-4808-95d2-fc20b26ccbb9 admin admin] PATH_INFO: `/v3/projects/6cc08951599b4506b42b24e9a8fd9eed` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:21:47.294127 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd701d2e-0039-4808-95d2-fc20b26ccbb9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6cc08951599b4506b42b24e9a8fd9eed)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:21:47.296334 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd701d2e-0039-4808-95d2-fc20b26ccbb9 admin admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:21:47.345442 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-bd701d2e-0039-4808-95d2-fc20b26ccbb9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6cc08951599b4506b42b24e9a8fd9eed', 'request_id': 'req-bd701d2e-0039-4808-95d2-fc20b26ccbb9'} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:21:47.351179 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-bd701d2e-0039-4808-95d2-fc20b26ccbb9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6cc08951599b4506b42b24e9a8fd9eed', 'request_id': 'req-bd701d2e-0039-4808-95d2-fc20b26ccbb9'} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:21:47.357225 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-bd701d2e-0039-4808-95d2-fc20b26ccbb9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6cc08951599b4506b42b24e9a8fd9eed', 'request_id': 'req-bd701d2e-0039-4808-95d2-fc20b26ccbb9'} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:21:47.370630 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-bd701d2e-0039-4808-95d2-fc20b26ccbb9 admin admin] The token cache is being invalidate because project 6cc08951599b4506b42b24e9a8fd9eed was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81400) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 03:21:47.370935 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-bd701d2e-0039-4808-95d2-fc20b26ccbb9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:21:47.371572 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-bd701d2e-0039-4808-95d2-fc20b26ccbb9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:21:47.373067 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 956/1916] 10.222.0.18 () {58 vars in 1233 bytes} [Tue Sep 13 03:21:47 2022] DELETE /identity/v3/projects/6cc08951599b4506b42b24e9a8fd9eed => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 03:21:47.378029 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad505c1f-6a03-44eb-b7cf-0d1b65096e1f None None] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:21:47.378342 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad505c1f-6a03-44eb-b7cf-0d1b65096e1f None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:21:47.378637 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad505c1f-6a03-44eb-b7cf-0d1b65096e1f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:21:47.398664 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.auth.core [None req-ad505c1f-6a03-44eb-b7cf-0d1b65096e1f None None] MFA Rules not processed for user `49f280ab086b4610855c4ffa378517d9`. Rule list: `[]` (Enabled: `True`). {{(pid=81399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 03:21:47.399873 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-ad505c1f-6a03-44eb-b7cf-0d1b65096e1f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:21:47.461362 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 961/1917] 10.222.0.18 () {58 vars in 958 bytes} [Tue Sep 13 03:21:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 03:21:47.469136 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6277c85a-9f35-4662-99fd-20b94db450c2 None None] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:21:47.469559 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6277c85a-9f35-4662-99fd-20b94db450c2 None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:21:47.469833 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6277c85a-9f35-4662-99fd-20b94db450c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:21:47.491951 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.auth.core [None req-6277c85a-9f35-4662-99fd-20b94db450c2 None None] MFA Rules not processed for user `49f280ab086b4610855c4ffa378517d9`. Rule list: `[]` (Enabled: `True`). {{(pid=81400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 03:21:47.493170 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-6277c85a-9f35-4662-99fd-20b94db450c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:21:47.500983 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 957/1918] 10.222.0.18 () {58 vars in 958 bytes} [Tue Sep 13 03:21:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 03:21:47.505189 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad505c1f-6a03-44eb-b7cf-0d1b65096e1f None None] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:21:47.514991 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad505c1f-6a03-44eb-b7cf-0d1b65096e1f None None] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:21:47.519570 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d1fa995-4f57-4a72-8e5e-9ce42ae86edd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76c5181fc7be45279ac00017bb4640b3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:21:47.520163 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d1fa995-4f57-4a72-8e5e-9ce42ae86edd admin admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:21:47.520443 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d1fa995-4f57-4a72-8e5e-9ce42ae86edd admin admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:21:47.520714 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d1fa995-4f57-4a72-8e5e-9ce42ae86edd admin admin] PATH_INFO: `/v3/domains` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:21:47.521352 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d1fa995-4f57-4a72-8e5e-9ce42ae86edd admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 03:21:47.521653 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d1fa995-4f57-4a72-8e5e-9ce42ae86edd admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:21:47.523275 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d1fa995-4f57-4a72-8e5e-9ce42ae86edd admin admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:21:47.534652 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 962/1919] 10.222.0.18 () {58 vars in 1153 bytes} [Tue Sep 13 03:21:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 03:21:47.539040 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6277c85a-9f35-4662-99fd-20b94db450c2 None None] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:21:47.548217 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6277c85a-9f35-4662-99fd-20b94db450c2 None None] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:21:47.553445 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00ed686c-444c-407b-9215-da124719623d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76c5181fc7be45279ac00017bb4640b3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:21:47.554027 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00ed686c-444c-407b-9215-da124719623d admin admin] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:21:47.554297 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00ed686c-444c-407b-9215-da124719623d admin admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:21:47.554563 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00ed686c-444c-407b-9215-da124719623d admin admin] PATH_INFO: `/v3/projects` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:21:47.555219 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00ed686c-444c-407b-9215-da124719623d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:21:47.557110 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00ed686c-444c-407b-9215-da124719623d admin admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:21:47.583463 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 958/1920] 10.222.0.18 () {60 vars in 1153 bytes} [Tue Sep 13 03:21:47 2022] POST /identity/v3/projects => generated 365 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 03:21:47.587421 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d1fa995-4f57-4a72-8e5e-9ce42ae86edd admin admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:21:47.597211 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d1fa995-4f57-4a72-8e5e-9ce42ae86edd admin admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:21:47.653187 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7e471b4-88e0-4c91-bdad-3d7ca056d5f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76c5181fc7be45279ac00017bb4640b3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:21:47.653785 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7e471b4-88e0-4c91-bdad-3d7ca056d5f6 admin admin] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:21:47.654060 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7e471b4-88e0-4c91-bdad-3d7ca056d5f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:21:47.654333 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7e471b4-88e0-4c91-bdad-3d7ca056d5f6 admin admin] PATH_INFO: `/v3/users` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:21:47.655000 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7e471b4-88e0-4c91-bdad-3d7ca056d5f6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:21:47.656913 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7e471b4-88e0-4c91-bdad-3d7ca056d5f6 admin admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:21:47.675163 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=86,keystone:DELETE=10,keystone:INSERT=4 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:21:47.681450 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 963/1921] 10.222.0.18 () {60 vars in 1144 bytes} [Tue Sep 13 03:21:47 2022] POST /identity/v3/users => generated 283 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 03:21:47.685778 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00ed686c-444c-407b-9215-da124719623d admin admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:21:47.696281 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00ed686c-444c-407b-9215-da124719623d admin admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:21:47.701181 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4d72663-74d6-4a68-9c36-c4bafd4b160f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76c5181fc7be45279ac00017bb4640b3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:21:47.701804 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d72663-74d6-4a68-9c36-c4bafd4b160f admin admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:21:47.702070 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d72663-74d6-4a68-9c36-c4bafd4b160f admin admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:21:47.702339 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d72663-74d6-4a68-9c36-c4bafd4b160f admin admin] PATH_INFO: `/v3/roles` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:21:47.703033 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4d72663-74d6-4a68-9c36-c4bafd4b160f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:21:47.704624 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4d72663-74d6-4a68-9c36-c4bafd4b160f admin admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:21:47.714926 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 959/1922] 10.222.0.18 () {58 vars in 1122 bytes} [Tue Sep 13 03:21:47 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 03:21:47.718988 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7e471b4-88e0-4c91-bdad-3d7ca056d5f6 admin admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:21:47.729488 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7e471b4-88e0-4c91-bdad-3d7ca056d5f6 admin admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:21:47.735173 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09853b78-abda-41cb-a68f-50bc1b799092 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76c5181fc7be45279ac00017bb4640b3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:21:47.735509 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09853b78-abda-41cb-a68f-50bc1b799092 admin admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:21:47.735791 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09853b78-abda-41cb-a68f-50bc1b799092 admin admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:21:47.736069 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09853b78-abda-41cb-a68f-50bc1b799092 admin admin] PATH_INFO: `/v3/roles` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:21:47.736715 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09853b78-abda-41cb-a68f-50bc1b799092 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:21:47.739364 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09853b78-abda-41cb-a68f-50bc1b799092 admin admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:21:47.751667 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 964/1923] 10.222.0.18 () {58 vars in 1122 bytes} [Tue Sep 13 03:21:47 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 03:21:47.755965 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4d72663-74d6-4a68-9c36-c4bafd4b160f admin admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:21:47.766157 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4d72663-74d6-4a68-9c36-c4bafd4b160f admin admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:21:47.771487 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3acac04f-8fbf-46f4-a92a-edac551a7c78 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76c5181fc7be45279ac00017bb4640b3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:21:47.772226 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3acac04f-8fbf-46f4-a92a-edac551a7c78 admin admin] REQUEST_METHOD: `PUT` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:21:47.772504 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3acac04f-8fbf-46f4-a92a-edac551a7c78 admin admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:21:47.772785 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3acac04f-8fbf-46f4-a92a-edac551a7c78 admin admin] PATH_INFO: `/v3/projects/32595c502ba648b79b4c48ba7017b192/users/3dd4fd8c6c454308987f4e98a901b51a/roles/1eb7a5b5203945c29e6d22d653f47847` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:21:47.802842 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3acac04f-8fbf-46f4-a92a-edac551a7c78 admin admin] RBAC: Authorizing `identity:create_grant(project_id=32595c502ba648b79b4c48ba7017b192, user_id=3dd4fd8c6c454308987f4e98a901b51a, role_id=1eb7a5b5203945c29e6d22d653f47847)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:21:47.802842 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3acac04f-8fbf-46f4-a92a-edac551a7c78 admin admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:21:47.809222 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 960/1924] 10.222.0.18 () {60 vars in 1483 bytes} [Tue Sep 13 03:21:47 2022] PUT /identity/v3/projects/32595c502ba648b79b4c48ba7017b192/users/3dd4fd8c6c454308987f4e98a901b51a/roles/1eb7a5b5203945c29e6d22d653f47847 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 13 03:21:47.820469 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bbebb51-7c37-4275-bd4b-39fd126d85e1 None None] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:21:47.820859 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bbebb51-7c37-4275-bd4b-39fd126d85e1 None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:21:47.821313 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bbebb51-7c37-4275-bd4b-39fd126d85e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:21:47.850264 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.auth.core [None req-4bbebb51-7c37-4275-bd4b-39fd126d85e1 None None] MFA Rules not processed for user `3dd4fd8c6c454308987f4e98a901b51a`. Rule list: `[]` (Enabled: `True`). {{(pid=81399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 03:21:47.851622 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-4bbebb51-7c37-4275-bd4b-39fd126d85e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:21:47.928390 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 965/1925] 10.222.0.18 () {58 vars in 958 bytes} [Tue Sep 13 03:21:47 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 03:21:47.929524 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3acac04f-8fbf-46f4-a92a-edac551a7c78 admin admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:21:48.006462 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:21:48.030101 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bbebb51-7c37-4275-bd4b-39fd126d85e1 None None] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:21:48.032538 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-4bbebb51-7c37-4275-bd4b-39fd126d85e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:21:48.035660 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3acac04f-8fbf-46f4-a92a-edac551a7c78 admin admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:21:48.041586 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7bce93f-6c0e-44b2-85a2-42bb1ca1f3a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76c5181fc7be45279ac00017bb4640b3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:21:48.042397 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7bce93f-6c0e-44b2-85a2-42bb1ca1f3a5 admin admin] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:21:48.042885 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7bce93f-6c0e-44b2-85a2-42bb1ca1f3a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:21:48.043328 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7bce93f-6c0e-44b2-85a2-42bb1ca1f3a5 admin admin] PATH_INFO: `/v3/projects` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:21:48.044163 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7bce93f-6c0e-44b2-85a2-42bb1ca1f3a5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:21:48.047104 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7bce93f-6c0e-44b2-85a2-42bb1ca1f3a5 admin admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:21:48.060514 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bbebb51-7c37-4275-bd4b-39fd126d85e1 None None] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:21:48.071169 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8d752b8-7b42-441b-b8e2-98a3ccd73c5d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76c5181fc7be45279ac00017bb4640b3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:21:48.072213 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d752b8-7b42-441b-b8e2-98a3ccd73c5d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:21:48.072719 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d752b8-7b42-441b-b8e2-98a3ccd73c5d admin admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:21:48.073093 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d752b8-7b42-441b-b8e2-98a3ccd73c5d admin admin] PATH_INFO: `/v3/users/cbde25e814c34f43a2135059a079c33f` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:21:48.077050 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8d752b8-7b42-441b-b8e2-98a3ccd73c5d admin admin] RBAC: Authorizing `identity:delete_user(user_id=cbde25e814c34f43a2135059a079c33f)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:21:48.079416 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8d752b8-7b42-441b-b8e2-98a3ccd73c5d admin admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:21:48.106686 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 961/1926] 10.222.0.18 () {60 vars in 1153 bytes} [Tue Sep 13 03:21:47 2022] POST /identity/v3/projects => generated 363 bytes in 174 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 03:21:48.109608 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7bce93f-6c0e-44b2-85a2-42bb1ca1f3a5 admin admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:21:48.203020 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-b8d752b8-7b42-441b-b8e2-98a3ccd73c5d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cbde25e814c34f43a2135059a079c33f'} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:21:48.224931 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-b8d752b8-7b42-441b-b8e2-98a3ccd73c5d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cbde25e814c34f43a2135059a079c33f'} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:21:48.257353 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-b8d752b8-7b42-441b-b8e2-98a3ccd73c5d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cbde25e814c34f43a2135059a079c33f'} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:21:48.258108 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-b8d752b8-7b42-441b-b8e2-98a3ccd73c5d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cbde25e814c34f43a2135059a079c33f'} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:21:48.266512 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7bce93f-6c0e-44b2-85a2-42bb1ca1f3a5 admin admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:21:48.270122 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-b8d752b8-7b42-441b-b8e2-98a3ccd73c5d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cbde25e814c34f43a2135059a079c33f'} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:21:48.274813 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4e9f47d-8b1d-4880-88a1-4eb39ed3121d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76c5181fc7be45279ac00017bb4640b3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:21:48.275669 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4e9f47d-8b1d-4880-88a1-4eb39ed3121d admin admin] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:21:48.275955 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4e9f47d-8b1d-4880-88a1-4eb39ed3121d admin admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:21:48.276230 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4e9f47d-8b1d-4880-88a1-4eb39ed3121d admin admin] PATH_INFO: `/v3/users` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:21:48.277397 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4e9f47d-8b1d-4880-88a1-4eb39ed3121d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:21:48.279158 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4e9f47d-8b1d-4880-88a1-4eb39ed3121d admin admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:21:48.283007 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 966/1927] 10.222.0.18 () {58 vars in 1224 bytes} [Tue Sep 13 03:21:48 2022] DELETE /identity/v3/users/cbde25e814c34f43a2135059a079c33f => generated 0 bytes in 253 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 03:21:48.287105 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d68c84-e6df-454a-bc4a-8eaf11535714 None None] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:21:48.288488 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d68c84-e6df-454a-bc4a-8eaf11535714 None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:21:48.289091 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d68c84-e6df-454a-bc4a-8eaf11535714 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:21:48.311542 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 962/1928] 10.222.0.18 () {60 vars in 1144 bytes} [Tue Sep 13 03:21:48 2022] POST /identity/v3/users => generated 288 bytes in 202 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 03:21:48.313061 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-791795b7-874a-419b-ac1b-9ccbf15ec11b None None] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:21:48.313450 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-791795b7-874a-419b-ac1b-9ccbf15ec11b None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:21:48.313821 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-791795b7-874a-419b-ac1b-9ccbf15ec11b None None] PATH_INFO: `/` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:21:48.315521 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 963/1929] 10.222.0.18 () {54 vars in 921 bytes} [Tue Sep 13 03:21:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 03:21:48.317249 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-791795b7-874a-419b-ac1b-9ccbf15ec11b None None] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:21:48.319139 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-791795b7-874a-419b-ac1b-9ccbf15ec11b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:21:48.321093 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.auth.core [None req-04d68c84-e6df-454a-bc4a-8eaf11535714 None None] MFA Rules not processed for user `49f280ab086b4610855c4ffa378517d9`. Rule list: `[]` (Enabled: `True`). {{(pid=81399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 03:21:48.322794 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-04d68c84-e6df-454a-bc4a-8eaf11535714 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:21:48.421573 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:21:48.434061 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-791795b7-874a-419b-ac1b-9ccbf15ec11b None None] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:21:48.439403 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92dad945-6e97-4f8d-8821-7f14249572d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76c5181fc7be45279ac00017bb4640b3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:21:48.442645 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92dad945-6e97-4f8d-8821-7f14249572d9 admin admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:21:48.442936 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92dad945-6e97-4f8d-8821-7f14249572d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:21:48.443209 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92dad945-6e97-4f8d-8821-7f14249572d9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:21:48.443865 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92dad945-6e97-4f8d-8821-7f14249572d9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:21:48.443965 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 967/1930] 10.222.0.18 () {58 vars in 958 bytes} [Tue Sep 13 03:21:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 03:21:48.445334 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b0dd8c-af84-409a-ba9e-72fd8ac1be32 None None] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:21:48.445629 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b0dd8c-af84-409a-ba9e-72fd8ac1be32 None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:21:48.445727 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92dad945-6e97-4f8d-8821-7f14249572d9 admin admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:21:48.445904 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b0dd8c-af84-409a-ba9e-72fd8ac1be32 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:21:48.461217 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 964/1931] 10.222.0.18 () {58 vars in 1122 bytes} [Tue Sep 13 03:21:48 2022] GET /identity/v3/roles => generated 1420 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 03:21:48.461217 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92dad945-6e97-4f8d-8821-7f14249572d9 admin admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:21:48.462878 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-92dad945-6e97-4f8d-8821-7f14249572d9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:21:48.466033 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.auth.core [None req-f4b0dd8c-af84-409a-ba9e-72fd8ac1be32 None None] MFA Rules not processed for user `c7377f5bee4a49959182e1bd34bc97a4`. Rule list: `[]` (Enabled: `True`). {{(pid=81399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 03:21:48.467245 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-f4b0dd8c-af84-409a-ba9e-72fd8ac1be32 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:21:48.542024 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92dad945-6e97-4f8d-8821-7f14249572d9 admin admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:21:48.552374 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f4182e6-235f-4da5-b92b-7f86de9d733a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdffbc851086441ebd01022d1e4c2609', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b9d94f6a68f7470d9494f112175620dc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:21:48.553078 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f4182e6-235f-4da5-b92b-7f86de9d733a service neutron] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:21:48.553380 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f4182e6-235f-4da5-b92b-7f86de9d733a service neutron] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:21:48.553649 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f4182e6-235f-4da5-b92b-7f86de9d733a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:21:48.555936 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 968/1932] 10.222.0.18 () {58 vars in 1018 bytes} [Tue Sep 13 03:21:48 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 03:21:48.560901 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:21:48.566694 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4b0dd8c-af84-409a-ba9e-72fd8ac1be32 None None] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:21:48.568839 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f4182e6-235f-4da5-b92b-7f86de9d733a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:21:48.568839 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f4182e6-235f-4da5-b92b-7f86de9d733a service neutron] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:21:48.584076 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4b0dd8c-af84-409a-ba9e-72fd8ac1be32 None None] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:21:48.584796 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 965/1933] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Sep 13 03:21:48 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 126 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 03:21:48.586073 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f4182e6-235f-4da5-b92b-7f86de9d733a service neutron] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:21:48.587083 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-2f4182e6-235f-4da5-b92b-7f86de9d733a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:21:48.589760 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbbdd125-8272-4817-8c7f-9e9041996b52 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76c5181fc7be45279ac00017bb4640b3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:21:48.591045 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbbdd125-8272-4817-8c7f-9e9041996b52 admin admin] REQUEST_METHOD: `PUT` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:21:48.591327 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbbdd125-8272-4817-8c7f-9e9041996b52 admin admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:21:48.591601 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbbdd125-8272-4817-8c7f-9e9041996b52 admin admin] PATH_INFO: `/v3/projects/ef3ba9c5d0e04081a26f45f72d834c60/users/438cfe0d626b49198cc06b1153ab3596/roles/425d4b481c7846e0bc07e7e805d5b15f` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:21:48.604818 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f4182e6-235f-4da5-b92b-7f86de9d733a service neutron] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:21:48.610083 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbbdd125-8272-4817-8c7f-9e9041996b52 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ef3ba9c5d0e04081a26f45f72d834c60, user_id=438cfe0d626b49198cc06b1153ab3596, role_id=425d4b481c7846e0bc07e7e805d5b15f)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:21:48.611143 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c25ba3e-1266-45bc-b6cc-536f945bc350 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7377f5bee4a49959182e1bd34bc97a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b9d94f6a68f7470d9494f112175620dc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:21:48.611942 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c25ba3e-1266-45bc-b6cc-536f945bc350 service nova] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:21:48.612289 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c25ba3e-1266-45bc-b6cc-536f945bc350 service nova] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:21:48.612662 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c25ba3e-1266-45bc-b6cc-536f945bc350 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:21:48.612769 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbbdd125-8272-4817-8c7f-9e9041996b52 admin admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:21:48.622903 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c25ba3e-1266-45bc-b6cc-536f945bc350 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:21:48.624606 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c25ba3e-1266-45bc-b6cc-536f945bc350 service nova] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:21:48.628711 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 969/1934] 10.222.0.18 () {60 vars in 1483 bytes} [Tue Sep 13 03:21:48 2022] PUT /identity/v3/projects/ef3ba9c5d0e04081a26f45f72d834c60/users/438cfe0d626b49198cc06b1153ab3596/roles/425d4b481c7846e0bc07e7e805d5b15f => generated 0 bytes in 72 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 03:21:48.633219 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbbdd125-8272-4817-8c7f-9e9041996b52 admin admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:21:48.636881 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 966/1935] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Sep 13 03:21:48 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 03:21:48.707234 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbbdd125-8272-4817-8c7f-9e9041996b52 admin admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:21:48.715488 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2180576b-ee8f-475d-9359-198b848aced5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76c5181fc7be45279ac00017bb4640b3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:21:48.716328 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2180576b-ee8f-475d-9359-198b848aced5 admin admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:21:48.716869 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2180576b-ee8f-475d-9359-198b848aced5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:21:48.717324 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2180576b-ee8f-475d-9359-198b848aced5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:21:48.718234 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2180576b-ee8f-475d-9359-198b848aced5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:21:48.720049 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2180576b-ee8f-475d-9359-198b848aced5 admin admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:21:48.734447 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 970/1936] 10.222.0.18 () {58 vars in 1122 bytes} [Tue Sep 13 03:21:48 2022] GET /identity/v3/roles => generated 1420 bytes in 99 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 13 03:21:48.735612 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c25ba3e-1266-45bc-b6cc-536f945bc350 service nova] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:21:48.746487 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c25ba3e-1266-45bc-b6cc-536f945bc350 service nova] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:21:48.751442 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f171abc-7277-4bfe-9315-8dac3fbbe545 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76c5181fc7be45279ac00017bb4640b3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:21:48.752165 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f171abc-7277-4bfe-9315-8dac3fbbe545 admin admin] REQUEST_METHOD: `PUT` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:21:48.752661 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f171abc-7277-4bfe-9315-8dac3fbbe545 admin admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:21:48.752846 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f171abc-7277-4bfe-9315-8dac3fbbe545 admin admin] PATH_INFO: `/v3/projects/ef3ba9c5d0e04081a26f45f72d834c60/users/438cfe0d626b49198cc06b1153ab3596/roles/425d4b481c7846e0bc07e7e805d5b15f` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:21:48.755623 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f171abc-7277-4bfe-9315-8dac3fbbe545 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ef3ba9c5d0e04081a26f45f72d834c60, user_id=438cfe0d626b49198cc06b1153ab3596, role_id=425d4b481c7846e0bc07e7e805d5b15f)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:21:48.758357 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f171abc-7277-4bfe-9315-8dac3fbbe545 admin admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:21:48.771031 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 967/1937] 10.222.0.18 () {60 vars in 1483 bytes} [Tue Sep 13 03:21:48 2022] PUT /identity/v3/projects/ef3ba9c5d0e04081a26f45f72d834c60/users/438cfe0d626b49198cc06b1153ab3596/roles/425d4b481c7846e0bc07e7e805d5b15f => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 13 03:21:48.781285 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae6cb4b1-b1e0-43a9-9ff6-eafc6ae0b119 None None] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:21:48.781573 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae6cb4b1-b1e0-43a9-9ff6-eafc6ae0b119 None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:21:48.781846 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae6cb4b1-b1e0-43a9-9ff6-eafc6ae0b119 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:21:48.820442 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.auth.core [None req-ae6cb4b1-b1e0-43a9-9ff6-eafc6ae0b119 None None] MFA Rules not processed for user `438cfe0d626b49198cc06b1153ab3596`. Rule list: `[]` (Enabled: `True`). {{(pid=81399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 03:21:48.821841 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-ae6cb4b1-b1e0-43a9-9ff6-eafc6ae0b119 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:21:48.878501 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:21:48.915126 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 971/1938] 10.222.0.18 () {58 vars in 958 bytes} [Tue Sep 13 03:21:48 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 03:21:48.928080 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f171abc-7277-4bfe-9315-8dac3fbbe545 admin admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:21:48.930175 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-7f171abc-7277-4bfe-9315-8dac3fbbe545 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:21:49.011227 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f171abc-7277-4bfe-9315-8dac3fbbe545 admin admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:21:49.015212 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4579c3de-6499-4449-bcc4-ce54158b2d83 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '924ee3f608d34a18b0ddc4695f102a56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b9d94f6a68f7470d9494f112175620dc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:21:49.015774 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4579c3de-6499-4449-bcc4-ce54158b2d83 service cinder] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:21:49.016056 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4579c3de-6499-4449-bcc4-ce54158b2d83 service cinder] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:21:49.016357 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4579c3de-6499-4449-bcc4-ce54158b2d83 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:21:49.031769 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4579c3de-6499-4449-bcc4-ce54158b2d83 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:21:49.033508 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4579c3de-6499-4449-bcc4-ce54158b2d83 service cinder] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:21:49.045774 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 968/1939] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Sep 13 03:21:48 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 118 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 03:21:49.196424 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae6cb4b1-b1e0-43a9-9ff6-eafc6ae0b119 None None] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:21:49.206588 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae6cb4b1-b1e0-43a9-9ff6-eafc6ae0b119 None None] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:21:49.211700 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b75cda9-deb0-4cd3-b28b-3b6784df8369 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '924ee3f608d34a18b0ddc4695f102a56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b9d94f6a68f7470d9494f112175620dc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:21:49.212765 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b75cda9-deb0-4cd3-b28b-3b6784df8369 service cinder] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:21:49.213196 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b75cda9-deb0-4cd3-b28b-3b6784df8369 service cinder] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:21:49.213647 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b75cda9-deb0-4cd3-b28b-3b6784df8369 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:21:49.216832 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-6b75cda9-deb0-4cd3-b28b-3b6784df8369 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:21:49.318010 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b75cda9-deb0-4cd3-b28b-3b6784df8369 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:21:49.319769 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b75cda9-deb0-4cd3-b28b-3b6784df8369 service cinder] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:21:49.331046 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 972/1940] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Sep 13 03:21:49 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 135 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 03:21:49.480313 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4579c3de-6499-4449-bcc4-ce54158b2d83 service cinder] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:21:49.481738 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-4579c3de-6499-4449-bcc4-ce54158b2d83 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:21:49.590283 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4579c3de-6499-4449-bcc4-ce54158b2d83 service cinder] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:21:49.597041 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8da4e2d-0609-4463-b972-27c41b30c5c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76c5181fc7be45279ac00017bb4640b3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:21:49.598246 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8da4e2d-0609-4463-b972-27c41b30c5c1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:21:49.598526 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8da4e2d-0609-4463-b972-27c41b30c5c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:21:49.598829 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8da4e2d-0609-4463-b972-27c41b30c5c1 admin admin] PATH_INFO: `/v3/projects/e5caec8137784904a0ef679b2f01729f` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:21:49.600004 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8da4e2d-0609-4463-b972-27c41b30c5c1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e5caec8137784904a0ef679b2f01729f)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:21:49.602100 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8da4e2d-0609-4463-b972-27c41b30c5c1 admin admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:21:49.657469 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-a8da4e2d-0609-4463-b972-27c41b30c5c1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e5caec8137784904a0ef679b2f01729f', 'request_id': 'req-a8da4e2d-0609-4463-b972-27c41b30c5c1'} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:21:49.664637 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-a8da4e2d-0609-4463-b972-27c41b30c5c1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e5caec8137784904a0ef679b2f01729f', 'request_id': 'req-a8da4e2d-0609-4463-b972-27c41b30c5c1'} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:21:49.672163 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-a8da4e2d-0609-4463-b972-27c41b30c5c1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e5caec8137784904a0ef679b2f01729f', 'request_id': 'req-a8da4e2d-0609-4463-b972-27c41b30c5c1'} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:21:49.686376 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:21:49.689391 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-a8da4e2d-0609-4463-b972-27c41b30c5c1 admin admin] The token cache is being invalidate because project e5caec8137784904a0ef679b2f01729f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81400) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 03:21:49.689733 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-a8da4e2d-0609-4463-b972-27c41b30c5c1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:21:49.692260 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-a8da4e2d-0609-4463-b972-27c41b30c5c1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:21:49.694249 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 969/1941] 10.222.0.18 () {58 vars in 1233 bytes} [Tue Sep 13 03:21:49 2022] DELETE /identity/v3/projects/e5caec8137784904a0ef679b2f01729f => generated 0 bytes in 214 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 03:21:49.697942 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b75cda9-deb0-4cd3-b28b-3b6784df8369 service cinder] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:21:49.709956 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b75cda9-deb0-4cd3-b28b-3b6784df8369 service cinder] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:21:49.799810 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c3959d2-ce76-44b3-8655-a8227773cef8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76c5181fc7be45279ac00017bb4640b3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:21:49.800948 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c3959d2-ce76-44b3-8655-a8227773cef8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:21:49.802152 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c3959d2-ce76-44b3-8655-a8227773cef8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:21:49.802869 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c3959d2-ce76-44b3-8655-a8227773cef8 admin admin] PATH_INFO: `/v3/users/ba68adca11f541c685d476e6d181fc4a` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:21:49.804944 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c3959d2-ce76-44b3-8655-a8227773cef8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ba68adca11f541c685d476e6d181fc4a)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:21:49.807200 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c3959d2-ce76-44b3-8655-a8227773cef8 admin admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:21:49.936768 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-2c3959d2-ce76-44b3-8655-a8227773cef8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ba68adca11f541c685d476e6d181fc4a'} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:21:49.958815 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-2c3959d2-ce76-44b3-8655-a8227773cef8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ba68adca11f541c685d476e6d181fc4a'} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:21:49.995706 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-2c3959d2-ce76-44b3-8655-a8227773cef8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ba68adca11f541c685d476e6d181fc4a'} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:21:49.997864 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-2c3959d2-ce76-44b3-8655-a8227773cef8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ba68adca11f541c685d476e6d181fc4a'} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:21:50.008978 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-2c3959d2-ce76-44b3-8655-a8227773cef8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ba68adca11f541c685d476e6d181fc4a'} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:21:50.011230 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:21:50.023093 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 973/1942] 10.222.0.18 () {58 vars in 1224 bytes} [Tue Sep 13 03:21:49 2022] DELETE /identity/v3/users/ba68adca11f541c685d476e6d181fc4a => generated 0 bytes in 326 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 03:21:50.602449 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8da4e2d-0609-4463-b972-27c41b30c5c1 admin admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:21:50.604015 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-a8da4e2d-0609-4463-b972-27c41b30c5c1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:21:50.630651 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8da4e2d-0609-4463-b972-27c41b30c5c1 admin admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:21:50.750714 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9743e66-7563-461d-b98f-fbfae8965c6b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76c5181fc7be45279ac00017bb4640b3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:21:50.751979 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9743e66-7563-461d-b98f-fbfae8965c6b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:21:50.752779 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9743e66-7563-461d-b98f-fbfae8965c6b admin admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:21:50.753423 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9743e66-7563-461d-b98f-fbfae8965c6b admin admin] PATH_INFO: `/v3/projects/a2f8da1d82034dd69e09f8ae8c48958d` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:21:50.756312 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9743e66-7563-461d-b98f-fbfae8965c6b admin admin] RBAC: Authorizing `identity:delete_project(project_id=a2f8da1d82034dd69e09f8ae8c48958d)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:21:50.759973 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9743e66-7563-461d-b98f-fbfae8965c6b admin admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:21:50.830196 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-e9743e66-7563-461d-b98f-fbfae8965c6b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a2f8da1d82034dd69e09f8ae8c48958d', 'request_id': 'req-e9743e66-7563-461d-b98f-fbfae8965c6b'} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:21:50.836968 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-e9743e66-7563-461d-b98f-fbfae8965c6b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a2f8da1d82034dd69e09f8ae8c48958d', 'request_id': 'req-e9743e66-7563-461d-b98f-fbfae8965c6b'} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:21:50.847242 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-e9743e66-7563-461d-b98f-fbfae8965c6b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a2f8da1d82034dd69e09f8ae8c48958d', 'request_id': 'req-e9743e66-7563-461d-b98f-fbfae8965c6b'} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:21:50.861037 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-e9743e66-7563-461d-b98f-fbfae8965c6b admin admin] The token cache is being invalidate because project a2f8da1d82034dd69e09f8ae8c48958d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81400) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 03:21:50.861326 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-e9743e66-7563-461d-b98f-fbfae8965c6b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:21:50.862383 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-e9743e66-7563-461d-b98f-fbfae8965c6b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:21:50.864513 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 970/1943] 10.222.0.18 () {58 vars in 1233 bytes} [Tue Sep 13 03:21:50 2022] DELETE /identity/v3/projects/a2f8da1d82034dd69e09f8ae8c48958d => generated 0 bytes in 263 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 03:21:50.868759 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c3959d2-ce76-44b3-8655-a8227773cef8 admin admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:21:50.870263 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-2c3959d2-ce76-44b3-8655-a8227773cef8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:21:50.883450 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c3959d2-ce76-44b3-8655-a8227773cef8 admin admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:21:50.941782 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54d13182-9d6e-4e86-a47e-f025092ca256 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76c5181fc7be45279ac00017bb4640b3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:21:50.942427 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54d13182-9d6e-4e86-a47e-f025092ca256 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:21:50.942735 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54d13182-9d6e-4e86-a47e-f025092ca256 admin admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:21:50.943012 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54d13182-9d6e-4e86-a47e-f025092ca256 admin admin] PATH_INFO: `/v3/users/655c24322f564615b084db8d2d8c4327` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:21:50.944140 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54d13182-9d6e-4e86-a47e-f025092ca256 admin admin] RBAC: Authorizing `identity:delete_user(user_id=655c24322f564615b084db8d2d8c4327)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:21:50.946060 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54d13182-9d6e-4e86-a47e-f025092ca256 admin admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:21:50.994122 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-54d13182-9d6e-4e86-a47e-f025092ca256 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '655c24322f564615b084db8d2d8c4327'} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:21:51.003336 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-54d13182-9d6e-4e86-a47e-f025092ca256 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '655c24322f564615b084db8d2d8c4327'} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:21:51.017232 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-54d13182-9d6e-4e86-a47e-f025092ca256 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '655c24322f564615b084db8d2d8c4327'} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:21:51.018015 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-54d13182-9d6e-4e86-a47e-f025092ca256 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '655c24322f564615b084db8d2d8c4327'} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:21:51.025931 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-54d13182-9d6e-4e86-a47e-f025092ca256 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '655c24322f564615b084db8d2d8c4327'} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:21:51.032025 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 974/1944] 10.222.0.18 () {58 vars in 1224 bytes} [Tue Sep 13 03:21:50 2022] DELETE /identity/v3/users/655c24322f564615b084db8d2d8c4327 => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 03:21:51.371158 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9743e66-7563-461d-b98f-fbfae8965c6b admin admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:21:51.372528 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-e9743e66-7563-461d-b98f-fbfae8965c6b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:21:51.387110 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9743e66-7563-461d-b98f-fbfae8965c6b admin admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:21:51.465887 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7396aeaa-46c4-4195-bc36-3721f25b87b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76c5181fc7be45279ac00017bb4640b3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:21:51.466525 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7396aeaa-46c4-4195-bc36-3721f25b87b6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:21:51.466849 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7396aeaa-46c4-4195-bc36-3721f25b87b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:21:51.467122 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7396aeaa-46c4-4195-bc36-3721f25b87b6 admin admin] PATH_INFO: `/v3/projects/16268c8e0db548089f1d7453d8d3dc0c` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:21:51.468247 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7396aeaa-46c4-4195-bc36-3721f25b87b6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=16268c8e0db548089f1d7453d8d3dc0c)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:21:51.470164 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7396aeaa-46c4-4195-bc36-3721f25b87b6 admin admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:21:51.512382 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:21:51.532691 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-7396aeaa-46c4-4195-bc36-3721f25b87b6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '16268c8e0db548089f1d7453d8d3dc0c', 'request_id': 'req-7396aeaa-46c4-4195-bc36-3721f25b87b6'} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:21:51.538992 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-7396aeaa-46c4-4195-bc36-3721f25b87b6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '16268c8e0db548089f1d7453d8d3dc0c', 'request_id': 'req-7396aeaa-46c4-4195-bc36-3721f25b87b6'} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:21:51.545794 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-7396aeaa-46c4-4195-bc36-3721f25b87b6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '16268c8e0db548089f1d7453d8d3dc0c', 'request_id': 'req-7396aeaa-46c4-4195-bc36-3721f25b87b6'} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:21:51.559686 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-7396aeaa-46c4-4195-bc36-3721f25b87b6 admin admin] The token cache is being invalidate because project 16268c8e0db548089f1d7453d8d3dc0c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81400) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 03:21:51.559990 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-7396aeaa-46c4-4195-bc36-3721f25b87b6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:21:51.560598 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-7396aeaa-46c4-4195-bc36-3721f25b87b6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:21:51.562136 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 971/1945] 10.222.0.18 () {58 vars in 1233 bytes} [Tue Sep 13 03:21:51 2022] DELETE /identity/v3/projects/16268c8e0db548089f1d7453d8d3dc0c => generated 0 bytes in 192 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 03:21:51.567706 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e60c76-f877-4677-bbf5-02801dbfb3d4 None None] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:21:51.568122 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e60c76-f877-4677-bbf5-02801dbfb3d4 None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:21:51.568517 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e60c76-f877-4677-bbf5-02801dbfb3d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:21:51.589930 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.auth.core [None req-a5e60c76-f877-4677-bbf5-02801dbfb3d4 None None] MFA Rules not processed for user `49f280ab086b4610855c4ffa378517d9`. Rule list: `[]` (Enabled: `True`). {{(pid=81399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 03:21:51.591296 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-a5e60c76-f877-4677-bbf5-02801dbfb3d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:21:51.670071 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 975/1946] 10.222.0.18 () {58 vars in 958 bytes} [Tue Sep 13 03:21:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 03:21:51.678047 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b78e1126-b7db-465b-8271-7b48776257b3 None None] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:21:51.678529 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b78e1126-b7db-465b-8271-7b48776257b3 None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:21:51.678669 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b78e1126-b7db-465b-8271-7b48776257b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:21:51.698960 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.auth.core [None req-b78e1126-b7db-465b-8271-7b48776257b3 None None] MFA Rules not processed for user `49f280ab086b4610855c4ffa378517d9`. Rule list: `[]` (Enabled: `True`). {{(pid=81400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 03:21:51.700174 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-b78e1126-b7db-465b-8271-7b48776257b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:21:51.708655 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 972/1947] 10.222.0.18 () {58 vars in 958 bytes} [Tue Sep 13 03:21:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 03:21:51.712890 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5e60c76-f877-4677-bbf5-02801dbfb3d4 None None] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:21:51.723450 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5e60c76-f877-4677-bbf5-02801dbfb3d4 None None] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:21:51.728121 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-030cbf14-5d2a-453d-8aba-dc8ab0db82ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76c5181fc7be45279ac00017bb4640b3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:21:51.728732 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-030cbf14-5d2a-453d-8aba-dc8ab0db82ab admin admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:21:51.729015 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-030cbf14-5d2a-453d-8aba-dc8ab0db82ab admin admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:21:51.729289 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-030cbf14-5d2a-453d-8aba-dc8ab0db82ab admin admin] PATH_INFO: `/v3/domains` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:21:51.729935 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-030cbf14-5d2a-453d-8aba-dc8ab0db82ab admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 03:21:51.730241 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-030cbf14-5d2a-453d-8aba-dc8ab0db82ab admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:21:51.731874 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-030cbf14-5d2a-453d-8aba-dc8ab0db82ab admin admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:21:51.743554 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 976/1948] 10.222.0.18 () {58 vars in 1153 bytes} [Tue Sep 13 03:21:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 03:21:51.747852 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b78e1126-b7db-465b-8271-7b48776257b3 None None] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:21:51.757009 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b78e1126-b7db-465b-8271-7b48776257b3 None None] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:21:51.761640 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbf7813e-7176-4363-bf5b-a71173c14ca7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76c5181fc7be45279ac00017bb4640b3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:21:51.762237 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbf7813e-7176-4363-bf5b-a71173c14ca7 admin admin] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:21:51.762513 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbf7813e-7176-4363-bf5b-a71173c14ca7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:21:51.762816 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbf7813e-7176-4363-bf5b-a71173c14ca7 admin admin] PATH_INFO: `/v3/projects` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:21:51.763442 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbf7813e-7176-4363-bf5b-a71173c14ca7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:21:51.765344 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbf7813e-7176-4363-bf5b-a71173c14ca7 admin admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:21:51.799822 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 973/1949] 10.222.0.18 () {60 vars in 1153 bytes} [Tue Sep 13 03:21:51 2022] POST /identity/v3/projects => generated 375 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 03:21:51.801392 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-030cbf14-5d2a-453d-8aba-dc8ab0db82ab admin admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:21:51.808180 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:21:51.812701 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-030cbf14-5d2a-453d-8aba-dc8ab0db82ab admin admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:21:51.873183 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-036649a6-cdf6-40b1-8587-1b33525ab387 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76c5181fc7be45279ac00017bb4640b3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:21:51.873876 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-036649a6-cdf6-40b1-8587-1b33525ab387 admin admin] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:21:51.874063 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-036649a6-cdf6-40b1-8587-1b33525ab387 admin admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:21:51.874337 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-036649a6-cdf6-40b1-8587-1b33525ab387 admin admin] PATH_INFO: `/v3/users` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:21:51.875006 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-036649a6-cdf6-40b1-8587-1b33525ab387 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:21:51.876955 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-036649a6-cdf6-40b1-8587-1b33525ab387 admin admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:21:51.901987 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 977/1950] 10.222.0.18 () {60 vars in 1144 bytes} [Tue Sep 13 03:21:51 2022] POST /identity/v3/users => generated 288 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 03:21:51.903331 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbf7813e-7176-4363-bf5b-a71173c14ca7 admin admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:21:51.914190 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbf7813e-7176-4363-bf5b-a71173c14ca7 admin admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:21:51.919086 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-812e47a7-fd09-469d-94c4-bc1f3e9dfd26 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76c5181fc7be45279ac00017bb4640b3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:21:51.919951 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-812e47a7-fd09-469d-94c4-bc1f3e9dfd26 admin admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:21:51.920259 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-812e47a7-fd09-469d-94c4-bc1f3e9dfd26 admin admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:21:51.920530 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-812e47a7-fd09-469d-94c4-bc1f3e9dfd26 admin admin] PATH_INFO: `/v3/roles` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:21:51.921166 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-812e47a7-fd09-469d-94c4-bc1f3e9dfd26 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:21:51.922807 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-812e47a7-fd09-469d-94c4-bc1f3e9dfd26 admin admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:21:51.933501 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 974/1951] 10.222.0.18 () {58 vars in 1122 bytes} [Tue Sep 13 03:21:51 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 03:21:51.937265 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-036649a6-cdf6-40b1-8587-1b33525ab387 admin admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:21:51.948571 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-036649a6-cdf6-40b1-8587-1b33525ab387 admin admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:21:51.953232 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c7bdad1-a561-4478-bd4a-749edfd11eda admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76c5181fc7be45279ac00017bb4640b3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:21:51.953841 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c7bdad1-a561-4478-bd4a-749edfd11eda admin admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:21:51.954124 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c7bdad1-a561-4478-bd4a-749edfd11eda admin admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:21:51.954400 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c7bdad1-a561-4478-bd4a-749edfd11eda admin admin] PATH_INFO: `/v3/roles` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:21:51.955072 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c7bdad1-a561-4478-bd4a-749edfd11eda admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:21:51.957439 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c7bdad1-a561-4478-bd4a-749edfd11eda admin admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:21:51.969408 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 978/1952] 10.222.0.18 () {58 vars in 1122 bytes} [Tue Sep 13 03:21:51 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 03:21:51.973461 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-812e47a7-fd09-469d-94c4-bc1f3e9dfd26 admin admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:21:51.985049 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-812e47a7-fd09-469d-94c4-bc1f3e9dfd26 admin admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:21:51.990191 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59c4b6b6-43b7-40d6-a4b4-355123137470 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76c5181fc7be45279ac00017bb4640b3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:21:51.990876 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59c4b6b6-43b7-40d6-a4b4-355123137470 admin admin] REQUEST_METHOD: `PUT` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:21:51.991158 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59c4b6b6-43b7-40d6-a4b4-355123137470 admin admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:21:51.991431 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59c4b6b6-43b7-40d6-a4b4-355123137470 admin admin] PATH_INFO: `/v3/projects/0468f6132b7c4087b2bc4ccae9e4f76a/users/3aa96fc2af374eeaa8a93085b35b1042/roles/1eb7a5b5203945c29e6d22d653f47847` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:21:52.006579 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59c4b6b6-43b7-40d6-a4b4-355123137470 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0468f6132b7c4087b2bc4ccae9e4f76a, user_id=3aa96fc2af374eeaa8a93085b35b1042, role_id=1eb7a5b5203945c29e6d22d653f47847)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:21:52.009286 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59c4b6b6-43b7-40d6-a4b4-355123137470 admin admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:21:52.022428 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 975/1953] 10.222.0.18 () {60 vars in 1483 bytes} [Tue Sep 13 03:21:51 2022] PUT /identity/v3/projects/0468f6132b7c4087b2bc4ccae9e4f76a/users/3aa96fc2af374eeaa8a93085b35b1042/roles/1eb7a5b5203945c29e6d22d653f47847 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 03:21:52.032654 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8200031f-e4ea-44b4-bc48-d87e45c743ff None None] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:21:52.032946 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8200031f-e4ea-44b4-bc48-d87e45c743ff None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:21:52.033248 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8200031f-e4ea-44b4-bc48-d87e45c743ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:21:52.062724 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.auth.core [None req-8200031f-e4ea-44b4-bc48-d87e45c743ff None None] MFA Rules not processed for user `3aa96fc2af374eeaa8a93085b35b1042`. Rule list: `[]` (Enabled: `True`). {{(pid=81399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 03:21:52.063959 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-8200031f-e4ea-44b4-bc48-d87e45c743ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:21:52.133757 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 979/1954] 10.222.0.18 () {58 vars in 958 bytes} [Tue Sep 13 03:21:52 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 03:21:52.144060 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59c4b6b6-43b7-40d6-a4b4-355123137470 admin admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:21:52.146428 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-59c4b6b6-43b7-40d6-a4b4-355123137470 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:21:52.156091 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8200031f-e4ea-44b4-bc48-d87e45c743ff None None] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:21:52.157469 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-8200031f-e4ea-44b4-bc48-d87e45c743ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:21:52.218098 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59c4b6b6-43b7-40d6-a4b4-355123137470 admin admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:21:52.228032 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-846c8d5c-3d76-4f92-9512-fce01a3ea3f7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '924ee3f608d34a18b0ddc4695f102a56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b9d94f6a68f7470d9494f112175620dc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:21:52.228652 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-846c8d5c-3d76-4f92-9512-fce01a3ea3f7 service cinder] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:21:52.228926 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-846c8d5c-3d76-4f92-9512-fce01a3ea3f7 service cinder] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:21:52.229209 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-846c8d5c-3d76-4f92-9512-fce01a3ea3f7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:21:52.229477 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8200031f-e4ea-44b4-bc48-d87e45c743ff None None] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:21:52.234321 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55398576-d41d-417a-b1c5-d9e1ca8f9776 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76c5181fc7be45279ac00017bb4640b3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:21:52.235102 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55398576-d41d-417a-b1c5-d9e1ca8f9776 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:21:52.235384 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55398576-d41d-417a-b1c5-d9e1ca8f9776 admin admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:21:52.235689 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55398576-d41d-417a-b1c5-d9e1ca8f9776 admin admin] PATH_INFO: `/v3/users/3dd4fd8c6c454308987f4e98a901b51a` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:21:52.236905 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55398576-d41d-417a-b1c5-d9e1ca8f9776 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3dd4fd8c6c454308987f4e98a901b51a)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:21:52.238355 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-846c8d5c-3d76-4f92-9512-fce01a3ea3f7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:21:52.238836 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55398576-d41d-417a-b1c5-d9e1ca8f9776 admin admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:21:52.239951 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-846c8d5c-3d76-4f92-9512-fce01a3ea3f7 service cinder] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:21:52.249565 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 976/1955] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Sep 13 03:21:52 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 03:21:52.253031 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=96,keystone:INSERT=3,keystone:DELETE=1 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:21:52.289996 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-55398576-d41d-417a-b1c5-d9e1ca8f9776 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3dd4fd8c6c454308987f4e98a901b51a'} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:21:52.299017 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-55398576-d41d-417a-b1c5-d9e1ca8f9776 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3dd4fd8c6c454308987f4e98a901b51a'} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:21:52.318475 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-55398576-d41d-417a-b1c5-d9e1ca8f9776 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3dd4fd8c6c454308987f4e98a901b51a'} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:21:52.319190 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-55398576-d41d-417a-b1c5-d9e1ca8f9776 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3dd4fd8c6c454308987f4e98a901b51a'} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:21:52.327148 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-55398576-d41d-417a-b1c5-d9e1ca8f9776 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3dd4fd8c6c454308987f4e98a901b51a'} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:21:52.340466 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 980/1956] 10.222.0.18 () {58 vars in 1224 bytes} [Tue Sep 13 03:21:52 2022] DELETE /identity/v3/users/3dd4fd8c6c454308987f4e98a901b51a => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 13 03:21:52.341099 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5368d40-2dce-4b5b-8029-d353babb4fdf None None] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:21:52.341386 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5368d40-2dce-4b5b-8029-d353babb4fdf None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:21:52.341669 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5368d40-2dce-4b5b-8029-d353babb4fdf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:21:52.362023 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.auth.core [None req-a5368d40-2dce-4b5b-8029-d353babb4fdf None None] MFA Rules not processed for user `49f280ab086b4610855c4ffa378517d9`. Rule list: `[]` (Enabled: `True`). {{(pid=81400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 03:21:52.363803 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-a5368d40-2dce-4b5b-8029-d353babb4fdf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:21:52.431129 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=92,keystone:DELETE=5,keystone:INSERT=3 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:21:52.447390 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 977/1957] 10.222.0.18 () {58 vars in 958 bytes} [Tue Sep 13 03:21:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 03:21:52.454858 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55398576-d41d-417a-b1c5-d9e1ca8f9776 admin admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:21:52.456505 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-55398576-d41d-417a-b1c5-d9e1ca8f9776 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:21:52.548024 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55398576-d41d-417a-b1c5-d9e1ca8f9776 admin admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:21:52.554708 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0de6fffd-e88e-4414-af55-a07c2bd9471d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdffbc851086441ebd01022d1e4c2609', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b9d94f6a68f7470d9494f112175620dc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:21:52.555792 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de6fffd-e88e-4414-af55-a07c2bd9471d service neutron] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:21:52.556123 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de6fffd-e88e-4414-af55-a07c2bd9471d service neutron] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:21:52.556487 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de6fffd-e88e-4414-af55-a07c2bd9471d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:21:52.569086 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0de6fffd-e88e-4414-af55-a07c2bd9471d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:21:52.570732 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0de6fffd-e88e-4414-af55-a07c2bd9471d service neutron] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:21:52.581791 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 981/1958] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Sep 13 03:21:52 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 128 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 03:21:54.312425 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5368d40-2dce-4b5b-8029-d353babb4fdf None None] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:21:54.313848 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-a5368d40-2dce-4b5b-8029-d353babb4fdf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:21:54.327026 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5368d40-2dce-4b5b-8029-d353babb4fdf None None] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:21:54.332611 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0560bbea-8b3c-4c84-9c87-e1d0d4ba72ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76c5181fc7be45279ac00017bb4640b3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:21:54.333341 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0560bbea-8b3c-4c84-9c87-e1d0d4ba72ae admin admin] REQUEST_METHOD: `DELETE` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:21:54.333803 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0560bbea-8b3c-4c84-9c87-e1d0d4ba72ae admin admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:21:54.334092 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0560bbea-8b3c-4c84-9c87-e1d0d4ba72ae admin admin] PATH_INFO: `/v3/projects/32595c502ba648b79b4c48ba7017b192` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:21:54.335371 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0560bbea-8b3c-4c84-9c87-e1d0d4ba72ae admin admin] RBAC: Authorizing `identity:delete_project(project_id=32595c502ba648b79b4c48ba7017b192)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:21:54.337419 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0560bbea-8b3c-4c84-9c87-e1d0d4ba72ae admin admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:21:54.447606 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-0560bbea-8b3c-4c84-9c87-e1d0d4ba72ae admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '32595c502ba648b79b4c48ba7017b192', 'request_id': 'req-0560bbea-8b3c-4c84-9c87-e1d0d4ba72ae'} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:21:54.454051 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-0560bbea-8b3c-4c84-9c87-e1d0d4ba72ae admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '32595c502ba648b79b4c48ba7017b192', 'request_id': 'req-0560bbea-8b3c-4c84-9c87-e1d0d4ba72ae'} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:21:54.461255 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-0560bbea-8b3c-4c84-9c87-e1d0d4ba72ae admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '32595c502ba648b79b4c48ba7017b192', 'request_id': 'req-0560bbea-8b3c-4c84-9c87-e1d0d4ba72ae'} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:21:54.475930 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-0560bbea-8b3c-4c84-9c87-e1d0d4ba72ae admin admin] The token cache is being invalidate because project 32595c502ba648b79b4c48ba7017b192 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81400) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 03:21:54.476442 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-0560bbea-8b3c-4c84-9c87-e1d0d4ba72ae admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:21:54.477279 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-0560bbea-8b3c-4c84-9c87-e1d0d4ba72ae admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:21:54.479480 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 978/1959] 10.222.0.18 () {58 vars in 1233 bytes} [Tue Sep 13 03:21:54 2022] DELETE /identity/v3/projects/32595c502ba648b79b4c48ba7017b192 => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 03:21:54.483405 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0de6fffd-e88e-4414-af55-a07c2bd9471d service neutron] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:21:54.494437 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0de6fffd-e88e-4414-af55-a07c2bd9471d service neutron] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:21:54.564032 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c57d72a4-d75b-4508-9616-f0ae39d03178 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76c5181fc7be45279ac00017bb4640b3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:21:54.566116 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c57d72a4-d75b-4508-9616-f0ae39d03178 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:21:54.566509 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c57d72a4-d75b-4508-9616-f0ae39d03178 admin admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:21:54.566940 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c57d72a4-d75b-4508-9616-f0ae39d03178 admin admin] PATH_INFO: `/v3/users/438cfe0d626b49198cc06b1153ab3596` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:21:54.569990 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c57d72a4-d75b-4508-9616-f0ae39d03178 admin admin] RBAC: Authorizing `identity:delete_user(user_id=438cfe0d626b49198cc06b1153ab3596)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:21:54.573325 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c57d72a4-d75b-4508-9616-f0ae39d03178 admin admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:21:54.645507 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:21:54.655706 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-c57d72a4-d75b-4508-9616-f0ae39d03178 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '438cfe0d626b49198cc06b1153ab3596'} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:21:54.670487 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-c57d72a4-d75b-4508-9616-f0ae39d03178 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '438cfe0d626b49198cc06b1153ab3596'} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:21:54.685709 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-c57d72a4-d75b-4508-9616-f0ae39d03178 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '438cfe0d626b49198cc06b1153ab3596'} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:21:54.686477 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-c57d72a4-d75b-4508-9616-f0ae39d03178 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '438cfe0d626b49198cc06b1153ab3596'} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:21:54.695101 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-c57d72a4-d75b-4508-9616-f0ae39d03178 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '438cfe0d626b49198cc06b1153ab3596'} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:21:54.701546 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 982/1960] 10.222.0.18 () {58 vars in 1224 bytes} [Tue Sep 13 03:21:54 2022] DELETE /identity/v3/users/438cfe0d626b49198cc06b1153ab3596 => generated 0 bytes in 219 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 03:21:55.058258 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0560bbea-8b3c-4c84-9c87-e1d0d4ba72ae admin admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:21:55.067595 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-0560bbea-8b3c-4c84-9c87-e1d0d4ba72ae admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:21:55.088682 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0560bbea-8b3c-4c84-9c87-e1d0d4ba72ae admin admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:21:55.145747 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04b2c744-8d01-4846-b74d-7cf84c4e473a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76c5181fc7be45279ac00017bb4640b3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:21:55.146440 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04b2c744-8d01-4846-b74d-7cf84c4e473a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:21:55.146753 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04b2c744-8d01-4846-b74d-7cf84c4e473a admin admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:21:55.147087 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04b2c744-8d01-4846-b74d-7cf84c4e473a admin admin] PATH_INFO: `/v3/projects/ef3ba9c5d0e04081a26f45f72d834c60` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:21:55.148262 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04b2c744-8d01-4846-b74d-7cf84c4e473a admin admin] RBAC: Authorizing `identity:delete_project(project_id=ef3ba9c5d0e04081a26f45f72d834c60)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:21:55.150202 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04b2c744-8d01-4846-b74d-7cf84c4e473a admin admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:21:55.195917 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-04b2c744-8d01-4846-b74d-7cf84c4e473a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ef3ba9c5d0e04081a26f45f72d834c60', 'request_id': 'req-04b2c744-8d01-4846-b74d-7cf84c4e473a'} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:21:55.201469 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-04b2c744-8d01-4846-b74d-7cf84c4e473a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ef3ba9c5d0e04081a26f45f72d834c60', 'request_id': 'req-04b2c744-8d01-4846-b74d-7cf84c4e473a'} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:21:55.208132 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-04b2c744-8d01-4846-b74d-7cf84c4e473a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ef3ba9c5d0e04081a26f45f72d834c60', 'request_id': 'req-04b2c744-8d01-4846-b74d-7cf84c4e473a'} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:21:55.220328 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-04b2c744-8d01-4846-b74d-7cf84c4e473a admin admin] The token cache is being invalidate because project ef3ba9c5d0e04081a26f45f72d834c60 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81400) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 03:21:55.220612 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-04b2c744-8d01-4846-b74d-7cf84c4e473a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:21:55.221222 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-04b2c744-8d01-4846-b74d-7cf84c4e473a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:21:55.222716 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 979/1961] 10.222.0.18 () {58 vars in 1233 bytes} [Tue Sep 13 03:21:55 2022] DELETE /identity/v3/projects/ef3ba9c5d0e04081a26f45f72d834c60 => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 03:21:55.227614 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e93851-b6ea-4a92-a0eb-30e79cd19d32 None None] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:21:55.227925 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e93851-b6ea-4a92-a0eb-30e79cd19d32 None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:21:55.228205 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e93851-b6ea-4a92-a0eb-30e79cd19d32 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:21:55.247480 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.auth.core [None req-83e93851-b6ea-4a92-a0eb-30e79cd19d32 None None] MFA Rules not processed for user `49f280ab086b4610855c4ffa378517d9`. Rule list: `[]` (Enabled: `True`). {{(pid=81399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 03:21:55.248748 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-83e93851-b6ea-4a92-a0eb-30e79cd19d32 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:21:55.307973 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 983/1962] 10.222.0.18 () {58 vars in 958 bytes} [Tue Sep 13 03:21:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 03:21:55.315701 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7efa4608-0318-43c2-aa31-6eb6d26841e4 None None] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:21:55.316671 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7efa4608-0318-43c2-aa31-6eb6d26841e4 None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:21:55.317020 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7efa4608-0318-43c2-aa31-6eb6d26841e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:21:55.337809 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.auth.core [None req-7efa4608-0318-43c2-aa31-6eb6d26841e4 None None] MFA Rules not processed for user `49f280ab086b4610855c4ffa378517d9`. Rule list: `[]` (Enabled: `True`). {{(pid=81400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 03:21:55.339023 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-7efa4608-0318-43c2-aa31-6eb6d26841e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:21:55.346968 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 980/1963] 10.222.0.18 () {58 vars in 958 bytes} [Tue Sep 13 03:21:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 03:21:55.351114 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83e93851-b6ea-4a92-a0eb-30e79cd19d32 None None] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:21:55.361209 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83e93851-b6ea-4a92-a0eb-30e79cd19d32 None None] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:21:55.366338 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-229840ea-72cb-4159-ac85-1a09aaf8c8fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76c5181fc7be45279ac00017bb4640b3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:21:55.366966 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-229840ea-72cb-4159-ac85-1a09aaf8c8fb admin admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:21:55.367249 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-229840ea-72cb-4159-ac85-1a09aaf8c8fb admin admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:21:55.367607 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-229840ea-72cb-4159-ac85-1a09aaf8c8fb admin admin] PATH_INFO: `/v3/domains` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:21:55.368291 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-229840ea-72cb-4159-ac85-1a09aaf8c8fb admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 03:21:55.368598 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-229840ea-72cb-4159-ac85-1a09aaf8c8fb admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:21:55.370177 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-229840ea-72cb-4159-ac85-1a09aaf8c8fb admin admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:21:55.385777 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 984/1964] 10.222.0.18 () {58 vars in 1153 bytes} [Tue Sep 13 03:21:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 03:21:55.386886 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7efa4608-0318-43c2-aa31-6eb6d26841e4 None None] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:21:55.396355 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7efa4608-0318-43c2-aa31-6eb6d26841e4 None None] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:21:55.401304 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b09e27c-8e8e-47f9-8e7a-bbc1a4294d01 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76c5181fc7be45279ac00017bb4640b3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:21:55.402139 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b09e27c-8e8e-47f9-8e7a-bbc1a4294d01 admin admin] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:21:55.402421 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b09e27c-8e8e-47f9-8e7a-bbc1a4294d01 admin admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:21:55.402731 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b09e27c-8e8e-47f9-8e7a-bbc1a4294d01 admin admin] PATH_INFO: `/v3/projects` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:21:55.403371 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b09e27c-8e8e-47f9-8e7a-bbc1a4294d01 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:21:55.405398 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b09e27c-8e8e-47f9-8e7a-bbc1a4294d01 admin admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:21:55.410406 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:21:55.446811 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 981/1965] 10.222.0.18 () {60 vars in 1153 bytes} [Tue Sep 13 03:21:55 2022] POST /identity/v3/projects => generated 387 bytes in 60 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 03:21:55.447396 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-229840ea-72cb-4159-ac85-1a09aaf8c8fb admin admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:21:55.458078 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-229840ea-72cb-4159-ac85-1a09aaf8c8fb admin admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:21:55.529970 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b809005f-257f-4b5a-840f-40914e54afa6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76c5181fc7be45279ac00017bb4640b3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:21:55.530775 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b809005f-257f-4b5a-840f-40914e54afa6 admin admin] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:21:55.531163 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b809005f-257f-4b5a-840f-40914e54afa6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:21:55.531515 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b809005f-257f-4b5a-840f-40914e54afa6 admin admin] PATH_INFO: `/v3/users` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:21:55.532239 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b809005f-257f-4b5a-840f-40914e54afa6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:21:55.536229 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b809005f-257f-4b5a-840f-40914e54afa6 admin admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:21:55.561296 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 985/1966] 10.222.0.18 () {60 vars in 1144 bytes} [Tue Sep 13 03:21:55 2022] POST /identity/v3/users => generated 294 bytes in 114 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 03:21:55.565395 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b09e27c-8e8e-47f9-8e7a-bbc1a4294d01 admin admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:21:55.576520 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b09e27c-8e8e-47f9-8e7a-bbc1a4294d01 admin admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:21:55.583468 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bea7717e-191f-4df3-a0b5-555e2d8a5d01 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76c5181fc7be45279ac00017bb4640b3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:21:55.584409 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bea7717e-191f-4df3-a0b5-555e2d8a5d01 admin admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:21:55.585107 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bea7717e-191f-4df3-a0b5-555e2d8a5d01 admin admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:21:55.585548 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bea7717e-191f-4df3-a0b5-555e2d8a5d01 admin admin] PATH_INFO: `/v3/roles` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:21:55.586600 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bea7717e-191f-4df3-a0b5-555e2d8a5d01 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:21:55.589340 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bea7717e-191f-4df3-a0b5-555e2d8a5d01 admin admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:21:55.632134 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 982/1967] 10.222.0.18 () {58 vars in 1122 bytes} [Tue Sep 13 03:21:55 2022] GET /identity/v3/roles => generated 1420 bytes in 67 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 03:21:55.632990 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b809005f-257f-4b5a-840f-40914e54afa6 admin admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:21:55.656159 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b809005f-257f-4b5a-840f-40914e54afa6 admin admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:21:55.680555 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48fd253c-e53e-4974-b83a-f71a14aa3988 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76c5181fc7be45279ac00017bb4640b3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:21:55.681386 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48fd253c-e53e-4974-b83a-f71a14aa3988 admin admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:21:55.681760 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48fd253c-e53e-4974-b83a-f71a14aa3988 admin admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:21:55.682179 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48fd253c-e53e-4974-b83a-f71a14aa3988 admin admin] PATH_INFO: `/v3/roles` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:21:55.683645 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48fd253c-e53e-4974-b83a-f71a14aa3988 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:21:55.688336 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48fd253c-e53e-4974-b83a-f71a14aa3988 admin admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:21:55.723240 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 986/1968] 10.222.0.18 () {58 vars in 1122 bytes} [Tue Sep 13 03:21:55 2022] GET /identity/v3/roles => generated 1420 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 03:21:55.725500 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bea7717e-191f-4df3-a0b5-555e2d8a5d01 admin admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:21:55.741735 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bea7717e-191f-4df3-a0b5-555e2d8a5d01 admin admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:21:55.756874 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb6160ac-cdc4-4d4d-abc7-c25169db78c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76c5181fc7be45279ac00017bb4640b3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:21:55.761715 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb6160ac-cdc4-4d4d-abc7-c25169db78c5 admin admin] REQUEST_METHOD: `PUT` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:21:55.762351 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb6160ac-cdc4-4d4d-abc7-c25169db78c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:21:55.763685 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb6160ac-cdc4-4d4d-abc7-c25169db78c5 admin admin] PATH_INFO: `/v3/projects/f4a3f5c6eb964c86b5851942d2edb461/users/d049b008841d442da26181985adee33f/roles/1eb7a5b5203945c29e6d22d653f47847` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:21:55.833554 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb6160ac-cdc4-4d4d-abc7-c25169db78c5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f4a3f5c6eb964c86b5851942d2edb461, user_id=d049b008841d442da26181985adee33f, role_id=1eb7a5b5203945c29e6d22d653f47847)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:21:55.839564 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb6160ac-cdc4-4d4d-abc7-c25169db78c5 admin admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:21:55.876944 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 983/1969] 10.222.0.18 () {60 vars in 1483 bytes} [Tue Sep 13 03:21:55 2022] PUT /identity/v3/projects/f4a3f5c6eb964c86b5851942d2edb461/users/d049b008841d442da26181985adee33f/roles/1eb7a5b5203945c29e6d22d653f47847 => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 03:21:55.888650 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbfc49ab-0327-49ee-a934-6a31e27fcb28 None None] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:21:55.888926 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbfc49ab-0327-49ee-a934-6a31e27fcb28 None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:21:55.889377 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbfc49ab-0327-49ee-a934-6a31e27fcb28 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:21:55.937047 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.auth.core [None req-fbfc49ab-0327-49ee-a934-6a31e27fcb28 None None] MFA Rules not processed for user `d049b008841d442da26181985adee33f`. Rule list: `[]` (Enabled: `True`). {{(pid=81399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 03:21:55.938709 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-fbfc49ab-0327-49ee-a934-6a31e27fcb28 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:21:55.948928 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=91,keystone:DELETE=5,keystone:INSERT=4 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:21:56.025697 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 987/1970] 10.222.0.18 () {58 vars in 958 bytes} [Tue Sep 13 03:21:55 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 13 03:21:56.030428 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb6160ac-cdc4-4d4d-abc7-c25169db78c5 admin admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:21:56.150712 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb6160ac-cdc4-4d4d-abc7-c25169db78c5 admin admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:21:56.156801 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adf51a7b-983a-4512-ae2a-f6a7ece8b611 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76c5181fc7be45279ac00017bb4640b3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:21:56.157530 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adf51a7b-983a-4512-ae2a-f6a7ece8b611 admin admin] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:21:56.157947 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adf51a7b-983a-4512-ae2a-f6a7ece8b611 admin admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:21:56.158282 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adf51a7b-983a-4512-ae2a-f6a7ece8b611 admin admin] PATH_INFO: `/v3/projects` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:21:56.159118 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adf51a7b-983a-4512-ae2a-f6a7ece8b611 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:21:56.162016 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adf51a7b-983a-4512-ae2a-f6a7ece8b611 admin admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:21:56.193069 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 984/1971] 10.222.0.18 () {60 vars in 1153 bytes} [Tue Sep 13 03:21:56 2022] POST /identity/v3/projects => generated 385 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 03:21:56.197353 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbfc49ab-0327-49ee-a934-6a31e27fcb28 None None] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:21:56.209439 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbfc49ab-0327-49ee-a934-6a31e27fcb28 None None] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:21:56.278021 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f7c2d25-89b2-4f0c-818e-12cb2a691359 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76c5181fc7be45279ac00017bb4640b3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:21:56.278652 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f7c2d25-89b2-4f0c-818e-12cb2a691359 admin admin] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:21:56.278931 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f7c2d25-89b2-4f0c-818e-12cb2a691359 admin admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:21:56.279198 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f7c2d25-89b2-4f0c-818e-12cb2a691359 admin admin] PATH_INFO: `/v3/users` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:21:56.279946 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f7c2d25-89b2-4f0c-818e-12cb2a691359 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:21:56.281737 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f7c2d25-89b2-4f0c-818e-12cb2a691359 admin admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:21:56.313529 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 988/1972] 10.222.0.18 () {60 vars in 1144 bytes} [Tue Sep 13 03:21:56 2022] POST /identity/v3/users => generated 293 bytes in 114 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 13 03:21:56.314789 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adf51a7b-983a-4512-ae2a-f6a7ece8b611 admin admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:21:56.325530 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adf51a7b-983a-4512-ae2a-f6a7ece8b611 admin admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:21:56.330520 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cc3d72a-09a1-4f1e-9737-c6989fb22857 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76c5181fc7be45279ac00017bb4640b3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:21:56.331246 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cc3d72a-09a1-4f1e-9737-c6989fb22857 admin admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:21:56.332145 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cc3d72a-09a1-4f1e-9737-c6989fb22857 admin admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:21:56.332613 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cc3d72a-09a1-4f1e-9737-c6989fb22857 admin admin] PATH_INFO: `/v3/roles` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:21:56.333614 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cc3d72a-09a1-4f1e-9737-c6989fb22857 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:21:56.335311 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cc3d72a-09a1-4f1e-9737-c6989fb22857 admin admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:21:56.347868 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 985/1973] 10.222.0.18 () {58 vars in 1122 bytes} [Tue Sep 13 03:21:56 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 03:21:56.352799 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f7c2d25-89b2-4f0c-818e-12cb2a691359 admin admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:21:56.363494 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f7c2d25-89b2-4f0c-818e-12cb2a691359 admin admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:21:56.369697 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f37bb90d-59a8-4e89-bb3f-e06b75e3ed7e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76c5181fc7be45279ac00017bb4640b3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:21:56.370395 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f37bb90d-59a8-4e89-bb3f-e06b75e3ed7e admin admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:21:56.370782 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f37bb90d-59a8-4e89-bb3f-e06b75e3ed7e admin admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:21:56.371129 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f37bb90d-59a8-4e89-bb3f-e06b75e3ed7e admin admin] PATH_INFO: `/v3/roles` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:21:56.371854 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f37bb90d-59a8-4e89-bb3f-e06b75e3ed7e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:21:56.373522 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f37bb90d-59a8-4e89-bb3f-e06b75e3ed7e admin admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:21:56.387161 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 989/1974] 10.222.0.18 () {58 vars in 1122 bytes} [Tue Sep 13 03:21:56 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 03:21:56.388813 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cc3d72a-09a1-4f1e-9737-c6989fb22857 admin admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:21:56.399960 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cc3d72a-09a1-4f1e-9737-c6989fb22857 admin admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:21:56.418918 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4150b18f-62cc-4df6-a5fe-401e424e3e89 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76c5181fc7be45279ac00017bb4640b3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:21:56.419926 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4150b18f-62cc-4df6-a5fe-401e424e3e89 admin admin] REQUEST_METHOD: `PUT` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:21:56.420274 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4150b18f-62cc-4df6-a5fe-401e424e3e89 admin admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:21:56.420613 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4150b18f-62cc-4df6-a5fe-401e424e3e89 admin admin] PATH_INFO: `/v3/projects/0ff834f08b5642d8a3fc9ef11f9d1d06/users/6ced7b4933344e2dae8d0cb2968cf60a/roles/1eb7a5b5203945c29e6d22d653f47847` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:21:56.442815 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4150b18f-62cc-4df6-a5fe-401e424e3e89 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0ff834f08b5642d8a3fc9ef11f9d1d06, user_id=6ced7b4933344e2dae8d0cb2968cf60a, role_id=1eb7a5b5203945c29e6d22d653f47847)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:21:56.442815 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4150b18f-62cc-4df6-a5fe-401e424e3e89 admin admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:21:56.458878 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 986/1975] 10.222.0.18 () {60 vars in 1483 bytes} [Tue Sep 13 03:21:56 2022] PUT /identity/v3/projects/0ff834f08b5642d8a3fc9ef11f9d1d06/users/6ced7b4933344e2dae8d0cb2968cf60a/roles/1eb7a5b5203945c29e6d22d653f47847 => generated 0 bytes in 71 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 03:21:56.469249 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f62a5a-3d87-44e1-84b6-1f45f655a398 None None] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:21:56.469538 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f62a5a-3d87-44e1-84b6-1f45f655a398 None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:21:56.469812 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f62a5a-3d87-44e1-84b6-1f45f655a398 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:21:56.497805 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.auth.core [None req-08f62a5a-3d87-44e1-84b6-1f45f655a398 None None] MFA Rules not processed for user `6ced7b4933344e2dae8d0cb2968cf60a`. Rule list: `[]` (Enabled: `True`). {{(pid=81399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 03:21:56.499096 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-08f62a5a-3d87-44e1-84b6-1f45f655a398 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:21:56.564064 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 990/1976] 10.222.0.18 () {58 vars in 958 bytes} [Tue Sep 13 03:21:56 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 03:21:56.569178 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4150b18f-62cc-4df6-a5fe-401e424e3e89 admin admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:21:56.629950 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4150b18f-62cc-4df6-a5fe-401e424e3e89 admin admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:21:56.634660 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35c6e299-835d-4d43-acee-ddff1224fed9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76c5181fc7be45279ac00017bb4640b3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:21:56.635190 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35c6e299-835d-4d43-acee-ddff1224fed9 admin admin] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:21:56.635458 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35c6e299-835d-4d43-acee-ddff1224fed9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:21:56.635739 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35c6e299-835d-4d43-acee-ddff1224fed9 admin admin] PATH_INFO: `/v3/projects` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:21:56.636370 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35c6e299-835d-4d43-acee-ddff1224fed9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:21:56.638283 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35c6e299-835d-4d43-acee-ddff1224fed9 admin admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:21:56.643118 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:21:56.686953 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 987/1977] 10.222.0.18 () {60 vars in 1153 bytes} [Tue Sep 13 03:21:56 2022] POST /identity/v3/projects => generated 387 bytes in 116 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 03:21:56.689072 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08f62a5a-3d87-44e1-84b6-1f45f655a398 None None] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:21:56.702636 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08f62a5a-3d87-44e1-84b6-1f45f655a398 None None] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:21:56.721567 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:21:56.776585 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c9eee4b-a961-4365-850f-82401dc4708c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76c5181fc7be45279ac00017bb4640b3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:21:56.777199 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c9eee4b-a961-4365-850f-82401dc4708c admin admin] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:21:56.777477 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c9eee4b-a961-4365-850f-82401dc4708c admin admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:21:56.777747 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c9eee4b-a961-4365-850f-82401dc4708c admin admin] PATH_INFO: `/v3/users` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:21:56.778412 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c9eee4b-a961-4365-850f-82401dc4708c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:21:56.781034 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c9eee4b-a961-4365-850f-82401dc4708c admin admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:21:56.811465 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35c6e299-835d-4d43-acee-ddff1224fed9 admin admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:21:56.811856 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 991/1978] 10.222.0.18 () {60 vars in 1144 bytes} [Tue Sep 13 03:21:56 2022] POST /identity/v3/users => generated 300 bytes in 122 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 03:21:56.823673 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35c6e299-835d-4d43-acee-ddff1224fed9 admin admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:21:56.829725 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6d3c65b-22fc-4618-b4fe-43f0b05bcc70 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76c5181fc7be45279ac00017bb4640b3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:21:56.830338 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d3c65b-22fc-4618-b4fe-43f0b05bcc70 admin admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:21:56.830763 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d3c65b-22fc-4618-b4fe-43f0b05bcc70 admin admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:21:56.831044 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d3c65b-22fc-4618-b4fe-43f0b05bcc70 admin admin] PATH_INFO: `/v3/roles` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:21:56.832239 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6d3c65b-22fc-4618-b4fe-43f0b05bcc70 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:21:56.833843 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6d3c65b-22fc-4618-b4fe-43f0b05bcc70 admin admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:21:56.850473 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 988/1979] 10.222.0.18 () {58 vars in 1122 bytes} [Tue Sep 13 03:21:56 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 13 03:21:56.851551 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c9eee4b-a961-4365-850f-82401dc4708c admin admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:21:56.861940 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c9eee4b-a961-4365-850f-82401dc4708c admin admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:21:56.867505 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd7a70fb-f361-4a5d-af47-3be7264d5b2e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76c5181fc7be45279ac00017bb4640b3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:21:56.868154 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd7a70fb-f361-4a5d-af47-3be7264d5b2e admin admin] REQUEST_METHOD: `PUT` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:21:56.868435 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd7a70fb-f361-4a5d-af47-3be7264d5b2e admin admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:21:56.868711 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd7a70fb-f361-4a5d-af47-3be7264d5b2e admin admin] PATH_INFO: `/v3/projects/fbf0d33f7f864bbb946ad6c7f404c212/users/f88ed4635011458bad80fa9c1cfed512/roles/425d4b481c7846e0bc07e7e805d5b15f` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:21:56.889624 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd7a70fb-f361-4a5d-af47-3be7264d5b2e admin admin] RBAC: Authorizing `identity:create_grant(project_id=fbf0d33f7f864bbb946ad6c7f404c212, user_id=f88ed4635011458bad80fa9c1cfed512, role_id=425d4b481c7846e0bc07e7e805d5b15f)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:21:56.892346 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd7a70fb-f361-4a5d-af47-3be7264d5b2e admin admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:21:56.914232 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 992/1980] 10.222.0.18 () {60 vars in 1483 bytes} [Tue Sep 13 03:21:56 2022] PUT /identity/v3/projects/fbf0d33f7f864bbb946ad6c7f404c212/users/f88ed4635011458bad80fa9c1cfed512/roles/425d4b481c7846e0bc07e7e805d5b15f => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 13 03:21:56.915528 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6d3c65b-22fc-4618-b4fe-43f0b05bcc70 admin admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:21:56.940973 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6d3c65b-22fc-4618-b4fe-43f0b05bcc70 admin admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:21:57.016175 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93e07a42-1d0a-46e2-bac1-6515ee9ffe64 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76c5181fc7be45279ac00017bb4640b3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:21:57.017024 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e07a42-1d0a-46e2-bac1-6515ee9ffe64 admin admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:21:57.017314 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e07a42-1d0a-46e2-bac1-6515ee9ffe64 admin admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:21:57.017609 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e07a42-1d0a-46e2-bac1-6515ee9ffe64 admin admin] PATH_INFO: `/v3/roles` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:21:57.018235 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93e07a42-1d0a-46e2-bac1-6515ee9ffe64 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:21:57.020003 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93e07a42-1d0a-46e2-bac1-6515ee9ffe64 admin admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:21:57.035011 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 989/1981] 10.222.0.18 () {58 vars in 1122 bytes} [Tue Sep 13 03:21:56 2022] GET /identity/v3/roles => generated 1420 bytes in 117 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 03:21:57.036233 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd7a70fb-f361-4a5d-af47-3be7264d5b2e admin admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:21:57.048321 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd7a70fb-f361-4a5d-af47-3be7264d5b2e admin admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:21:57.056220 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87e99ffa-32f6-4afd-a223-0c81f2aaa591 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76c5181fc7be45279ac00017bb4640b3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:21:57.057007 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e99ffa-32f6-4afd-a223-0c81f2aaa591 admin admin] REQUEST_METHOD: `PUT` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:21:57.057509 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e99ffa-32f6-4afd-a223-0c81f2aaa591 admin admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:21:57.057849 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e99ffa-32f6-4afd-a223-0c81f2aaa591 admin admin] PATH_INFO: `/v3/projects/fbf0d33f7f864bbb946ad6c7f404c212/users/f88ed4635011458bad80fa9c1cfed512/roles/425d4b481c7846e0bc07e7e805d5b15f` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:21:57.063710 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87e99ffa-32f6-4afd-a223-0c81f2aaa591 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fbf0d33f7f864bbb946ad6c7f404c212, user_id=f88ed4635011458bad80fa9c1cfed512, role_id=425d4b481c7846e0bc07e7e805d5b15f)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:21:57.070380 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87e99ffa-32f6-4afd-a223-0c81f2aaa591 admin admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:21:57.090037 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 993/1982] 10.222.0.18 () {60 vars in 1483 bytes} [Tue Sep 13 03:21:57 2022] PUT /identity/v3/projects/fbf0d33f7f864bbb946ad6c7f404c212/users/f88ed4635011458bad80fa9c1cfed512/roles/425d4b481c7846e0bc07e7e805d5b15f => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 03:21:57.105226 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d725e4-245e-4f43-8dce-929091a859b5 None None] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:21:57.105945 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d725e4-245e-4f43-8dce-929091a859b5 None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:21:57.106604 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d725e4-245e-4f43-8dce-929091a859b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:21:57.149280 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.auth.core [None req-35d725e4-245e-4f43-8dce-929091a859b5 None None] MFA Rules not processed for user `f88ed4635011458bad80fa9c1cfed512`. Rule list: `[]` (Enabled: `True`). {{(pid=81400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 03:21:57.150826 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-35d725e4-245e-4f43-8dce-929091a859b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:21:57.275694 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 990/1983] 10.222.0.18 () {58 vars in 958 bytes} [Tue Sep 13 03:21:57 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 03:21:57.304712 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87e99ffa-32f6-4afd-a223-0c81f2aaa591 admin admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:21:57.310657 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-87e99ffa-32f6-4afd-a223-0c81f2aaa591 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:21:57.384171 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87e99ffa-32f6-4afd-a223-0c81f2aaa591 admin admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:21:57.388275 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb9f7de3-14a3-43fe-ae74-a26c5f040561 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '924ee3f608d34a18b0ddc4695f102a56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b9d94f6a68f7470d9494f112175620dc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:21:57.388976 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb9f7de3-14a3-43fe-ae74-a26c5f040561 service cinder] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:21:57.389336 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb9f7de3-14a3-43fe-ae74-a26c5f040561 service cinder] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:21:57.389656 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb9f7de3-14a3-43fe-ae74-a26c5f040561 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:21:57.400031 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb9f7de3-14a3-43fe-ae74-a26c5f040561 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:21:57.401596 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb9f7de3-14a3-43fe-ae74-a26c5f040561 service cinder] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:21:57.413488 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 994/1984] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Sep 13 03:21:57 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 03:21:58.082499 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35d725e4-245e-4f43-8dce-929091a859b5 None None] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:21:58.093061 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35d725e4-245e-4f43-8dce-929091a859b5 None None] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:21:58.097902 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eac550c7-35d2-421d-898b-f385e48091c6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '924ee3f608d34a18b0ddc4695f102a56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b9d94f6a68f7470d9494f112175620dc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:21:58.098944 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eac550c7-35d2-421d-898b-f385e48091c6 service cinder] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:21:58.099380 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eac550c7-35d2-421d-898b-f385e48091c6 service cinder] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:21:58.099650 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eac550c7-35d2-421d-898b-f385e48091c6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:21:58.199946 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:21:58.202690 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eac550c7-35d2-421d-898b-f385e48091c6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:21:58.204572 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eac550c7-35d2-421d-898b-f385e48091c6 service cinder] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:21:58.217726 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 991/1985] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Sep 13 03:21:58 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 136 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 03:22:02.437344 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb9f7de3-14a3-43fe-ae74-a26c5f040561 service cinder] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:22:02.448180 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb9f7de3-14a3-43fe-ae74-a26c5f040561 service cinder] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:22:02.451136 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c1a328f-c644-4e38-9ae5-b12f5018ab46 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '924ee3f608d34a18b0ddc4695f102a56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b9d94f6a68f7470d9494f112175620dc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:22:02.451815 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c1a328f-c644-4e38-9ae5-b12f5018ab46 service cinder] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:22:02.452401 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c1a328f-c644-4e38-9ae5-b12f5018ab46 service cinder] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:22:02.452401 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c1a328f-c644-4e38-9ae5-b12f5018ab46 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:22:02.521541 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c1a328f-c644-4e38-9ae5-b12f5018ab46 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:22:02.523047 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c1a328f-c644-4e38-9ae5-b12f5018ab46 service cinder] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:22:02.532656 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 995/1986] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Sep 13 03:22:02 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 03:22:03.578350 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eac550c7-35d2-421d-898b-f385e48091c6 service cinder] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:22:03.579930 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-eac550c7-35d2-421d-898b-f385e48091c6 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:22:03.648630 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eac550c7-35d2-421d-898b-f385e48091c6 service cinder] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:22:03.653125 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5b53de8-f8a1-49af-9a33-13ab512d3415 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76c5181fc7be45279ac00017bb4640b3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:22:03.653750 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b53de8-f8a1-49af-9a33-13ab512d3415 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:22:03.654027 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b53de8-f8a1-49af-9a33-13ab512d3415 admin admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:22:03.654303 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b53de8-f8a1-49af-9a33-13ab512d3415 admin admin] PATH_INFO: `/v3/users/3aa96fc2af374eeaa8a93085b35b1042` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:22:03.655479 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5b53de8-f8a1-49af-9a33-13ab512d3415 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3aa96fc2af374eeaa8a93085b35b1042)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:22:03.657402 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5b53de8-f8a1-49af-9a33-13ab512d3415 admin admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:22:03.705079 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-f5b53de8-f8a1-49af-9a33-13ab512d3415 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3aa96fc2af374eeaa8a93085b35b1042'} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:22:03.713935 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-f5b53de8-f8a1-49af-9a33-13ab512d3415 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3aa96fc2af374eeaa8a93085b35b1042'} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:22:03.714629 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-f5b53de8-f8a1-49af-9a33-13ab512d3415 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3aa96fc2af374eeaa8a93085b35b1042'} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:22:03.734596 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-f5b53de8-f8a1-49af-9a33-13ab512d3415 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3aa96fc2af374eeaa8a93085b35b1042'} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:22:03.746507 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-f5b53de8-f8a1-49af-9a33-13ab512d3415 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3aa96fc2af374eeaa8a93085b35b1042'} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:22:03.752931 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 992/1987] 10.222.0.18 () {58 vars in 1224 bytes} [Tue Sep 13 03:22:03 2022] DELETE /identity/v3/users/3aa96fc2af374eeaa8a93085b35b1042 => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 13 03:22:03.757875 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8b862d9-12a6-4ba6-a921-24bd25b64a6d None None] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:22:03.758343 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8b862d9-12a6-4ba6-a921-24bd25b64a6d None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:22:03.758478 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8b862d9-12a6-4ba6-a921-24bd25b64a6d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:22:03.777702 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.auth.core [None req-b8b862d9-12a6-4ba6-a921-24bd25b64a6d None None] MFA Rules not processed for user `49f280ab086b4610855c4ffa378517d9`. Rule list: `[]` (Enabled: `True`). {{(pid=81399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 03:22:03.778922 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-b8b862d9-12a6-4ba6-a921-24bd25b64a6d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:22:03.784579 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:22:03.841609 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 996/1988] 10.222.0.18 () {58 vars in 958 bytes} [Tue Sep 13 03:22:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 03:22:03.851420 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5b53de8-f8a1-49af-9a33-13ab512d3415 admin admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:22:03.852823 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-f5b53de8-f8a1-49af-9a33-13ab512d3415 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:22:03.910416 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5b53de8-f8a1-49af-9a33-13ab512d3415 admin admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:22:03.914005 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f8ba064-b2f4-4a61-b0dd-3e31b61f605b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdffbc851086441ebd01022d1e4c2609', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b9d94f6a68f7470d9494f112175620dc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:22:03.914603 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f8ba064-b2f4-4a61-b0dd-3e31b61f605b service neutron] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:22:03.914941 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f8ba064-b2f4-4a61-b0dd-3e31b61f605b service neutron] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:22:03.915215 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f8ba064-b2f4-4a61-b0dd-3e31b61f605b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:22:03.924343 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f8ba064-b2f4-4a61-b0dd-3e31b61f605b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:22:03.926273 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f8ba064-b2f4-4a61-b0dd-3e31b61f605b service neutron] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:22:03.935629 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 993/1989] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Sep 13 03:22:03 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 13 03:22:04.212373 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8b862d9-12a6-4ba6-a921-24bd25b64a6d None None] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:22:04.214335 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-b8b862d9-12a6-4ba6-a921-24bd25b64a6d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:22:04.232870 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8b862d9-12a6-4ba6-a921-24bd25b64a6d None None] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:22:04.238269 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab2eea9f-92b5-4821-91e1-1dc3aa496f24 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76c5181fc7be45279ac00017bb4640b3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:22:04.239641 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab2eea9f-92b5-4821-91e1-1dc3aa496f24 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:22:04.239928 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab2eea9f-92b5-4821-91e1-1dc3aa496f24 admin admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:22:04.240200 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab2eea9f-92b5-4821-91e1-1dc3aa496f24 admin admin] PATH_INFO: `/v3/projects/0468f6132b7c4087b2bc4ccae9e4f76a` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:22:04.241418 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab2eea9f-92b5-4821-91e1-1dc3aa496f24 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0468f6132b7c4087b2bc4ccae9e4f76a)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:22:04.243429 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab2eea9f-92b5-4821-91e1-1dc3aa496f24 admin admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:22:04.344910 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-ab2eea9f-92b5-4821-91e1-1dc3aa496f24 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0468f6132b7c4087b2bc4ccae9e4f76a', 'request_id': 'req-ab2eea9f-92b5-4821-91e1-1dc3aa496f24'} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:22:04.351391 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-ab2eea9f-92b5-4821-91e1-1dc3aa496f24 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0468f6132b7c4087b2bc4ccae9e4f76a', 'request_id': 'req-ab2eea9f-92b5-4821-91e1-1dc3aa496f24'} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:22:04.358043 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-ab2eea9f-92b5-4821-91e1-1dc3aa496f24 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0468f6132b7c4087b2bc4ccae9e4f76a', 'request_id': 'req-ab2eea9f-92b5-4821-91e1-1dc3aa496f24'} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:22:04.373086 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-ab2eea9f-92b5-4821-91e1-1dc3aa496f24 admin admin] The token cache is being invalidate because project 0468f6132b7c4087b2bc4ccae9e4f76a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81399) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 03:22:04.373380 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-ab2eea9f-92b5-4821-91e1-1dc3aa496f24 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:22:04.374019 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-ab2eea9f-92b5-4821-91e1-1dc3aa496f24 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:22:04.375529 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 997/1990] 10.222.0.18 () {58 vars in 1233 bytes} [Tue Sep 13 03:22:04 2022] DELETE /identity/v3/projects/0468f6132b7c4087b2bc4ccae9e4f76a => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 03:22:04.725931 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f8ba064-b2f4-4a61-b0dd-3e31b61f605b service neutron] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:22:04.727348 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-3f8ba064-b2f4-4a61-b0dd-3e31b61f605b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:22:04.739976 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f8ba064-b2f4-4a61-b0dd-3e31b61f605b service neutron] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:22:04.751769 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:22:04.812649 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a934a6d-0f57-4de0-9b1f-de53dda9712c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76c5181fc7be45279ac00017bb4640b3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:22:04.813301 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a934a6d-0f57-4de0-9b1f-de53dda9712c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:22:04.813574 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a934a6d-0f57-4de0-9b1f-de53dda9712c admin admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:22:04.813853 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a934a6d-0f57-4de0-9b1f-de53dda9712c admin admin] PATH_INFO: `/v3/users/d049b008841d442da26181985adee33f` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:22:04.815074 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a934a6d-0f57-4de0-9b1f-de53dda9712c admin admin] RBAC: Authorizing `identity:delete_user(user_id=d049b008841d442da26181985adee33f)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:22:04.817541 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a934a6d-0f57-4de0-9b1f-de53dda9712c admin admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:22:04.863481 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-4a934a6d-0f57-4de0-9b1f-de53dda9712c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd049b008841d442da26181985adee33f'} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:22:04.871114 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-4a934a6d-0f57-4de0-9b1f-de53dda9712c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd049b008841d442da26181985adee33f'} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:22:04.871720 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-4a934a6d-0f57-4de0-9b1f-de53dda9712c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd049b008841d442da26181985adee33f'} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:22:04.884728 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-4a934a6d-0f57-4de0-9b1f-de53dda9712c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd049b008841d442da26181985adee33f'} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:22:04.891658 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-4a934a6d-0f57-4de0-9b1f-de53dda9712c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd049b008841d442da26181985adee33f'} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:22:04.899675 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 994/1991] 10.222.0.18 () {58 vars in 1224 bytes} [Tue Sep 13 03:22:04 2022] DELETE /identity/v3/users/d049b008841d442da26181985adee33f => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 03:22:04.903993 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d616f900-da48-43e3-878e-06372925439a None None] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:22:04.904460 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d616f900-da48-43e3-878e-06372925439a None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:22:04.904842 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d616f900-da48-43e3-878e-06372925439a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:22:04.928564 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.auth.core [None req-d616f900-da48-43e3-878e-06372925439a None None] MFA Rules not processed for user `49f280ab086b4610855c4ffa378517d9`. Rule list: `[]` (Enabled: `True`). {{(pid=81399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 03:22:04.930090 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-d616f900-da48-43e3-878e-06372925439a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:22:05.015766 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 998/1992] 10.222.0.18 () {58 vars in 958 bytes} [Tue Sep 13 03:22:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 03:22:05.025287 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a934a6d-0f57-4de0-9b1f-de53dda9712c admin admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:22:05.026861 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-4a934a6d-0f57-4de0-9b1f-de53dda9712c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:22:05.095223 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a934a6d-0f57-4de0-9b1f-de53dda9712c admin admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:22:05.099779 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df3fe4c1-7c79-4d3c-a9c7-82fd38589acd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdffbc851086441ebd01022d1e4c2609', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b9d94f6a68f7470d9494f112175620dc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:22:05.100387 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df3fe4c1-7c79-4d3c-a9c7-82fd38589acd service neutron] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:22:05.100660 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df3fe4c1-7c79-4d3c-a9c7-82fd38589acd service neutron] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:22:05.100925 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df3fe4c1-7c79-4d3c-a9c7-82fd38589acd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:22:05.110984 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df3fe4c1-7c79-4d3c-a9c7-82fd38589acd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:22:05.112469 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df3fe4c1-7c79-4d3c-a9c7-82fd38589acd service neutron] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:22:05.123231 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 995/1993] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Sep 13 03:22:05 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 03:22:05.387950 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d616f900-da48-43e3-878e-06372925439a None None] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:22:05.389361 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-d616f900-da48-43e3-878e-06372925439a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:22:05.402298 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d616f900-da48-43e3-878e-06372925439a None None] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:22:05.407000 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60545ce4-a4ef-4632-8d6c-85dd8ecd60cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76c5181fc7be45279ac00017bb4640b3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:22:05.407629 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60545ce4-a4ef-4632-8d6c-85dd8ecd60cc admin admin] REQUEST_METHOD: `DELETE` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:22:05.407901 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60545ce4-a4ef-4632-8d6c-85dd8ecd60cc admin admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:22:05.408169 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60545ce4-a4ef-4632-8d6c-85dd8ecd60cc admin admin] PATH_INFO: `/v3/projects/f4a3f5c6eb964c86b5851942d2edb461` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:22:05.409443 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60545ce4-a4ef-4632-8d6c-85dd8ecd60cc admin admin] RBAC: Authorizing `identity:delete_project(project_id=f4a3f5c6eb964c86b5851942d2edb461)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:22:05.411944 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60545ce4-a4ef-4632-8d6c-85dd8ecd60cc admin admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:22:05.452614 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:22:05.469103 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-60545ce4-a4ef-4632-8d6c-85dd8ecd60cc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f4a3f5c6eb964c86b5851942d2edb461', 'request_id': 'req-60545ce4-a4ef-4632-8d6c-85dd8ecd60cc'} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:22:05.475783 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-60545ce4-a4ef-4632-8d6c-85dd8ecd60cc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f4a3f5c6eb964c86b5851942d2edb461', 'request_id': 'req-60545ce4-a4ef-4632-8d6c-85dd8ecd60cc'} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:22:05.482991 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-60545ce4-a4ef-4632-8d6c-85dd8ecd60cc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f4a3f5c6eb964c86b5851942d2edb461', 'request_id': 'req-60545ce4-a4ef-4632-8d6c-85dd8ecd60cc'} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:22:05.496112 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-60545ce4-a4ef-4632-8d6c-85dd8ecd60cc admin admin] The token cache is being invalidate because project f4a3f5c6eb964c86b5851942d2edb461 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81399) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 03:22:05.496242 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-60545ce4-a4ef-4632-8d6c-85dd8ecd60cc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:22:05.496873 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-60545ce4-a4ef-4632-8d6c-85dd8ecd60cc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:22:05.498654 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 999/1994] 10.222.0.18 () {58 vars in 1233 bytes} [Tue Sep 13 03:22:05 2022] DELETE /identity/v3/projects/f4a3f5c6eb964c86b5851942d2edb461 => generated 0 bytes in 111 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 13 03:22:05.502404 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df3fe4c1-7c79-4d3c-a9c7-82fd38589acd service neutron] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:22:05.512988 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df3fe4c1-7c79-4d3c-a9c7-82fd38589acd service neutron] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:22:05.546334 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:22:05.571348 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84031dc0-c496-4809-a31e-a0a77bb2e9f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76c5181fc7be45279ac00017bb4640b3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:22:05.571971 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84031dc0-c496-4809-a31e-a0a77bb2e9f5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:22:05.572250 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84031dc0-c496-4809-a31e-a0a77bb2e9f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:22:05.572520 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84031dc0-c496-4809-a31e-a0a77bb2e9f5 admin admin] PATH_INFO: `/v3/users/6ced7b4933344e2dae8d0cb2968cf60a` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:22:05.573630 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84031dc0-c496-4809-a31e-a0a77bb2e9f5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6ced7b4933344e2dae8d0cb2968cf60a)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:22:05.575592 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84031dc0-c496-4809-a31e-a0a77bb2e9f5 admin admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:22:05.621745 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-84031dc0-c496-4809-a31e-a0a77bb2e9f5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6ced7b4933344e2dae8d0cb2968cf60a'} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:22:05.628606 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-84031dc0-c496-4809-a31e-a0a77bb2e9f5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6ced7b4933344e2dae8d0cb2968cf60a'} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:22:05.629435 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-84031dc0-c496-4809-a31e-a0a77bb2e9f5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6ced7b4933344e2dae8d0cb2968cf60a'} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:22:05.641525 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-84031dc0-c496-4809-a31e-a0a77bb2e9f5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6ced7b4933344e2dae8d0cb2968cf60a'} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:22:05.648407 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-84031dc0-c496-4809-a31e-a0a77bb2e9f5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6ced7b4933344e2dae8d0cb2968cf60a'} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:22:05.654237 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 996/1995] 10.222.0.18 () {58 vars in 1224 bytes} [Tue Sep 13 03:22:05 2022] DELETE /identity/v3/users/6ced7b4933344e2dae8d0cb2968cf60a => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 03:22:05.987427 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60545ce4-a4ef-4632-8d6c-85dd8ecd60cc admin admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:22:05.990094 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-60545ce4-a4ef-4632-8d6c-85dd8ecd60cc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:22:06.020827 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60545ce4-a4ef-4632-8d6c-85dd8ecd60cc admin admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:22:06.085772 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54720e75-d407-4a91-9994-db370e2c9c9a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76c5181fc7be45279ac00017bb4640b3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:22:06.086449 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54720e75-d407-4a91-9994-db370e2c9c9a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:22:06.086763 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54720e75-d407-4a91-9994-db370e2c9c9a admin admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:22:06.087034 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54720e75-d407-4a91-9994-db370e2c9c9a admin admin] PATH_INFO: `/v3/projects/0ff834f08b5642d8a3fc9ef11f9d1d06` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:22:06.088158 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54720e75-d407-4a91-9994-db370e2c9c9a admin admin] RBAC: Authorizing `identity:delete_project(project_id=0ff834f08b5642d8a3fc9ef11f9d1d06)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:22:06.090122 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54720e75-d407-4a91-9994-db370e2c9c9a admin admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:22:06.137115 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-54720e75-d407-4a91-9994-db370e2c9c9a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0ff834f08b5642d8a3fc9ef11f9d1d06', 'request_id': 'req-54720e75-d407-4a91-9994-db370e2c9c9a'} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:22:06.142870 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-54720e75-d407-4a91-9994-db370e2c9c9a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0ff834f08b5642d8a3fc9ef11f9d1d06', 'request_id': 'req-54720e75-d407-4a91-9994-db370e2c9c9a'} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:22:06.150293 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-54720e75-d407-4a91-9994-db370e2c9c9a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0ff834f08b5642d8a3fc9ef11f9d1d06', 'request_id': 'req-54720e75-d407-4a91-9994-db370e2c9c9a'} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:22:06.164879 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-54720e75-d407-4a91-9994-db370e2c9c9a admin admin] The token cache is being invalidate because project 0ff834f08b5642d8a3fc9ef11f9d1d06 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81399) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 03:22:06.165284 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-54720e75-d407-4a91-9994-db370e2c9c9a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:22:06.165788 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-54720e75-d407-4a91-9994-db370e2c9c9a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:22:06.167982 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 1000/1996] 10.222.0.18 () {58 vars in 1233 bytes} [Tue Sep 13 03:22:05 2022] DELETE /identity/v3/projects/0ff834f08b5642d8a3fc9ef11f9d1d06 => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 03:22:06.171784 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84031dc0-c496-4809-a31e-a0a77bb2e9f5 admin admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:22:06.173187 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-84031dc0-c496-4809-a31e-a0a77bb2e9f5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:22:06.197181 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84031dc0-c496-4809-a31e-a0a77bb2e9f5 admin admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:22:06.270858 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2edf4d3c-f44c-4c93-89e2-f501915135d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76c5181fc7be45279ac00017bb4640b3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:22:06.271498 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2edf4d3c-f44c-4c93-89e2-f501915135d1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:22:06.271772 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2edf4d3c-f44c-4c93-89e2-f501915135d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:22:06.272046 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2edf4d3c-f44c-4c93-89e2-f501915135d1 admin admin] PATH_INFO: `/v3/users/f88ed4635011458bad80fa9c1cfed512` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:22:06.273184 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2edf4d3c-f44c-4c93-89e2-f501915135d1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f88ed4635011458bad80fa9c1cfed512)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:22:06.275155 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2edf4d3c-f44c-4c93-89e2-f501915135d1 admin admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:22:06.335641 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-2edf4d3c-f44c-4c93-89e2-f501915135d1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f88ed4635011458bad80fa9c1cfed512'} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:22:06.349201 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-2edf4d3c-f44c-4c93-89e2-f501915135d1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f88ed4635011458bad80fa9c1cfed512'} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:22:06.350183 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-2edf4d3c-f44c-4c93-89e2-f501915135d1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f88ed4635011458bad80fa9c1cfed512'} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:22:06.363482 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=80,keystone:DELETE=18,keystone:INSERT=2 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:22:06.387469 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-2edf4d3c-f44c-4c93-89e2-f501915135d1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f88ed4635011458bad80fa9c1cfed512'} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:22:06.412054 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-2edf4d3c-f44c-4c93-89e2-f501915135d1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f88ed4635011458bad80fa9c1cfed512'} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:22:06.431891 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 997/1997] 10.222.0.18 () {58 vars in 1224 bytes} [Tue Sep 13 03:22:06 2022] DELETE /identity/v3/users/f88ed4635011458bad80fa9c1cfed512 => generated 0 bytes in 257 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 03:22:06.751951 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54720e75-d407-4a91-9994-db370e2c9c9a admin admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:22:06.753410 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-54720e75-d407-4a91-9994-db370e2c9c9a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:22:06.767695 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54720e75-d407-4a91-9994-db370e2c9c9a admin admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:22:06.816426 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:22:06.831368 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65d61385-ae41-4f8b-8a71-02e4216d0c71 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76c5181fc7be45279ac00017bb4640b3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:22:06.832007 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65d61385-ae41-4f8b-8a71-02e4216d0c71 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:22:06.832310 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65d61385-ae41-4f8b-8a71-02e4216d0c71 admin admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:22:06.832582 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65d61385-ae41-4f8b-8a71-02e4216d0c71 admin admin] PATH_INFO: `/v3/projects/fbf0d33f7f864bbb946ad6c7f404c212` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:22:06.833686 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65d61385-ae41-4f8b-8a71-02e4216d0c71 admin admin] RBAC: Authorizing `identity:delete_project(project_id=fbf0d33f7f864bbb946ad6c7f404c212)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:22:06.835653 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65d61385-ae41-4f8b-8a71-02e4216d0c71 admin admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:22:06.880228 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-65d61385-ae41-4f8b-8a71-02e4216d0c71 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fbf0d33f7f864bbb946ad6c7f404c212', 'request_id': 'req-65d61385-ae41-4f8b-8a71-02e4216d0c71'} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:22:06.886250 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-65d61385-ae41-4f8b-8a71-02e4216d0c71 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fbf0d33f7f864bbb946ad6c7f404c212', 'request_id': 'req-65d61385-ae41-4f8b-8a71-02e4216d0c71'} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:22:06.892484 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-65d61385-ae41-4f8b-8a71-02e4216d0c71 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fbf0d33f7f864bbb946ad6c7f404c212', 'request_id': 'req-65d61385-ae41-4f8b-8a71-02e4216d0c71'} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:22:06.904440 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-65d61385-ae41-4f8b-8a71-02e4216d0c71 admin admin] The token cache is being invalidate because project fbf0d33f7f864bbb946ad6c7f404c212 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81399) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 03:22:06.904758 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-65d61385-ae41-4f8b-8a71-02e4216d0c71 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:22:06.905388 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-65d61385-ae41-4f8b-8a71-02e4216d0c71 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:22:06.906919 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 1001/1998] 10.222.0.18 () {58 vars in 1233 bytes} [Tue Sep 13 03:22:06 2022] DELETE /identity/v3/projects/fbf0d33f7f864bbb946ad6c7f404c212 => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 03:22:06.912843 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c9812d5-4299-4fb9-89ff-0e7eebba533d None None] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:22:06.913122 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c9812d5-4299-4fb9-89ff-0e7eebba533d None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:22:06.913387 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c9812d5-4299-4fb9-89ff-0e7eebba533d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:22:06.933434 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.auth.core [None req-6c9812d5-4299-4fb9-89ff-0e7eebba533d None None] MFA Rules not processed for user `49f280ab086b4610855c4ffa378517d9`. Rule list: `[]` (Enabled: `True`). {{(pid=81400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 03:22:06.934621 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-6c9812d5-4299-4fb9-89ff-0e7eebba533d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:22:06.992268 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 998/1999] 10.222.0.18 () {58 vars in 958 bytes} [Tue Sep 13 03:22:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 13 03:22:07.000791 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8094f79b-bde7-4cd8-8fb1-9d2d1b413d05 None None] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:22:07.001279 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8094f79b-bde7-4cd8-8fb1-9d2d1b413d05 None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:22:07.001390 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8094f79b-bde7-4cd8-8fb1-9d2d1b413d05 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:22:07.021772 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.auth.core [None req-8094f79b-bde7-4cd8-8fb1-9d2d1b413d05 None None] MFA Rules not processed for user `49f280ab086b4610855c4ffa378517d9`. Rule list: `[]` (Enabled: `True`). {{(pid=81399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 03:22:07.023000 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-8094f79b-bde7-4cd8-8fb1-9d2d1b413d05 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:22:07.031215 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 1002/2000] 10.222.0.18 () {58 vars in 958 bytes} [Tue Sep 13 03:22:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 03:22:07.036118 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c9812d5-4299-4fb9-89ff-0e7eebba533d None None] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:22:07.047097 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c9812d5-4299-4fb9-89ff-0e7eebba533d None None] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:22:07.052420 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17c699ed-e96c-4c65-bc18-be1041268038 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76c5181fc7be45279ac00017bb4640b3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:22:07.053063 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17c699ed-e96c-4c65-bc18-be1041268038 admin admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:22:07.053336 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17c699ed-e96c-4c65-bc18-be1041268038 admin admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:22:07.053598 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17c699ed-e96c-4c65-bc18-be1041268038 admin admin] PATH_INFO: `/v3/domains` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:22:07.054226 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17c699ed-e96c-4c65-bc18-be1041268038 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 03:22:07.054554 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17c699ed-e96c-4c65-bc18-be1041268038 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:22:07.056107 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17c699ed-e96c-4c65-bc18-be1041268038 admin admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:22:07.068170 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 999/2001] 10.222.0.18 () {58 vars in 1153 bytes} [Tue Sep 13 03:22:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 03:22:07.072799 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8094f79b-bde7-4cd8-8fb1-9d2d1b413d05 None None] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:22:07.096717 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8094f79b-bde7-4cd8-8fb1-9d2d1b413d05 None None] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:22:07.103109 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c12c19c-8b70-4c92-b908-479211a2eecb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76c5181fc7be45279ac00017bb4640b3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:22:07.104021 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c12c19c-8b70-4c92-b908-479211a2eecb admin admin] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:22:07.104313 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c12c19c-8b70-4c92-b908-479211a2eecb admin admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:22:07.104578 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c12c19c-8b70-4c92-b908-479211a2eecb admin admin] PATH_INFO: `/v3/projects` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:22:07.105849 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c12c19c-8b70-4c92-b908-479211a2eecb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:22:07.109147 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c12c19c-8b70-4c92-b908-479211a2eecb admin admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:22:07.144395 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 1003/2002] 10.222.0.18 () {60 vars in 1153 bytes} [Tue Sep 13 03:22:07 2022] POST /identity/v3/projects => generated 393 bytes in 69 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 03:22:07.145879 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17c699ed-e96c-4c65-bc18-be1041268038 admin admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:22:07.157699 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17c699ed-e96c-4c65-bc18-be1041268038 admin admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:22:07.287706 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16814110-f6db-4307-8b5b-31877ad9e8e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76c5181fc7be45279ac00017bb4640b3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:22:07.288410 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16814110-f6db-4307-8b5b-31877ad9e8e6 admin admin] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:22:07.288630 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16814110-f6db-4307-8b5b-31877ad9e8e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:22:07.288851 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16814110-f6db-4307-8b5b-31877ad9e8e6 admin admin] PATH_INFO: `/v3/users` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:22:07.289577 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16814110-f6db-4307-8b5b-31877ad9e8e6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:22:07.291655 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16814110-f6db-4307-8b5b-31877ad9e8e6 admin admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:22:07.324187 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 1000/2003] 10.222.0.18 () {60 vars in 1144 bytes} [Tue Sep 13 03:22:07 2022] POST /identity/v3/users => generated 297 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 03:22:07.325733 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c12c19c-8b70-4c92-b908-479211a2eecb admin admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:22:07.336557 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c12c19c-8b70-4c92-b908-479211a2eecb admin admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:22:07.341124 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-087c542e-0a28-42c9-b05a-0365ce9976bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76c5181fc7be45279ac00017bb4640b3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:22:07.341724 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-087c542e-0a28-42c9-b05a-0365ce9976bf admin admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:22:07.341998 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-087c542e-0a28-42c9-b05a-0365ce9976bf admin admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:22:07.342267 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-087c542e-0a28-42c9-b05a-0365ce9976bf admin admin] PATH_INFO: `/v3/roles` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:22:07.342963 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-087c542e-0a28-42c9-b05a-0365ce9976bf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:22:07.344558 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-087c542e-0a28-42c9-b05a-0365ce9976bf admin admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:22:07.355523 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 1004/2004] 10.222.0.18 () {58 vars in 1122 bytes} [Tue Sep 13 03:22:07 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 03:22:07.359444 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16814110-f6db-4307-8b5b-31877ad9e8e6 admin admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:22:07.369818 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16814110-f6db-4307-8b5b-31877ad9e8e6 admin admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:22:07.382935 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aee11f5-ce5d-4617-bfb7-25719dc992b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76c5181fc7be45279ac00017bb4640b3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:22:07.383560 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aee11f5-ce5d-4617-bfb7-25719dc992b6 admin admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:22:07.383835 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aee11f5-ce5d-4617-bfb7-25719dc992b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:22:07.384103 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aee11f5-ce5d-4617-bfb7-25719dc992b6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:22:07.384743 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aee11f5-ce5d-4617-bfb7-25719dc992b6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:22:07.386461 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aee11f5-ce5d-4617-bfb7-25719dc992b6 admin admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:22:07.398719 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 1001/2005] 10.222.0.18 () {58 vars in 1122 bytes} [Tue Sep 13 03:22:07 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 03:22:07.402746 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-087c542e-0a28-42c9-b05a-0365ce9976bf admin admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:22:07.412957 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-087c542e-0a28-42c9-b05a-0365ce9976bf admin admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:22:07.417516 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ed92c31-2196-434c-9beb-784205689d1e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76c5181fc7be45279ac00017bb4640b3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:22:07.418159 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ed92c31-2196-434c-9beb-784205689d1e admin admin] REQUEST_METHOD: `PUT` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:22:07.418434 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ed92c31-2196-434c-9beb-784205689d1e admin admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:22:07.418739 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ed92c31-2196-434c-9beb-784205689d1e admin admin] PATH_INFO: `/v3/projects/a28eed0fdf7548c280cb37c3bcf60345/users/4698d6221409483e88bf4639b8f54265/roles/1eb7a5b5203945c29e6d22d653f47847` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:22:07.433953 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ed92c31-2196-434c-9beb-784205689d1e admin admin] RBAC: Authorizing `identity:create_grant(project_id=a28eed0fdf7548c280cb37c3bcf60345, user_id=4698d6221409483e88bf4639b8f54265, role_id=1eb7a5b5203945c29e6d22d653f47847)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:22:07.436631 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ed92c31-2196-434c-9beb-784205689d1e admin admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:22:07.450431 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 1005/2006] 10.222.0.18 () {60 vars in 1483 bytes} [Tue Sep 13 03:22:07 2022] PUT /identity/v3/projects/a28eed0fdf7548c280cb37c3bcf60345/users/4698d6221409483e88bf4639b8f54265/roles/1eb7a5b5203945c29e6d22d653f47847 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 13 03:22:07.462020 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7912f63a-4756-4ac9-802e-f97584e5d02d None None] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:22:07.462300 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7912f63a-4756-4ac9-802e-f97584e5d02d None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:22:07.462566 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7912f63a-4756-4ac9-802e-f97584e5d02d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:22:07.492999 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.auth.core [None req-7912f63a-4756-4ac9-802e-f97584e5d02d None None] MFA Rules not processed for user `4698d6221409483e88bf4639b8f54265`. Rule list: `[]` (Enabled: `True`). {{(pid=81400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 03:22:07.494204 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-7912f63a-4756-4ac9-802e-f97584e5d02d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:22:07.524279 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:DELETE=2,keystone:SELECT=95,keystone:INSERT=3 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:22:07.562175 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 1002/2007] 10.222.0.18 () {58 vars in 958 bytes} [Tue Sep 13 03:22:07 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 03:22:07.566757 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ed92c31-2196-434c-9beb-784205689d1e admin admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:22:07.627441 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ed92c31-2196-434c-9beb-784205689d1e admin admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:22:07.632763 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03ce72b2-c2ca-4cbd-a03d-fbf80e9cacea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76c5181fc7be45279ac00017bb4640b3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:22:07.633374 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ce72b2-c2ca-4cbd-a03d-fbf80e9cacea admin admin] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:22:07.633653 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ce72b2-c2ca-4cbd-a03d-fbf80e9cacea admin admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:22:07.634361 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ce72b2-c2ca-4cbd-a03d-fbf80e9cacea admin admin] PATH_INFO: `/v3/projects` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:22:07.634556 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03ce72b2-c2ca-4cbd-a03d-fbf80e9cacea admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:22:07.636481 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03ce72b2-c2ca-4cbd-a03d-fbf80e9cacea admin admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:22:07.658169 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:22:07.666795 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 1006/2008] 10.222.0.18 () {60 vars in 1153 bytes} [Tue Sep 13 03:22:07 2022] POST /identity/v3/projects => generated 391 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 03:22:07.671566 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7912f63a-4756-4ac9-802e-f97584e5d02d None None] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:22:07.683708 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7912f63a-4756-4ac9-802e-f97584e5d02d None None] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:22:07.742962 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d396b6b-9747-48ed-b0cf-7d40d4d9dc51 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76c5181fc7be45279ac00017bb4640b3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:22:07.743592 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d396b6b-9747-48ed-b0cf-7d40d4d9dc51 admin admin] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:22:07.743865 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d396b6b-9747-48ed-b0cf-7d40d4d9dc51 admin admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:22:07.744131 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d396b6b-9747-48ed-b0cf-7d40d4d9dc51 admin admin] PATH_INFO: `/v3/users` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:22:07.744747 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d396b6b-9747-48ed-b0cf-7d40d4d9dc51 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:22:07.746718 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d396b6b-9747-48ed-b0cf-7d40d4d9dc51 admin admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:22:07.770641 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 1003/2009] 10.222.0.18 () {60 vars in 1144 bytes} [Tue Sep 13 03:22:07 2022] POST /identity/v3/users => generated 302 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 03:22:07.773200 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03ce72b2-c2ca-4cbd-a03d-fbf80e9cacea admin admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:22:07.783319 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03ce72b2-c2ca-4cbd-a03d-fbf80e9cacea admin admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:22:07.788012 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-186524e3-b5fa-4c99-870e-4dc8b3efac8f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76c5181fc7be45279ac00017bb4640b3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:22:07.788623 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-186524e3-b5fa-4c99-870e-4dc8b3efac8f admin admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:22:07.788901 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-186524e3-b5fa-4c99-870e-4dc8b3efac8f admin admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:22:07.789161 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-186524e3-b5fa-4c99-870e-4dc8b3efac8f admin admin] PATH_INFO: `/v3/roles` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:22:07.789798 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-186524e3-b5fa-4c99-870e-4dc8b3efac8f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:22:07.791428 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-186524e3-b5fa-4c99-870e-4dc8b3efac8f admin admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:22:07.808328 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 1007/2010] 10.222.0.18 () {58 vars in 1122 bytes} [Tue Sep 13 03:22:07 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 03:22:07.813240 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d396b6b-9747-48ed-b0cf-7d40d4d9dc51 admin admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:22:07.825334 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d396b6b-9747-48ed-b0cf-7d40d4d9dc51 admin admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:22:07.830667 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd52278c-10f7-4cf9-8fd0-22e513bd4777 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76c5181fc7be45279ac00017bb4640b3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:22:07.831348 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd52278c-10f7-4cf9-8fd0-22e513bd4777 admin admin] REQUEST_METHOD: `PUT` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:22:07.831628 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd52278c-10f7-4cf9-8fd0-22e513bd4777 admin admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:22:07.831903 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd52278c-10f7-4cf9-8fd0-22e513bd4777 admin admin] PATH_INFO: `/v3/projects/40cc8fe96a044ad78adb15f27173fdab/users/7b14a4aa8c774264a13239d7939aede8/roles/425d4b481c7846e0bc07e7e805d5b15f` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:22:07.850345 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd52278c-10f7-4cf9-8fd0-22e513bd4777 admin admin] RBAC: Authorizing `identity:create_grant(project_id=40cc8fe96a044ad78adb15f27173fdab, user_id=7b14a4aa8c774264a13239d7939aede8, role_id=425d4b481c7846e0bc07e7e805d5b15f)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:22:07.853363 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd52278c-10f7-4cf9-8fd0-22e513bd4777 admin admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:22:07.866847 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 1004/2011] 10.222.0.18 () {60 vars in 1483 bytes} [Tue Sep 13 03:22:07 2022] PUT /identity/v3/projects/40cc8fe96a044ad78adb15f27173fdab/users/7b14a4aa8c774264a13239d7939aede8/roles/425d4b481c7846e0bc07e7e805d5b15f => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 03:22:07.871019 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-186524e3-b5fa-4c99-870e-4dc8b3efac8f admin admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:22:07.881645 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-186524e3-b5fa-4c99-870e-4dc8b3efac8f admin admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:22:07.945586 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7525e30-85a5-42bb-8387-53ec859ec0b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76c5181fc7be45279ac00017bb4640b3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:22:07.946205 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7525e30-85a5-42bb-8387-53ec859ec0b5 admin admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:22:07.946481 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7525e30-85a5-42bb-8387-53ec859ec0b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:22:07.946825 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7525e30-85a5-42bb-8387-53ec859ec0b5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:22:07.947465 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7525e30-85a5-42bb-8387-53ec859ec0b5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:22:07.949051 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7525e30-85a5-42bb-8387-53ec859ec0b5 admin admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:22:07.959545 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 1008/2012] 10.222.0.18 () {58 vars in 1122 bytes} [Tue Sep 13 03:22:07 2022] GET /identity/v3/roles => generated 1420 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 13 03:22:07.963856 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd52278c-10f7-4cf9-8fd0-22e513bd4777 admin admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:22:07.974371 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd52278c-10f7-4cf9-8fd0-22e513bd4777 admin admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:22:07.979562 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3a1ccbc-a544-4dbf-8e08-7c87867d3bb2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76c5181fc7be45279ac00017bb4640b3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:22:07.980281 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3a1ccbc-a544-4dbf-8e08-7c87867d3bb2 admin admin] REQUEST_METHOD: `PUT` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:22:07.980629 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3a1ccbc-a544-4dbf-8e08-7c87867d3bb2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:22:07.980969 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3a1ccbc-a544-4dbf-8e08-7c87867d3bb2 admin admin] PATH_INFO: `/v3/projects/40cc8fe96a044ad78adb15f27173fdab/users/7b14a4aa8c774264a13239d7939aede8/roles/425d4b481c7846e0bc07e7e805d5b15f` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:22:07.983026 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3a1ccbc-a544-4dbf-8e08-7c87867d3bb2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=40cc8fe96a044ad78adb15f27173fdab, user_id=7b14a4aa8c774264a13239d7939aede8, role_id=425d4b481c7846e0bc07e7e805d5b15f)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:22:07.985993 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3a1ccbc-a544-4dbf-8e08-7c87867d3bb2 admin admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:22:07.996702 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 1005/2013] 10.222.0.18 () {60 vars in 1483 bytes} [Tue Sep 13 03:22:07 2022] PUT /identity/v3/projects/40cc8fe96a044ad78adb15f27173fdab/users/7b14a4aa8c774264a13239d7939aede8/roles/425d4b481c7846e0bc07e7e805d5b15f => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 03:22:08.008339 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-080e8306-97df-40f4-8e18-2bc56f400ba9 None None] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:22:08.008797 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-080e8306-97df-40f4-8e18-2bc56f400ba9 None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:22:08.008889 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-080e8306-97df-40f4-8e18-2bc56f400ba9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:22:08.036754 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.auth.core [None req-080e8306-97df-40f4-8e18-2bc56f400ba9 None None] MFA Rules not processed for user `7b14a4aa8c774264a13239d7939aede8`. Rule list: `[]` (Enabled: `True`). {{(pid=81399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 03:22:08.037945 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-080e8306-97df-40f4-8e18-2bc56f400ba9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:22:08.105130 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 1009/2014] 10.222.0.18 () {58 vars in 958 bytes} [Tue Sep 13 03:22:08 2022] POST /identity/v3/auth/tokens => generated 3335 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 03:22:08.116719 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3a1ccbc-a544-4dbf-8e08-7c87867d3bb2 admin admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:22:08.118378 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-b3a1ccbc-a544-4dbf-8e08-7c87867d3bb2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:22:08.175704 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3a1ccbc-a544-4dbf-8e08-7c87867d3bb2 admin admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:22:08.179831 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61d4efc6-99ce-43cc-9706-c857c696c3b4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '924ee3f608d34a18b0ddc4695f102a56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b9d94f6a68f7470d9494f112175620dc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:22:08.180433 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d4efc6-99ce-43cc-9706-c857c696c3b4 service cinder] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:22:08.180709 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d4efc6-99ce-43cc-9706-c857c696c3b4 service cinder] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:22:08.180979 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d4efc6-99ce-43cc-9706-c857c696c3b4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:22:08.192503 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61d4efc6-99ce-43cc-9706-c857c696c3b4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:22:08.193951 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61d4efc6-99ce-43cc-9706-c857c696c3b4 service cinder] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:22:08.203957 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 1006/2015] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Sep 13 03:22:08 2022] GET /identity/v3/auth/tokens => generated 3335 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 13 03:22:08.250803 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-080e8306-97df-40f4-8e18-2bc56f400ba9 None None] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:22:08.252333 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-080e8306-97df-40f4-8e18-2bc56f400ba9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:22:08.264796 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-080e8306-97df-40f4-8e18-2bc56f400ba9 None None] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:22:08.268400 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e2070fe-fc09-4f0e-adc2-faab5e72a600 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '924ee3f608d34a18b0ddc4695f102a56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b9d94f6a68f7470d9494f112175620dc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:22:08.268993 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e2070fe-fc09-4f0e-adc2-faab5e72a600 service cinder] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:22:08.269270 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e2070fe-fc09-4f0e-adc2-faab5e72a600 service cinder] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:22:08.269539 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e2070fe-fc09-4f0e-adc2-faab5e72a600 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:22:08.338951 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e2070fe-fc09-4f0e-adc2-faab5e72a600 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:22:08.339730 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e2070fe-fc09-4f0e-adc2-faab5e72a600 service cinder] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:22:08.349665 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 1010/2016] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Sep 13 03:22:08 2022] GET /identity/v3/auth/tokens => generated 3270 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 03:22:12.014733 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61d4efc6-99ce-43cc-9706-c857c696c3b4 service cinder] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:22:12.052151 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:22:12.090412 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61d4efc6-99ce-43cc-9706-c857c696c3b4 service cinder] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:22:12.094975 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e3fdb95-62c8-4f2a-bd07-072f4f4596de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76c5181fc7be45279ac00017bb4640b3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:22:12.095807 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e3fdb95-62c8-4f2a-bd07-072f4f4596de admin admin] REQUEST_METHOD: `DELETE` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:22:12.096286 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e3fdb95-62c8-4f2a-bd07-072f4f4596de admin admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:22:12.096721 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e3fdb95-62c8-4f2a-bd07-072f4f4596de admin admin] PATH_INFO: `/v3/users/4698d6221409483e88bf4639b8f54265` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:22:12.097871 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e3fdb95-62c8-4f2a-bd07-072f4f4596de admin admin] RBAC: Authorizing `identity:delete_user(user_id=4698d6221409483e88bf4639b8f54265)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:22:12.099960 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e3fdb95-62c8-4f2a-bd07-072f4f4596de admin admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:22:12.158070 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-5e3fdb95-62c8-4f2a-bd07-072f4f4596de admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4698d6221409483e88bf4639b8f54265'} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:22:12.167492 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-5e3fdb95-62c8-4f2a-bd07-072f4f4596de admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4698d6221409483e88bf4639b8f54265'} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:22:12.168348 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-5e3fdb95-62c8-4f2a-bd07-072f4f4596de admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4698d6221409483e88bf4639b8f54265'} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:22:12.183511 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-5e3fdb95-62c8-4f2a-bd07-072f4f4596de admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4698d6221409483e88bf4639b8f54265'} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:22:12.191567 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-5e3fdb95-62c8-4f2a-bd07-072f4f4596de admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4698d6221409483e88bf4639b8f54265'} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:22:12.198354 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 1007/2017] 10.222.0.18 () {58 vars in 1224 bytes} [Tue Sep 13 03:22:12 2022] DELETE /identity/v3/users/4698d6221409483e88bf4639b8f54265 => generated 0 bytes in 184 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 03:22:12.202642 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce849d0-e61b-47f0-b7c6-f1c5c484719d None None] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:22:12.203166 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce849d0-e61b-47f0-b7c6-f1c5c484719d None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:22:12.203166 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce849d0-e61b-47f0-b7c6-f1c5c484719d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:22:12.229849 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.auth.core [None req-fce849d0-e61b-47f0-b7c6-f1c5c484719d None None] MFA Rules not processed for user `49f280ab086b4610855c4ffa378517d9`. Rule list: `[]` (Enabled: `True`). {{(pid=81399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 03:22:12.231099 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-fce849d0-e61b-47f0-b7c6-f1c5c484719d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:22:12.235366 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:22:12.299611 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 1011/2018] 10.222.0.18 () {58 vars in 958 bytes} [Tue Sep 13 03:22:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 03:22:12.305712 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e3fdb95-62c8-4f2a-bd07-072f4f4596de admin admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:22:12.308336 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-5e3fdb95-62c8-4f2a-bd07-072f4f4596de admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:22:12.377036 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e3fdb95-62c8-4f2a-bd07-072f4f4596de admin admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:22:12.381410 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8ceafa8-0a50-4f20-8a6e-efa28790f45d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdffbc851086441ebd01022d1e4c2609', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b9d94f6a68f7470d9494f112175620dc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:22:12.392432 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ceafa8-0a50-4f20-8a6e-efa28790f45d service neutron] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:22:12.393102 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ceafa8-0a50-4f20-8a6e-efa28790f45d service neutron] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:22:12.393553 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ceafa8-0a50-4f20-8a6e-efa28790f45d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:22:12.418008 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8ceafa8-0a50-4f20-8a6e-efa28790f45d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:22:12.420047 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8ceafa8-0a50-4f20-8a6e-efa28790f45d service neutron] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:22:12.444583 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 1008/2019] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Sep 13 03:22:12 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 139 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 13 03:22:12.929077 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fce849d0-e61b-47f0-b7c6-f1c5c484719d None None] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:22:12.931097 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-fce849d0-e61b-47f0-b7c6-f1c5c484719d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:22:12.951112 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fce849d0-e61b-47f0-b7c6-f1c5c484719d None None] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:22:12.956583 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b5fb11a-a5e1-4f60-be28-974216359ed6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76c5181fc7be45279ac00017bb4640b3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:22:12.958553 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b5fb11a-a5e1-4f60-be28-974216359ed6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:22:12.958553 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b5fb11a-a5e1-4f60-be28-974216359ed6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:22:12.958553 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b5fb11a-a5e1-4f60-be28-974216359ed6 admin admin] PATH_INFO: `/v3/projects/a28eed0fdf7548c280cb37c3bcf60345` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:22:12.959975 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b5fb11a-a5e1-4f60-be28-974216359ed6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a28eed0fdf7548c280cb37c3bcf60345)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:22:12.961863 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b5fb11a-a5e1-4f60-be28-974216359ed6 admin admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:22:13.020316 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-4b5fb11a-a5e1-4f60-be28-974216359ed6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a28eed0fdf7548c280cb37c3bcf60345', 'request_id': 'req-4b5fb11a-a5e1-4f60-be28-974216359ed6'} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:22:13.026318 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-4b5fb11a-a5e1-4f60-be28-974216359ed6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a28eed0fdf7548c280cb37c3bcf60345', 'request_id': 'req-4b5fb11a-a5e1-4f60-be28-974216359ed6'} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:22:13.033241 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-4b5fb11a-a5e1-4f60-be28-974216359ed6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a28eed0fdf7548c280cb37c3bcf60345', 'request_id': 'req-4b5fb11a-a5e1-4f60-be28-974216359ed6'} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:22:13.048427 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-4b5fb11a-a5e1-4f60-be28-974216359ed6 admin admin] The token cache is being invalidate because project a28eed0fdf7548c280cb37c3bcf60345 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81399) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 03:22:13.048719 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-4b5fb11a-a5e1-4f60-be28-974216359ed6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:22:13.049355 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-4b5fb11a-a5e1-4f60-be28-974216359ed6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:22:13.051299 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 1012/2020] 10.222.0.18 () {58 vars in 1233 bytes} [Tue Sep 13 03:22:12 2022] DELETE /identity/v3/projects/a28eed0fdf7548c280cb37c3bcf60345 => generated 0 bytes in 123 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 03:22:13.055240 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8ceafa8-0a50-4f20-8a6e-efa28790f45d service neutron] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:22:13.069272 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8ceafa8-0a50-4f20-8a6e-efa28790f45d service neutron] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:22:13.115739 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:22:13.129413 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a651a3f-79c0-4a74-a14d-0f1acbb0101f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76c5181fc7be45279ac00017bb4640b3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:22:13.130079 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a651a3f-79c0-4a74-a14d-0f1acbb0101f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:22:13.130349 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a651a3f-79c0-4a74-a14d-0f1acbb0101f admin admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:22:13.130629 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a651a3f-79c0-4a74-a14d-0f1acbb0101f admin admin] PATH_INFO: `/v3/users/7b14a4aa8c774264a13239d7939aede8` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:22:13.131761 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a651a3f-79c0-4a74-a14d-0f1acbb0101f admin admin] RBAC: Authorizing `identity:delete_user(user_id=7b14a4aa8c774264a13239d7939aede8)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:22:13.133705 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a651a3f-79c0-4a74-a14d-0f1acbb0101f admin admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:22:13.181230 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-9a651a3f-79c0-4a74-a14d-0f1acbb0101f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7b14a4aa8c774264a13239d7939aede8'} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:22:13.190107 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-9a651a3f-79c0-4a74-a14d-0f1acbb0101f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7b14a4aa8c774264a13239d7939aede8'} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:22:13.190741 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-9a651a3f-79c0-4a74-a14d-0f1acbb0101f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7b14a4aa8c774264a13239d7939aede8'} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:22:13.204078 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-9a651a3f-79c0-4a74-a14d-0f1acbb0101f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7b14a4aa8c774264a13239d7939aede8'} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:22:13.211535 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-9a651a3f-79c0-4a74-a14d-0f1acbb0101f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7b14a4aa8c774264a13239d7939aede8'} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:22:13.217422 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 1009/2021] 10.222.0.18 () {58 vars in 1224 bytes} [Tue Sep 13 03:22:13 2022] DELETE /identity/v3/users/7b14a4aa8c774264a13239d7939aede8 => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 03:22:13.532141 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b5fb11a-a5e1-4f60-be28-974216359ed6 admin admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:22:13.533564 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-4b5fb11a-a5e1-4f60-be28-974216359ed6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:22:13.548178 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b5fb11a-a5e1-4f60-be28-974216359ed6 admin admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:22:13.605223 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c0e8d1f-34e0-4e35-af3a-34fdd145a82a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76c5181fc7be45279ac00017bb4640b3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:22:13.605800 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c0e8d1f-34e0-4e35-af3a-34fdd145a82a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:22:13.606114 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c0e8d1f-34e0-4e35-af3a-34fdd145a82a admin admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:22:13.606351 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c0e8d1f-34e0-4e35-af3a-34fdd145a82a admin admin] PATH_INFO: `/v3/projects/40cc8fe96a044ad78adb15f27173fdab` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:22:13.607990 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c0e8d1f-34e0-4e35-af3a-34fdd145a82a admin admin] RBAC: Authorizing `identity:delete_project(project_id=40cc8fe96a044ad78adb15f27173fdab)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:22:13.609917 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c0e8d1f-34e0-4e35-af3a-34fdd145a82a admin admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:22:13.655687 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:22:13.669672 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-0c0e8d1f-34e0-4e35-af3a-34fdd145a82a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '40cc8fe96a044ad78adb15f27173fdab', 'request_id': 'req-0c0e8d1f-34e0-4e35-af3a-34fdd145a82a'} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:22:13.676242 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-0c0e8d1f-34e0-4e35-af3a-34fdd145a82a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '40cc8fe96a044ad78adb15f27173fdab', 'request_id': 'req-0c0e8d1f-34e0-4e35-af3a-34fdd145a82a'} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:22:13.684127 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-0c0e8d1f-34e0-4e35-af3a-34fdd145a82a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '40cc8fe96a044ad78adb15f27173fdab', 'request_id': 'req-0c0e8d1f-34e0-4e35-af3a-34fdd145a82a'} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:22:13.698314 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-0c0e8d1f-34e0-4e35-af3a-34fdd145a82a admin admin] The token cache is being invalidate because project 40cc8fe96a044ad78adb15f27173fdab was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81399) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 03:22:13.698667 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-0c0e8d1f-34e0-4e35-af3a-34fdd145a82a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:22:13.699419 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-0c0e8d1f-34e0-4e35-af3a-34fdd145a82a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:22:13.701523 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 1013/2022] 10.222.0.18 () {58 vars in 1233 bytes} [Tue Sep 13 03:22:13 2022] DELETE /identity/v3/projects/40cc8fe96a044ad78adb15f27173fdab => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 03:22:13.705885 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fdd11c6-32aa-47d6-993f-c2fd623487b8 None None] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:22:13.706108 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fdd11c6-32aa-47d6-993f-c2fd623487b8 None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:22:13.706374 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fdd11c6-32aa-47d6-993f-c2fd623487b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:22:13.726414 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.auth.core [None req-4fdd11c6-32aa-47d6-993f-c2fd623487b8 None None] MFA Rules not processed for user `49f280ab086b4610855c4ffa378517d9`. Rule list: `[]` (Enabled: `True`). {{(pid=81400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 03:22:13.727637 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-4fdd11c6-32aa-47d6-993f-c2fd623487b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:22:13.795190 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 1010/2023] 10.222.0.18 () {58 vars in 958 bytes} [Tue Sep 13 03:22:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 03:22:13.799160 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9234f580-eda3-47dd-a779-ef587014d24a None None] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:22:13.799444 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9234f580-eda3-47dd-a779-ef587014d24a None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:22:13.799780 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9234f580-eda3-47dd-a779-ef587014d24a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:22:13.821512 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.auth.core [None req-9234f580-eda3-47dd-a779-ef587014d24a None None] MFA Rules not processed for user `49f280ab086b4610855c4ffa378517d9`. Rule list: `[]` (Enabled: `True`). {{(pid=81399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 03:22:13.822786 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-9234f580-eda3-47dd-a779-ef587014d24a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:22:13.833539 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 1014/2024] 10.222.0.18 () {58 vars in 958 bytes} [Tue Sep 13 03:22:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 13 03:22:13.835001 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fdd11c6-32aa-47d6-993f-c2fd623487b8 None None] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:22:13.844900 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fdd11c6-32aa-47d6-993f-c2fd623487b8 None None] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:22:13.850232 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a37deb30-a440-400d-b9de-9b95c6b5198f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76c5181fc7be45279ac00017bb4640b3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:22:13.850925 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a37deb30-a440-400d-b9de-9b95c6b5198f admin admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:22:13.851298 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a37deb30-a440-400d-b9de-9b95c6b5198f admin admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:22:13.851630 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a37deb30-a440-400d-b9de-9b95c6b5198f admin admin] PATH_INFO: `/v3/domains` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:22:13.852332 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a37deb30-a440-400d-b9de-9b95c6b5198f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 03:22:13.852697 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a37deb30-a440-400d-b9de-9b95c6b5198f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:22:13.854372 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a37deb30-a440-400d-b9de-9b95c6b5198f admin admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:22:13.869726 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 1011/2025] 10.222.0.18 () {58 vars in 1153 bytes} [Tue Sep 13 03:22:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 03:22:13.871622 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9234f580-eda3-47dd-a779-ef587014d24a None None] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:22:13.880821 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9234f580-eda3-47dd-a779-ef587014d24a None None] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:22:13.885589 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-881fe22c-9a88-4667-97a5-8aae18fe73d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76c5181fc7be45279ac00017bb4640b3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:22:13.886178 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-881fe22c-9a88-4667-97a5-8aae18fe73d1 admin admin] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:22:13.886492 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-881fe22c-9a88-4667-97a5-8aae18fe73d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:22:13.886761 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-881fe22c-9a88-4667-97a5-8aae18fe73d1 admin admin] PATH_INFO: `/v3/projects` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:22:13.887380 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-881fe22c-9a88-4667-97a5-8aae18fe73d1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:22:13.889264 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-881fe22c-9a88-4667-97a5-8aae18fe73d1 admin admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:22:13.926995 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 1015/2026] 10.222.0.18 () {60 vars in 1153 bytes} [Tue Sep 13 03:22:13 2022] POST /identity/v3/projects => generated 371 bytes in 56 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 13 03:22:13.927710 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a37deb30-a440-400d-b9de-9b95c6b5198f admin admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:22:13.939541 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a37deb30-a440-400d-b9de-9b95c6b5198f admin admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:22:14.001692 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:22:14.012704 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aa790c3-bb0a-4671-9e90-7a9c6f151138 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76c5181fc7be45279ac00017bb4640b3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:22:14.013320 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aa790c3-bb0a-4671-9e90-7a9c6f151138 admin admin] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:22:14.013650 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aa790c3-bb0a-4671-9e90-7a9c6f151138 admin admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:22:14.013952 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aa790c3-bb0a-4671-9e90-7a9c6f151138 admin admin] PATH_INFO: `/v3/users` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:22:14.015140 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aa790c3-bb0a-4671-9e90-7a9c6f151138 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:22:14.019310 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aa790c3-bb0a-4671-9e90-7a9c6f151138 admin admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:22:14.049419 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 1012/2027] 10.222.0.18 () {60 vars in 1144 bytes} [Tue Sep 13 03:22:13 2022] POST /identity/v3/users => generated 286 bytes in 119 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 13 03:22:14.051431 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-881fe22c-9a88-4667-97a5-8aae18fe73d1 admin admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:22:14.062683 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-881fe22c-9a88-4667-97a5-8aae18fe73d1 admin admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:22:14.068544 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5289982a-cd49-460b-9a22-7be46b03e557 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76c5181fc7be45279ac00017bb4640b3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:22:14.069239 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5289982a-cd49-460b-9a22-7be46b03e557 admin admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:22:14.069588 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5289982a-cd49-460b-9a22-7be46b03e557 admin admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:22:14.069941 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5289982a-cd49-460b-9a22-7be46b03e557 admin admin] PATH_INFO: `/v3/roles` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:22:14.070713 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5289982a-cd49-460b-9a22-7be46b03e557 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:22:14.075277 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5289982a-cd49-460b-9a22-7be46b03e557 admin admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:22:14.090017 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 1016/2028] 10.222.0.18 () {58 vars in 1122 bytes} [Tue Sep 13 03:22:14 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 03:22:14.101030 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aa790c3-bb0a-4671-9e90-7a9c6f151138 admin admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:22:14.111380 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aa790c3-bb0a-4671-9e90-7a9c6f151138 admin admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:22:14.117868 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c251e62a-6209-4900-9224-2afb43607dd8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76c5181fc7be45279ac00017bb4640b3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:22:14.118592 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c251e62a-6209-4900-9224-2afb43607dd8 admin admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:22:14.118906 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c251e62a-6209-4900-9224-2afb43607dd8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:22:14.119259 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c251e62a-6209-4900-9224-2afb43607dd8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:22:14.120332 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c251e62a-6209-4900-9224-2afb43607dd8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:22:14.122943 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c251e62a-6209-4900-9224-2afb43607dd8 admin admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:22:14.138347 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 1013/2029] 10.222.0.18 () {58 vars in 1122 bytes} [Tue Sep 13 03:22:14 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 03:22:14.142446 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5289982a-cd49-460b-9a22-7be46b03e557 admin admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:22:14.153691 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5289982a-cd49-460b-9a22-7be46b03e557 admin admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:22:14.159512 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21ff3b05-c5dc-439d-859e-dac9b0c041c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76c5181fc7be45279ac00017bb4640b3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:22:14.160779 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21ff3b05-c5dc-439d-859e-dac9b0c041c5 admin admin] REQUEST_METHOD: `PUT` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:22:14.161175 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21ff3b05-c5dc-439d-859e-dac9b0c041c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:22:14.161477 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21ff3b05-c5dc-439d-859e-dac9b0c041c5 admin admin] PATH_INFO: `/v3/projects/322376a26bd249f886ef576c231ff662/users/148570ad0fa440f5a9a3f3dbb46d01f0/roles/1eb7a5b5203945c29e6d22d653f47847` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:22:14.179492 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21ff3b05-c5dc-439d-859e-dac9b0c041c5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=322376a26bd249f886ef576c231ff662, user_id=148570ad0fa440f5a9a3f3dbb46d01f0, role_id=1eb7a5b5203945c29e6d22d653f47847)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:22:14.182244 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21ff3b05-c5dc-439d-859e-dac9b0c041c5 admin admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:22:14.197824 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 1017/2030] 10.222.0.18 () {60 vars in 1483 bytes} [Tue Sep 13 03:22:14 2022] PUT /identity/v3/projects/322376a26bd249f886ef576c231ff662/users/148570ad0fa440f5a9a3f3dbb46d01f0/roles/1eb7a5b5203945c29e6d22d653f47847 => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 03:22:14.202972 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2435007-9a2d-4d8c-a225-7a2436823f9a None None] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:22:14.203233 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2435007-9a2d-4d8c-a225-7a2436823f9a None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:22:14.203505 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2435007-9a2d-4d8c-a225-7a2436823f9a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:22:14.225432 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.auth.core [None req-f2435007-9a2d-4d8c-a225-7a2436823f9a None None] MFA Rules not processed for user `49f280ab086b4610855c4ffa378517d9`. Rule list: `[]` (Enabled: `True`). {{(pid=81400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 03:22:14.226627 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-f2435007-9a2d-4d8c-a225-7a2436823f9a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:22:14.299821 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 1014/2031] 10.222.0.18 () {58 vars in 958 bytes} [Tue Sep 13 03:22:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 03:22:14.310778 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21ff3b05-c5dc-439d-859e-dac9b0c041c5 admin admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:22:14.311073 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-21ff3b05-c5dc-439d-859e-dac9b0c041c5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:22:14.370211 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21ff3b05-c5dc-439d-859e-dac9b0c041c5 admin admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:22:14.373984 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e27c55f-cd5f-4061-b8b3-c4852c860696 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdffbc851086441ebd01022d1e4c2609', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b9d94f6a68f7470d9494f112175620dc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:22:14.374639 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e27c55f-cd5f-4061-b8b3-c4852c860696 service neutron] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:22:14.374948 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e27c55f-cd5f-4061-b8b3-c4852c860696 service neutron] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:22:14.375286 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e27c55f-cd5f-4061-b8b3-c4852c860696 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:22:14.384838 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e27c55f-cd5f-4061-b8b3-c4852c860696 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:22:14.386292 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e27c55f-cd5f-4061-b8b3-c4852c860696 service neutron] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:22:14.396134 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 1018/2032] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Sep 13 03:22:14 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 03:22:15.042441 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-353d7f4c-1f8f-4682-9825-8531ddf095a6 None None] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:22:15.042996 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-353d7f4c-1f8f-4682-9825-8531ddf095a6 None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:22:15.043272 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-353d7f4c-1f8f-4682-9825-8531ddf095a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:22:15.092388 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.auth.core [None req-353d7f4c-1f8f-4682-9825-8531ddf095a6 None None] MFA Rules not processed for user `148570ad0fa440f5a9a3f3dbb46d01f0`. Rule list: `[]` (Enabled: `True`). {{(pid=81400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 03:22:15.093855 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-353d7f4c-1f8f-4682-9825-8531ddf095a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:22:15.196495 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 1015/2033] 10.222.0.18 () {58 vars in 958 bytes} [Tue Sep 13 03:22:15 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 03:22:15.202987 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e27c55f-cd5f-4061-b8b3-c4852c860696 service neutron] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:22:15.226722 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e27c55f-cd5f-4061-b8b3-c4852c860696 service neutron] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:22:15.240233 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3032a8a6-2b1b-454a-8b03-30a4c715de79 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76c5181fc7be45279ac00017bb4640b3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:22:15.240870 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3032a8a6-2b1b-454a-8b03-30a4c715de79 admin admin] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:22:15.241149 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3032a8a6-2b1b-454a-8b03-30a4c715de79 admin admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:22:15.241419 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3032a8a6-2b1b-454a-8b03-30a4c715de79 admin admin] PATH_INFO: `/v3/projects` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:22:15.242064 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3032a8a6-2b1b-454a-8b03-30a4c715de79 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:22:15.244012 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3032a8a6-2b1b-454a-8b03-30a4c715de79 admin admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:22:15.304206 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 1019/2034] 10.222.0.18 () {60 vars in 1153 bytes} [Tue Sep 13 03:22:15 2022] POST /identity/v3/projects => generated 373 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 03:22:15.308452 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-353d7f4c-1f8f-4682-9825-8531ddf095a6 None None] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:22:15.319803 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-353d7f4c-1f8f-4682-9825-8531ddf095a6 None None] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:22:15.408659 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ba1a875-9b19-488b-8970-fcf54003898b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76c5181fc7be45279ac00017bb4640b3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:22:15.409267 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ba1a875-9b19-488b-8970-fcf54003898b admin admin] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:22:15.409540 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ba1a875-9b19-488b-8970-fcf54003898b admin admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:22:15.409808 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ba1a875-9b19-488b-8970-fcf54003898b admin admin] PATH_INFO: `/v3/users` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:22:15.411034 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ba1a875-9b19-488b-8970-fcf54003898b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:22:15.412992 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ba1a875-9b19-488b-8970-fcf54003898b admin admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:22:15.422998 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:22:15.435795 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 1016/2035] 10.222.0.18 () {60 vars in 1144 bytes} [Tue Sep 13 03:22:15 2022] POST /identity/v3/users => generated 293 bytes in 128 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 13 03:22:15.441490 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3032a8a6-2b1b-454a-8b03-30a4c715de79 admin admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:22:15.452536 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3032a8a6-2b1b-454a-8b03-30a4c715de79 admin admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:22:15.457289 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9644bd8b-25e2-42cd-b864-0a0f58868ade admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76c5181fc7be45279ac00017bb4640b3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:22:15.457888 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9644bd8b-25e2-42cd-b864-0a0f58868ade admin admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:22:15.458168 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9644bd8b-25e2-42cd-b864-0a0f58868ade admin admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:22:15.458438 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9644bd8b-25e2-42cd-b864-0a0f58868ade admin admin] PATH_INFO: `/v3/roles` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:22:15.459108 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9644bd8b-25e2-42cd-b864-0a0f58868ade admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:22:15.460717 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9644bd8b-25e2-42cd-b864-0a0f58868ade admin admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:22:15.471275 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 1020/2036] 10.222.0.18 () {58 vars in 1122 bytes} [Tue Sep 13 03:22:15 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 03:22:15.475913 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ba1a875-9b19-488b-8970-fcf54003898b admin admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:22:15.486559 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ba1a875-9b19-488b-8970-fcf54003898b admin admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:22:15.492520 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a15130ee-79dc-45bf-82a1-ea01606e6425 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76c5181fc7be45279ac00017bb4640b3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:22:15.493223 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a15130ee-79dc-45bf-82a1-ea01606e6425 admin admin] REQUEST_METHOD: `PUT` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:22:15.493501 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a15130ee-79dc-45bf-82a1-ea01606e6425 admin admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:22:15.493780 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a15130ee-79dc-45bf-82a1-ea01606e6425 admin admin] PATH_INFO: `/v3/projects/5d323bc56b4343159d1ccf46aaaeb1f5/users/2f03dbe607ee4531aa76ff2e5aff90bd/roles/425d4b481c7846e0bc07e7e805d5b15f` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:22:15.509170 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a15130ee-79dc-45bf-82a1-ea01606e6425 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5d323bc56b4343159d1ccf46aaaeb1f5, user_id=2f03dbe607ee4531aa76ff2e5aff90bd, role_id=425d4b481c7846e0bc07e7e805d5b15f)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:22:15.511888 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a15130ee-79dc-45bf-82a1-ea01606e6425 admin admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:22:15.525899 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 1017/2037] 10.222.0.18 () {60 vars in 1483 bytes} [Tue Sep 13 03:22:15 2022] PUT /identity/v3/projects/5d323bc56b4343159d1ccf46aaaeb1f5/users/2f03dbe607ee4531aa76ff2e5aff90bd/roles/425d4b481c7846e0bc07e7e805d5b15f => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 03:22:15.530399 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9644bd8b-25e2-42cd-b864-0a0f58868ade admin admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:22:15.543714 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9644bd8b-25e2-42cd-b864-0a0f58868ade admin admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:22:15.570942 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:DELETE=5,keystone:SELECT=91,keystone:INSERT=4 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:22:15.604670 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f4cf565-dc21-4cbe-b8ab-810206411e5d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76c5181fc7be45279ac00017bb4640b3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:22:15.605287 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f4cf565-dc21-4cbe-b8ab-810206411e5d admin admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:22:15.605560 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f4cf565-dc21-4cbe-b8ab-810206411e5d admin admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:22:15.605826 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f4cf565-dc21-4cbe-b8ab-810206411e5d admin admin] PATH_INFO: `/v3/roles` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:22:15.606459 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f4cf565-dc21-4cbe-b8ab-810206411e5d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:22:15.608082 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f4cf565-dc21-4cbe-b8ab-810206411e5d admin admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:22:15.617775 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 1021/2038] 10.222.0.18 () {58 vars in 1122 bytes} [Tue Sep 13 03:22:15 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 03:22:15.622040 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a15130ee-79dc-45bf-82a1-ea01606e6425 admin admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:22:15.632357 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a15130ee-79dc-45bf-82a1-ea01606e6425 admin admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:22:15.637022 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-483cf592-e95d-4c76-820d-3a0fb2bd6228 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76c5181fc7be45279ac00017bb4640b3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:22:15.637670 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-483cf592-e95d-4c76-820d-3a0fb2bd6228 admin admin] REQUEST_METHOD: `PUT` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:22:15.637938 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-483cf592-e95d-4c76-820d-3a0fb2bd6228 admin admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:22:15.638206 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-483cf592-e95d-4c76-820d-3a0fb2bd6228 admin admin] PATH_INFO: `/v3/projects/5d323bc56b4343159d1ccf46aaaeb1f5/users/2f03dbe607ee4531aa76ff2e5aff90bd/roles/425d4b481c7846e0bc07e7e805d5b15f` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:22:15.640143 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-483cf592-e95d-4c76-820d-3a0fb2bd6228 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5d323bc56b4343159d1ccf46aaaeb1f5, user_id=2f03dbe607ee4531aa76ff2e5aff90bd, role_id=425d4b481c7846e0bc07e7e805d5b15f)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:22:15.642835 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-483cf592-e95d-4c76-820d-3a0fb2bd6228 admin admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:22:15.653535 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 1018/2039] 10.222.0.18 () {60 vars in 1483 bytes} [Tue Sep 13 03:22:15 2022] PUT /identity/v3/projects/5d323bc56b4343159d1ccf46aaaeb1f5/users/2f03dbe607ee4531aa76ff2e5aff90bd/roles/425d4b481c7846e0bc07e7e805d5b15f => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 03:22:16.313929 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b54709b5-b668-44f8-82ee-ce8447d6370c None None] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:22:16.314459 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b54709b5-b668-44f8-82ee-ce8447d6370c None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:22:16.314459 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b54709b5-b668-44f8-82ee-ce8447d6370c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:22:16.346416 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.auth.core [None req-b54709b5-b668-44f8-82ee-ce8447d6370c None None] MFA Rules not processed for user `2f03dbe607ee4531aa76ff2e5aff90bd`. Rule list: `[]` (Enabled: `True`). {{(pid=81399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 03:22:16.347536 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-b54709b5-b668-44f8-82ee-ce8447d6370c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:22:16.463805 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 1022/2040] 10.222.0.18 () {58 vars in 958 bytes} [Tue Sep 13 03:22:16 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 03:22:16.476212 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-483cf592-e95d-4c76-820d-3a0fb2bd6228 admin admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:22:16.477411 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-483cf592-e95d-4c76-820d-3a0fb2bd6228 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:22:16.550577 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-483cf592-e95d-4c76-820d-3a0fb2bd6228 admin admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:22:16.554696 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dc6c7cf-796a-4016-9b4c-703627a38b6e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '924ee3f608d34a18b0ddc4695f102a56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b9d94f6a68f7470d9494f112175620dc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:22:16.555295 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc6c7cf-796a-4016-9b4c-703627a38b6e service cinder] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:22:16.555567 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc6c7cf-796a-4016-9b4c-703627a38b6e service cinder] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:22:16.555830 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc6c7cf-796a-4016-9b4c-703627a38b6e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:22:16.615040 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dc6c7cf-796a-4016-9b4c-703627a38b6e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:22:16.616397 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dc6c7cf-796a-4016-9b4c-703627a38b6e service cinder] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:22:16.629812 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 1019/2041] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Sep 13 03:22:16 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 155 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 03:22:19.101851 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b54709b5-b668-44f8-82ee-ce8447d6370c None None] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:22:19.111384 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b54709b5-b668-44f8-82ee-ce8447d6370c None None] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:22:19.114905 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d71ebee-1005-4605-9c8f-5fbd8354d414 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '924ee3f608d34a18b0ddc4695f102a56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b9d94f6a68f7470d9494f112175620dc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:22:19.115499 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d71ebee-1005-4605-9c8f-5fbd8354d414 service cinder] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:22:19.115777 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d71ebee-1005-4605-9c8f-5fbd8354d414 service cinder] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:22:19.116049 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d71ebee-1005-4605-9c8f-5fbd8354d414 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:22:19.124805 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d71ebee-1005-4605-9c8f-5fbd8354d414 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:22:19.126224 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d71ebee-1005-4605-9c8f-5fbd8354d414 service cinder] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:22:19.136132 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 1023/2042] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Sep 13 03:22:19 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 03:22:26.622201 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:INSERT=3,keystone:SELECT=65 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:22:29.132886 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=56 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:22:32.734484 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3020d30-8c37-4848-9e0c-e8598aaf35b5 None None] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:22:32.735022 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3020d30-8c37-4848-9e0c-e8598aaf35b5 None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:22:32.735315 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3020d30-8c37-4848-9e0c-e8598aaf35b5 None None] PATH_INFO: `/` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:22:32.736858 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 1020/2043] 10.222.0.18 () {54 vars in 921 bytes} [Tue Sep 13 03:22:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 03:22:33.039177 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af91dff8-d0f6-43f2-8352-0b1f276f6fa8 None None] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:22:33.040172 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af91dff8-d0f6-43f2-8352-0b1f276f6fa8 None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:22:33.040450 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af91dff8-d0f6-43f2-8352-0b1f276f6fa8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:22:33.063499 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.auth.core [None req-af91dff8-d0f6-43f2-8352-0b1f276f6fa8 None None] MFA Rules not processed for user `4b99f1b2374d4d2f8db78306316180d9`. Rule list: `[]` (Enabled: `True`). {{(pid=81399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 03:22:33.065390 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-af91dff8-d0f6-43f2-8352-0b1f276f6fa8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:22:33.103214 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3020d30-8c37-4848-9e0c-e8598aaf35b5 None None] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:22:33.105358 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-e3020d30-8c37-4848-9e0c-e8598aaf35b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:22:33.153340 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 1024/2044] 10.222.0.18 () {58 vars in 1018 bytes} [Tue Sep 13 03:22:33 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 03:22:33.178599 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da2239fa-b4b3-4cf8-ac37-09623cfac4b7 None None] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:22:33.178993 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da2239fa-b4b3-4cf8-ac37-09623cfac4b7 None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:22:33.179344 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da2239fa-b4b3-4cf8-ac37-09623cfac4b7 None None] PATH_INFO: `/` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:22:33.180700 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 1025/2045] 10.222.0.18 () {54 vars in 961 bytes} [Tue Sep 13 03:22:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 13 03:22:33.183674 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3020d30-8c37-4848-9e0c-e8598aaf35b5 None None] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:22:33.185280 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14e65719-4dd3-4fb6-b4d6-8161a7595b56 None None] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:22:33.185566 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14e65719-4dd3-4fb6-b4d6-8161a7595b56 None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:22:33.185845 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14e65719-4dd3-4fb6-b4d6-8161a7595b56 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:22:33.188518 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c66ea0a5-9e53-4ae8-9030-ac302c6afe1c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76c5181fc7be45279ac00017bb4640b3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:22:33.189215 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c66ea0a5-9e53-4ae8-9030-ac302c6afe1c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:22:33.189563 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c66ea0a5-9e53-4ae8-9030-ac302c6afe1c admin admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:22:33.189838 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c66ea0a5-9e53-4ae8-9030-ac302c6afe1c admin admin] PATH_INFO: `/v3/users/616dd34b9fde4ed6b42ec22421ebb504` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:22:33.191481 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c66ea0a5-9e53-4ae8-9030-ac302c6afe1c admin admin] RBAC: Authorizing `identity:delete_user(user_id=616dd34b9fde4ed6b42ec22421ebb504)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:22:33.193454 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c66ea0a5-9e53-4ae8-9030-ac302c6afe1c admin admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:22:33.205788 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.auth.core [None req-14e65719-4dd3-4fb6-b4d6-8161a7595b56 None None] MFA Rules not processed for user `4b99f1b2374d4d2f8db78306316180d9`. Rule list: `[]` (Enabled: `True`). {{(pid=81399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 03:22:33.206968 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-14e65719-4dd3-4fb6-b4d6-8161a7595b56 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:22:33.216160 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 1026/2046] 10.222.0.18 () {58 vars in 1058 bytes} [Tue Sep 13 03:22:33 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 13 03:22:33.225373 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14e65719-4dd3-4fb6-b4d6-8161a7595b56 None None] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:22:33.237350 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14e65719-4dd3-4fb6-b4d6-8161a7595b56 None None] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:22:33.242550 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edffdd1b-c41b-4057-a9b9-402c79ac3efa service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b99f1b2374d4d2f8db78306316180d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b9d94f6a68f7470d9494f112175620dc', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:22:33.243189 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edffdd1b-c41b-4057-a9b9-402c79ac3efa service placement] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:22:33.243470 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edffdd1b-c41b-4057-a9b9-402c79ac3efa service placement] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:22:33.243741 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edffdd1b-c41b-4057-a9b9-402c79ac3efa service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:22:33.248333 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-c66ea0a5-9e53-4ae8-9030-ac302c6afe1c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '616dd34b9fde4ed6b42ec22421ebb504'} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:22:33.252245 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edffdd1b-c41b-4057-a9b9-402c79ac3efa service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:22:33.253885 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edffdd1b-c41b-4057-a9b9-402c79ac3efa service placement] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:22:33.259798 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-c66ea0a5-9e53-4ae8-9030-ac302c6afe1c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '616dd34b9fde4ed6b42ec22421ebb504'} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:22:33.260433 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-c66ea0a5-9e53-4ae8-9030-ac302c6afe1c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '616dd34b9fde4ed6b42ec22421ebb504'} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:22:33.263938 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 1027/2047] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Sep 13 03:22:33 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 13 03:22:33.273475 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-c66ea0a5-9e53-4ae8-9030-ac302c6afe1c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '616dd34b9fde4ed6b42ec22421ebb504'} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:22:33.280952 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-c66ea0a5-9e53-4ae8-9030-ac302c6afe1c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '616dd34b9fde4ed6b42ec22421ebb504'} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:22:33.287739 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 1021/2048] 10.222.0.18 () {58 vars in 1224 bytes} [Tue Sep 13 03:22:33 2022] DELETE /identity/v3/users/616dd34b9fde4ed6b42ec22421ebb504 => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 13 03:22:33.469752 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edffdd1b-c41b-4057-a9b9-402c79ac3efa service placement] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:22:33.488075 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edffdd1b-c41b-4057-a9b9-402c79ac3efa service placement] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:22:33.554055 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-322ecfd4-d7f1-486d-a452-5baa378b9a24 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76c5181fc7be45279ac00017bb4640b3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:22:33.554738 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-322ecfd4-d7f1-486d-a452-5baa378b9a24 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:22:33.555019 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-322ecfd4-d7f1-486d-a452-5baa378b9a24 admin admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:22:33.555292 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-322ecfd4-d7f1-486d-a452-5baa378b9a24 admin admin] PATH_INFO: `/v3/projects/53d5e604b12545a68da3655cb6b881c5` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:22:33.556440 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-322ecfd4-d7f1-486d-a452-5baa378b9a24 admin admin] RBAC: Authorizing `identity:delete_project(project_id=53d5e604b12545a68da3655cb6b881c5)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:22:33.558364 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-322ecfd4-d7f1-486d-a452-5baa378b9a24 admin admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:22:33.616814 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-322ecfd4-d7f1-486d-a452-5baa378b9a24 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '53d5e604b12545a68da3655cb6b881c5', 'request_id': 'req-322ecfd4-d7f1-486d-a452-5baa378b9a24'} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:22:33.626475 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-322ecfd4-d7f1-486d-a452-5baa378b9a24 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '53d5e604b12545a68da3655cb6b881c5', 'request_id': 'req-322ecfd4-d7f1-486d-a452-5baa378b9a24'} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:22:33.633798 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-322ecfd4-d7f1-486d-a452-5baa378b9a24 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '53d5e604b12545a68da3655cb6b881c5', 'request_id': 'req-322ecfd4-d7f1-486d-a452-5baa378b9a24'} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:22:33.649671 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-322ecfd4-d7f1-486d-a452-5baa378b9a24 admin admin] The token cache is being invalidate because project 53d5e604b12545a68da3655cb6b881c5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81399) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 03:22:33.649965 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-322ecfd4-d7f1-486d-a452-5baa378b9a24 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:22:33.650599 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-322ecfd4-d7f1-486d-a452-5baa378b9a24 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:22:33.652126 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 1028/2049] 10.222.0.18 () {58 vars in 1233 bytes} [Tue Sep 13 03:22:33 2022] DELETE /identity/v3/projects/53d5e604b12545a68da3655cb6b881c5 => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 03:22:33.657132 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a8b7a9e-4090-4a04-83ea-9a4b344d63c3 None None] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:22:33.657414 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a8b7a9e-4090-4a04-83ea-9a4b344d63c3 None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:22:33.657705 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a8b7a9e-4090-4a04-83ea-9a4b344d63c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:22:33.679176 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.auth.core [None req-6a8b7a9e-4090-4a04-83ea-9a4b344d63c3 None None] MFA Rules not processed for user `49f280ab086b4610855c4ffa378517d9`. Rule list: `[]` (Enabled: `True`). {{(pid=81400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 03:22:33.680823 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-6a8b7a9e-4090-4a04-83ea-9a4b344d63c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:22:33.739864 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 1022/2050] 10.222.0.18 () {58 vars in 958 bytes} [Tue Sep 13 03:22:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 03:22:33.747868 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e09e214-3a7b-436b-9102-fd0cf3bbbeba None None] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:22:33.748154 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e09e214-3a7b-436b-9102-fd0cf3bbbeba None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:22:33.748430 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e09e214-3a7b-436b-9102-fd0cf3bbbeba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:22:33.763440 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:22:33.769202 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.auth.core [None req-4e09e214-3a7b-436b-9102-fd0cf3bbbeba None None] MFA Rules not processed for user `49f280ab086b4610855c4ffa378517d9`. Rule list: `[]` (Enabled: `True`). {{(pid=81399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 03:22:33.771833 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-4e09e214-3a7b-436b-9102-fd0cf3bbbeba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:22:33.780480 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 1029/2051] 10.222.0.18 () {58 vars in 958 bytes} [Tue Sep 13 03:22:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 13 03:22:33.784637 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a8b7a9e-4090-4a04-83ea-9a4b344d63c3 None None] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:22:33.794439 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a8b7a9e-4090-4a04-83ea-9a4b344d63c3 None None] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:22:33.799324 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c12d71ac-d62f-404b-9cab-15e6a81f6dc2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76c5181fc7be45279ac00017bb4640b3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:22:33.800047 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c12d71ac-d62f-404b-9cab-15e6a81f6dc2 admin admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:22:33.800282 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c12d71ac-d62f-404b-9cab-15e6a81f6dc2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:22:33.800751 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c12d71ac-d62f-404b-9cab-15e6a81f6dc2 admin admin] PATH_INFO: `/v3/domains` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:22:33.801546 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c12d71ac-d62f-404b-9cab-15e6a81f6dc2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 03:22:33.802077 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c12d71ac-d62f-404b-9cab-15e6a81f6dc2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:22:33.804320 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c12d71ac-d62f-404b-9cab-15e6a81f6dc2 admin admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:22:33.816969 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 1023/2052] 10.222.0.18 () {58 vars in 1153 bytes} [Tue Sep 13 03:22:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 03:22:33.821153 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e09e214-3a7b-436b-9102-fd0cf3bbbeba None None] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:22:33.830897 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e09e214-3a7b-436b-9102-fd0cf3bbbeba None None] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:22:33.837224 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eb9984f-2d7b-4227-adf4-e02a35dba8c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76c5181fc7be45279ac00017bb4640b3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:22:33.837845 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb9984f-2d7b-4227-adf4-e02a35dba8c3 admin admin] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:22:33.838159 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb9984f-2d7b-4227-adf4-e02a35dba8c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:22:33.838432 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb9984f-2d7b-4227-adf4-e02a35dba8c3 admin admin] PATH_INFO: `/v3/projects` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:22:33.841241 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eb9984f-2d7b-4227-adf4-e02a35dba8c3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:22:33.843246 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eb9984f-2d7b-4227-adf4-e02a35dba8c3 admin admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:22:33.874976 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 1030/2053] 10.222.0.18 () {60 vars in 1153 bytes} [Tue Sep 13 03:22:33 2022] POST /identity/v3/projects => generated 363 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 13 03:22:33.879683 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c12d71ac-d62f-404b-9cab-15e6a81f6dc2 admin admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:22:33.889751 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c12d71ac-d62f-404b-9cab-15e6a81f6dc2 admin admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:22:33.893409 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:22:33.951354 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12759abb-c79b-4170-9ef4-45106c05c3f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76c5181fc7be45279ac00017bb4640b3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:22:33.951966 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12759abb-c79b-4170-9ef4-45106c05c3f7 admin admin] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:22:33.952240 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12759abb-c79b-4170-9ef4-45106c05c3f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:22:33.952504 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12759abb-c79b-4170-9ef4-45106c05c3f7 admin admin] PATH_INFO: `/v3/users` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:22:33.953131 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12759abb-c79b-4170-9ef4-45106c05c3f7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:22:33.955109 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12759abb-c79b-4170-9ef4-45106c05c3f7 admin admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:22:33.977663 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 1024/2054] 10.222.0.18 () {60 vars in 1144 bytes} [Tue Sep 13 03:22:33 2022] POST /identity/v3/users => generated 282 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 03:22:33.981928 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eb9984f-2d7b-4227-adf4-e02a35dba8c3 admin admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:22:33.993108 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eb9984f-2d7b-4227-adf4-e02a35dba8c3 admin admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:22:33.997936 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec976619-fbfd-4ddc-8406-f8dd1d77e268 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76c5181fc7be45279ac00017bb4640b3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:22:33.998546 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec976619-fbfd-4ddc-8406-f8dd1d77e268 admin admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:22:33.998857 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec976619-fbfd-4ddc-8406-f8dd1d77e268 admin admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:22:33.999297 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec976619-fbfd-4ddc-8406-f8dd1d77e268 admin admin] PATH_INFO: `/v3/roles` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:22:33.999941 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec976619-fbfd-4ddc-8406-f8dd1d77e268 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:22:34.001572 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec976619-fbfd-4ddc-8406-f8dd1d77e268 admin admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:22:34.012276 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 1031/2055] 10.222.0.18 () {58 vars in 1122 bytes} [Tue Sep 13 03:22:33 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 03:22:34.016284 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12759abb-c79b-4170-9ef4-45106c05c3f7 admin admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:22:34.026248 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12759abb-c79b-4170-9ef4-45106c05c3f7 admin admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:22:34.030895 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d31c4885-5490-49c2-8454-c1ace713399d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76c5181fc7be45279ac00017bb4640b3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:22:34.031495 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d31c4885-5490-49c2-8454-c1ace713399d admin admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:22:34.031774 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d31c4885-5490-49c2-8454-c1ace713399d admin admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:22:34.032046 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d31c4885-5490-49c2-8454-c1ace713399d admin admin] PATH_INFO: `/v3/roles` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:22:34.032685 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d31c4885-5490-49c2-8454-c1ace713399d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:22:34.035062 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d31c4885-5490-49c2-8454-c1ace713399d admin admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:22:34.046081 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 1025/2056] 10.222.0.18 () {58 vars in 1122 bytes} [Tue Sep 13 03:22:34 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 03:22:34.050358 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec976619-fbfd-4ddc-8406-f8dd1d77e268 admin admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:22:34.061256 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec976619-fbfd-4ddc-8406-f8dd1d77e268 admin admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:22:34.068869 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2001bb10-8b0b-4d5c-a886-f2c6da18e5f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76c5181fc7be45279ac00017bb4640b3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:22:34.069541 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2001bb10-8b0b-4d5c-a886-f2c6da18e5f0 admin admin] REQUEST_METHOD: `PUT` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:22:34.069821 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2001bb10-8b0b-4d5c-a886-f2c6da18e5f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:22:34.070096 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2001bb10-8b0b-4d5c-a886-f2c6da18e5f0 admin admin] PATH_INFO: `/v3/projects/f9cacf3bdc8f43fe9cd2ae60a8cb4e82/users/0b030a8e1147430c83093300f5599fc7/roles/1eb7a5b5203945c29e6d22d653f47847` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:22:34.085091 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2001bb10-8b0b-4d5c-a886-f2c6da18e5f0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f9cacf3bdc8f43fe9cd2ae60a8cb4e82, user_id=0b030a8e1147430c83093300f5599fc7, role_id=1eb7a5b5203945c29e6d22d653f47847)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:22:34.087776 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2001bb10-8b0b-4d5c-a886-f2c6da18e5f0 admin admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:22:34.100654 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 1032/2057] 10.222.0.18 () {60 vars in 1483 bytes} [Tue Sep 13 03:22:34 2022] PUT /identity/v3/projects/f9cacf3bdc8f43fe9cd2ae60a8cb4e82/users/0b030a8e1147430c83093300f5599fc7/roles/1eb7a5b5203945c29e6d22d653f47847 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 03:22:34.110447 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bda94df-aece-4e20-915d-40053b37755b None None] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:22:34.110768 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bda94df-aece-4e20-915d-40053b37755b None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:22:34.111042 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bda94df-aece-4e20-915d-40053b37755b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:22:34.139757 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.auth.core [None req-0bda94df-aece-4e20-915d-40053b37755b None None] MFA Rules not processed for user `0b030a8e1147430c83093300f5599fc7`. Rule list: `[]` (Enabled: `True`). {{(pid=81400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 03:22:34.140948 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-0bda94df-aece-4e20-915d-40053b37755b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:22:34.215446 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c76f87de-0f24-4f37-a2af-74413c8d1374 None None] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:22:34.215895 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c76f87de-0f24-4f37-a2af-74413c8d1374 None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:22:34.216424 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c76f87de-0f24-4f37-a2af-74413c8d1374 None None] PATH_INFO: `/` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:22:34.218062 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 1033/2058] 10.222.0.18 () {54 vars in 921 bytes} [Tue Sep 13 03:22:34 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 13 03:22:34.220012 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 1026/2059] 10.222.0.18 () {58 vars in 958 bytes} [Tue Sep 13 03:22:34 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 03:22:34.224297 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c76f87de-0f24-4f37-a2af-74413c8d1374 None None] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:22:34.248706 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79068d4a-072e-465a-95cc-8801e2e1e5bf None None] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:22:34.249030 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79068d4a-072e-465a-95cc-8801e2e1e5bf None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:22:34.249315 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79068d4a-072e-465a-95cc-8801e2e1e5bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:22:34.271646 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.auth.core [None req-79068d4a-072e-465a-95cc-8801e2e1e5bf None None] MFA Rules not processed for user `4b99f1b2374d4d2f8db78306316180d9`. Rule list: `[]` (Enabled: `True`). {{(pid=81400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 03:22:34.273001 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-79068d4a-072e-465a-95cc-8801e2e1e5bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:22:34.297534 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c76f87de-0f24-4f37-a2af-74413c8d1374 None None] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:22:34.303967 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce1dc480-a8d8-4e23-bee5-f4da05f11bea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76c5181fc7be45279ac00017bb4640b3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:22:34.304657 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce1dc480-a8d8-4e23-bee5-f4da05f11bea admin admin] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:22:34.304657 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce1dc480-a8d8-4e23-bee5-f4da05f11bea admin admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:22:34.305067 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce1dc480-a8d8-4e23-bee5-f4da05f11bea admin admin] PATH_INFO: `/v3/projects` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:22:34.305793 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce1dc480-a8d8-4e23-bee5-f4da05f11bea admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:22:34.308278 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce1dc480-a8d8-4e23-bee5-f4da05f11bea admin admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:22:34.354508 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 1027/2060] 10.222.0.18 () {58 vars in 1018 bytes} [Tue Sep 13 03:22:34 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 13 03:22:34.360669 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 1034/2061] 10.222.0.18 () {60 vars in 1153 bytes} [Tue Sep 13 03:22:34 2022] POST /identity/v3/projects => generated 361 bytes in 137 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 03:22:34.364926 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79068d4a-072e-465a-95cc-8801e2e1e5bf None None] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:22:34.374514 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce1dc480-a8d8-4e23-bee5-f4da05f11bea admin admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:22:34.376408 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-ce1dc480-a8d8-4e23-bee5-f4da05f11bea admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:22:34.382536 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79068d4a-072e-465a-95cc-8801e2e1e5bf None None] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:22:34.393182 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:22:34.470704 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7236f8d2-4867-4eeb-9fab-385638696b1a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76c5181fc7be45279ac00017bb4640b3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:22:34.473801 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7236f8d2-4867-4eeb-9fab-385638696b1a admin admin] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:22:34.473801 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7236f8d2-4867-4eeb-9fab-385638696b1a admin admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:22:34.474038 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7236f8d2-4867-4eeb-9fab-385638696b1a admin admin] PATH_INFO: `/v3/users` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:22:34.474649 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7236f8d2-4867-4eeb-9fab-385638696b1a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:22:34.476898 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7236f8d2-4867-4eeb-9fab-385638696b1a admin admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:22:34.481105 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce1dc480-a8d8-4e23-bee5-f4da05f11bea admin admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:22:34.487334 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-856da76a-e92c-4713-8a09-284c5288d0c4 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b99f1b2374d4d2f8db78306316180d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b9d94f6a68f7470d9494f112175620dc', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:22:34.488234 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-856da76a-e92c-4713-8a09-284c5288d0c4 service placement] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:22:34.489113 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-856da76a-e92c-4713-8a09-284c5288d0c4 service placement] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:22:34.489533 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-856da76a-e92c-4713-8a09-284c5288d0c4 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:22:34.500194 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-856da76a-e92c-4713-8a09-284c5288d0c4 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:22:34.501877 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-856da76a-e92c-4713-8a09-284c5288d0c4 service placement] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:22:34.506761 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 1028/2062] 10.222.0.18 () {60 vars in 1144 bytes} [Tue Sep 13 03:22:34 2022] POST /identity/v3/users => generated 287 bytes in 139 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 13 03:22:34.507780 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7236f8d2-4867-4eeb-9fab-385638696b1a admin admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:22:34.513959 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 1035/2063] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Sep 13 03:22:34 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 140 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 13 03:22:34.520237 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7236f8d2-4867-4eeb-9fab-385638696b1a admin admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:22:34.525144 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14eab568-a891-4b10-8543-3a24badab536 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76c5181fc7be45279ac00017bb4640b3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:22:34.526965 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14eab568-a891-4b10-8543-3a24badab536 admin admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:22:34.527479 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14eab568-a891-4b10-8543-3a24badab536 admin admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:22:34.527790 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14eab568-a891-4b10-8543-3a24badab536 admin admin] PATH_INFO: `/v3/roles` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:22:34.528540 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14eab568-a891-4b10-8543-3a24badab536 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:22:34.530651 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14eab568-a891-4b10-8543-3a24badab536 admin admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:22:34.541202 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 1029/2064] 10.222.0.18 () {58 vars in 1122 bytes} [Tue Sep 13 03:22:34 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 13 03:22:34.547525 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-856da76a-e92c-4713-8a09-284c5288d0c4 service placement] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:22:34.559820 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-856da76a-e92c-4713-8a09-284c5288d0c4 service placement] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:22:34.565173 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-721c3674-90e4-46ce-9288-6f5c3b537e3d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76c5181fc7be45279ac00017bb4640b3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:22:34.565830 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-721c3674-90e4-46ce-9288-6f5c3b537e3d admin admin] REQUEST_METHOD: `PUT` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:22:34.566104 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-721c3674-90e4-46ce-9288-6f5c3b537e3d admin admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:22:34.566402 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-721c3674-90e4-46ce-9288-6f5c3b537e3d admin admin] PATH_INFO: `/v3/projects/19bbaf664bef43129e553015baad9ca0/users/e20dec95f803485bb1e04a61e456d24a/roles/425d4b481c7846e0bc07e7e805d5b15f` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:22:34.584991 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-721c3674-90e4-46ce-9288-6f5c3b537e3d admin admin] RBAC: Authorizing `identity:create_grant(project_id=19bbaf664bef43129e553015baad9ca0, user_id=e20dec95f803485bb1e04a61e456d24a, role_id=425d4b481c7846e0bc07e7e805d5b15f)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:22:34.588169 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-721c3674-90e4-46ce-9288-6f5c3b537e3d admin admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:22:34.590995 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:22:34.608496 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 1036/2065] 10.222.0.18 () {60 vars in 1483 bytes} [Tue Sep 13 03:22:34 2022] PUT /identity/v3/projects/19bbaf664bef43129e553015baad9ca0/users/e20dec95f803485bb1e04a61e456d24a/roles/425d4b481c7846e0bc07e7e805d5b15f => generated 0 bytes in 62 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 03:22:34.612665 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14eab568-a891-4b10-8543-3a24badab536 admin admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:22:34.623661 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14eab568-a891-4b10-8543-3a24badab536 admin admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:22:34.681656 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-562d9313-bce8-41e8-b7a7-3b55d83b3af7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76c5181fc7be45279ac00017bb4640b3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:22:34.682317 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-562d9313-bce8-41e8-b7a7-3b55d83b3af7 admin admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:22:34.682687 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-562d9313-bce8-41e8-b7a7-3b55d83b3af7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:22:34.682960 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-562d9313-bce8-41e8-b7a7-3b55d83b3af7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:22:34.683613 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-562d9313-bce8-41e8-b7a7-3b55d83b3af7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:22:34.685191 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-562d9313-bce8-41e8-b7a7-3b55d83b3af7 admin admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:22:34.695327 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 1030/2066] 10.222.0.18 () {58 vars in 1122 bytes} [Tue Sep 13 03:22:34 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 03:22:34.699273 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-721c3674-90e4-46ce-9288-6f5c3b537e3d admin admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:22:34.710165 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-721c3674-90e4-46ce-9288-6f5c3b537e3d admin admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:22:34.714843 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c02fdf4-6043-487c-a008-fb789dbc9156 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76c5181fc7be45279ac00017bb4640b3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:22:34.715486 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c02fdf4-6043-487c-a008-fb789dbc9156 admin admin] REQUEST_METHOD: `PUT` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:22:34.715760 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c02fdf4-6043-487c-a008-fb789dbc9156 admin admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:22:34.716161 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c02fdf4-6043-487c-a008-fb789dbc9156 admin admin] PATH_INFO: `/v3/projects/19bbaf664bef43129e553015baad9ca0/users/e20dec95f803485bb1e04a61e456d24a/roles/425d4b481c7846e0bc07e7e805d5b15f` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:22:34.718039 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c02fdf4-6043-487c-a008-fb789dbc9156 admin admin] RBAC: Authorizing `identity:create_grant(project_id=19bbaf664bef43129e553015baad9ca0, user_id=e20dec95f803485bb1e04a61e456d24a, role_id=425d4b481c7846e0bc07e7e805d5b15f)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:22:34.720694 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c02fdf4-6043-487c-a008-fb789dbc9156 admin admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:22:34.731416 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 1037/2067] 10.222.0.18 () {60 vars in 1483 bytes} [Tue Sep 13 03:22:34 2022] PUT /identity/v3/projects/19bbaf664bef43129e553015baad9ca0/users/e20dec95f803485bb1e04a61e456d24a/roles/425d4b481c7846e0bc07e7e805d5b15f => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 03:22:34.742758 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d01dc4-0b0d-4621-b99a-df474b56f644 None None] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:22:34.743199 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d01dc4-0b0d-4621-b99a-df474b56f644 None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:22:34.743293 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d01dc4-0b0d-4621-b99a-df474b56f644 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:22:34.771648 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.auth.core [None req-95d01dc4-0b0d-4621-b99a-df474b56f644 None None] MFA Rules not processed for user `e20dec95f803485bb1e04a61e456d24a`. Rule list: `[]` (Enabled: `True`). {{(pid=81400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 03:22:34.772867 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-95d01dc4-0b0d-4621-b99a-df474b56f644 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:22:34.844934 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 1031/2068] 10.222.0.18 () {58 vars in 958 bytes} [Tue Sep 13 03:22:34 2022] POST /identity/v3/auth/tokens => generated 3305 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 03:22:34.858070 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c02fdf4-6043-487c-a008-fb789dbc9156 admin admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:22:34.860111 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-1c02fdf4-6043-487c-a008-fb789dbc9156 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:22:34.868525 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95d01dc4-0b0d-4621-b99a-df474b56f644 None None] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:22:34.870986 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-95d01dc4-0b0d-4621-b99a-df474b56f644 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:22:34.888233 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:22:34.923739 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c02fdf4-6043-487c-a008-fb789dbc9156 admin admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:22:34.927602 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e09fc4bd-9522-414e-a54d-ea960440b1bc service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '924ee3f608d34a18b0ddc4695f102a56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b9d94f6a68f7470d9494f112175620dc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:22:34.928200 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e09fc4bd-9522-414e-a54d-ea960440b1bc service cinder] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:22:34.928472 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e09fc4bd-9522-414e-a54d-ea960440b1bc service cinder] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:22:34.928736 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e09fc4bd-9522-414e-a54d-ea960440b1bc service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:22:34.937817 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e09fc4bd-9522-414e-a54d-ea960440b1bc service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:22:34.939277 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e09fc4bd-9522-414e-a54d-ea960440b1bc service cinder] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:22:34.942788 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95d01dc4-0b0d-4621-b99a-df474b56f644 None None] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:22:34.947386 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-815358e3-33d0-4a57-896f-623e74561693 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76c5181fc7be45279ac00017bb4640b3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:22:34.948007 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-815358e3-33d0-4a57-896f-623e74561693 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:22:34.948282 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-815358e3-33d0-4a57-896f-623e74561693 admin admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:22:34.948550 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-815358e3-33d0-4a57-896f-623e74561693 admin admin] PATH_INFO: `/v3/users/148570ad0fa440f5a9a3f3dbb46d01f0` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:22:34.948921 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 1038/2069] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Sep 13 03:22:34 2022] GET /identity/v3/auth/tokens => generated 3305 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 03:22:34.951394 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-815358e3-33d0-4a57-896f-623e74561693 admin admin] RBAC: Authorizing `identity:delete_user(user_id=148570ad0fa440f5a9a3f3dbb46d01f0)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:22:34.953323 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-815358e3-33d0-4a57-896f-623e74561693 admin admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:22:35.013369 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-815358e3-33d0-4a57-896f-623e74561693 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '148570ad0fa440f5a9a3f3dbb46d01f0'} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:22:35.020910 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-815358e3-33d0-4a57-896f-623e74561693 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '148570ad0fa440f5a9a3f3dbb46d01f0'} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:22:35.021624 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-815358e3-33d0-4a57-896f-623e74561693 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '148570ad0fa440f5a9a3f3dbb46d01f0'} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:22:35.028567 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e09fc4bd-9522-414e-a54d-ea960440b1bc service cinder] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:22:35.033662 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-815358e3-33d0-4a57-896f-623e74561693 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '148570ad0fa440f5a9a3f3dbb46d01f0'} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:22:35.037395 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e09fc4bd-9522-414e-a54d-ea960440b1bc service cinder] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:22:35.041116 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eaa3a0e-c7c4-4f85-a051-0758469b7429 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '924ee3f608d34a18b0ddc4695f102a56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b9d94f6a68f7470d9494f112175620dc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:22:35.041678 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-815358e3-33d0-4a57-896f-623e74561693 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '148570ad0fa440f5a9a3f3dbb46d01f0'} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:22:35.041799 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eaa3a0e-c7c4-4f85-a051-0758469b7429 service cinder] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:22:35.041963 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eaa3a0e-c7c4-4f85-a051-0758469b7429 service cinder] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:22:35.042232 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eaa3a0e-c7c4-4f85-a051-0758469b7429 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:22:35.044222 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-3eaa3a0e-c7c4-4f85-a051-0758469b7429 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:22:35.048171 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 1032/2070] 10.222.0.18 () {58 vars in 1224 bytes} [Tue Sep 13 03:22:34 2022] DELETE /identity/v3/users/148570ad0fa440f5a9a3f3dbb46d01f0 => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 03:22:35.106258 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eaa3a0e-c7c4-4f85-a051-0758469b7429 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:22:35.107745 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eaa3a0e-c7c4-4f85-a051-0758469b7429 service cinder] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:22:35.116992 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 1039/2071] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Sep 13 03:22:35 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 13 03:22:35.221271 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-815358e3-33d0-4a57-896f-623e74561693 admin admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:22:35.223561 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-815358e3-33d0-4a57-896f-623e74561693 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:22:35.301199 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-815358e3-33d0-4a57-896f-623e74561693 admin admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:22:35.305868 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d149794e-3801-4825-a3ad-cb1546ae6d6e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76c5181fc7be45279ac00017bb4640b3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:22:35.306491 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d149794e-3801-4825-a3ad-cb1546ae6d6e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:22:35.306813 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d149794e-3801-4825-a3ad-cb1546ae6d6e admin admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:22:35.307085 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d149794e-3801-4825-a3ad-cb1546ae6d6e admin admin] PATH_INFO: `/v3/projects/322376a26bd249f886ef576c231ff662` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:22:35.308270 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d149794e-3801-4825-a3ad-cb1546ae6d6e admin admin] RBAC: Authorizing `identity:delete_project(project_id=322376a26bd249f886ef576c231ff662)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:22:35.310168 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d149794e-3801-4825-a3ad-cb1546ae6d6e admin admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:22:35.346391 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:22:35.360266 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-d149794e-3801-4825-a3ad-cb1546ae6d6e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '322376a26bd249f886ef576c231ff662', 'request_id': 'req-d149794e-3801-4825-a3ad-cb1546ae6d6e'} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:22:35.365736 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-d149794e-3801-4825-a3ad-cb1546ae6d6e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '322376a26bd249f886ef576c231ff662', 'request_id': 'req-d149794e-3801-4825-a3ad-cb1546ae6d6e'} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:22:35.372281 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-d149794e-3801-4825-a3ad-cb1546ae6d6e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '322376a26bd249f886ef576c231ff662', 'request_id': 'req-d149794e-3801-4825-a3ad-cb1546ae6d6e'} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:22:35.384311 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-d149794e-3801-4825-a3ad-cb1546ae6d6e admin admin] The token cache is being invalidate because project 322376a26bd249f886ef576c231ff662 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81400) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 03:22:35.384593 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-d149794e-3801-4825-a3ad-cb1546ae6d6e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:22:35.385202 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-d149794e-3801-4825-a3ad-cb1546ae6d6e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:22:35.386690 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 1033/2072] 10.222.0.18 () {58 vars in 1233 bytes} [Tue Sep 13 03:22:35 2022] DELETE /identity/v3/projects/322376a26bd249f886ef576c231ff662 => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 03:22:35.390631 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eaa3a0e-c7c4-4f85-a051-0758469b7429 service cinder] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:22:35.401713 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eaa3a0e-c7c4-4f85-a051-0758469b7429 service cinder] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:22:35.458187 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5219d97-60fa-4c76-835a-b88d7cb0e6f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76c5181fc7be45279ac00017bb4640b3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:22:35.459049 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5219d97-60fa-4c76-835a-b88d7cb0e6f9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:22:35.459299 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5219d97-60fa-4c76-835a-b88d7cb0e6f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:22:35.459580 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5219d97-60fa-4c76-835a-b88d7cb0e6f9 admin admin] PATH_INFO: `/v3/users/2f03dbe607ee4531aa76ff2e5aff90bd` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:22:35.460679 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5219d97-60fa-4c76-835a-b88d7cb0e6f9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2f03dbe607ee4531aa76ff2e5aff90bd)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:22:35.462775 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5219d97-60fa-4c76-835a-b88d7cb0e6f9 admin admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:22:35.485261 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:INSERT=2,keystone:SELECT=94,keystone:DELETE=4 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:22:35.522137 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-a5219d97-60fa-4c76-835a-b88d7cb0e6f9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2f03dbe607ee4531aa76ff2e5aff90bd'} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:22:35.530187 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-a5219d97-60fa-4c76-835a-b88d7cb0e6f9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2f03dbe607ee4531aa76ff2e5aff90bd'} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:22:35.543122 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-a5219d97-60fa-4c76-835a-b88d7cb0e6f9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2f03dbe607ee4531aa76ff2e5aff90bd'} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:22:35.543795 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-a5219d97-60fa-4c76-835a-b88d7cb0e6f9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2f03dbe607ee4531aa76ff2e5aff90bd'} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:22:35.551479 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-a5219d97-60fa-4c76-835a-b88d7cb0e6f9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2f03dbe607ee4531aa76ff2e5aff90bd'} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:22:35.557244 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 1040/2073] 10.222.0.18 () {58 vars in 1224 bytes} [Tue Sep 13 03:22:35 2022] DELETE /identity/v3/users/2f03dbe607ee4531aa76ff2e5aff90bd => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 03:22:35.727008 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d149794e-3801-4825-a3ad-cb1546ae6d6e admin admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:22:35.728697 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-d149794e-3801-4825-a3ad-cb1546ae6d6e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:22:35.742590 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d149794e-3801-4825-a3ad-cb1546ae6d6e admin admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:22:35.800338 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ecc7a73-6abe-45e0-ada4-1ffcbf9cc2f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76c5181fc7be45279ac00017bb4640b3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:22:35.801516 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ecc7a73-6abe-45e0-ada4-1ffcbf9cc2f0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:22:35.801924 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ecc7a73-6abe-45e0-ada4-1ffcbf9cc2f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:22:35.802195 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ecc7a73-6abe-45e0-ada4-1ffcbf9cc2f0 admin admin] PATH_INFO: `/v3/projects/5d323bc56b4343159d1ccf46aaaeb1f5` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:22:35.803545 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ecc7a73-6abe-45e0-ada4-1ffcbf9cc2f0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5d323bc56b4343159d1ccf46aaaeb1f5)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:22:35.805679 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ecc7a73-6abe-45e0-ada4-1ffcbf9cc2f0 admin admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:22:35.853036 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-7ecc7a73-6abe-45e0-ada4-1ffcbf9cc2f0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5d323bc56b4343159d1ccf46aaaeb1f5', 'request_id': 'req-7ecc7a73-6abe-45e0-ada4-1ffcbf9cc2f0'} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:22:35.859494 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-7ecc7a73-6abe-45e0-ada4-1ffcbf9cc2f0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5d323bc56b4343159d1ccf46aaaeb1f5', 'request_id': 'req-7ecc7a73-6abe-45e0-ada4-1ffcbf9cc2f0'} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:22:35.866110 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-7ecc7a73-6abe-45e0-ada4-1ffcbf9cc2f0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5d323bc56b4343159d1ccf46aaaeb1f5', 'request_id': 'req-7ecc7a73-6abe-45e0-ada4-1ffcbf9cc2f0'} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:22:35.878150 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-7ecc7a73-6abe-45e0-ada4-1ffcbf9cc2f0 admin admin] The token cache is being invalidate because project 5d323bc56b4343159d1ccf46aaaeb1f5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81400) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 03:22:35.878431 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-7ecc7a73-6abe-45e0-ada4-1ffcbf9cc2f0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:22:35.879102 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-7ecc7a73-6abe-45e0-ada4-1ffcbf9cc2f0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:22:35.880563 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 1034/2074] 10.222.0.18 () {58 vars in 1233 bytes} [Tue Sep 13 03:22:35 2022] DELETE /identity/v3/projects/5d323bc56b4343159d1ccf46aaaeb1f5 => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 03:22:35.886361 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2e83f91-c698-4af9-9305-eeb980a850fe None None] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:22:35.886672 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2e83f91-c698-4af9-9305-eeb980a850fe None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:22:35.886967 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2e83f91-c698-4af9-9305-eeb980a850fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:22:35.906075 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.auth.core [None req-b2e83f91-c698-4af9-9305-eeb980a850fe None None] MFA Rules not processed for user `49f280ab086b4610855c4ffa378517d9`. Rule list: `[]` (Enabled: `True`). {{(pid=81399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 03:22:35.907303 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-b2e83f91-c698-4af9-9305-eeb980a850fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:22:35.965608 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 1041/2075] 10.222.0.18 () {58 vars in 958 bytes} [Tue Sep 13 03:22:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 13 03:22:35.973984 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8614c997-878b-4539-b963-458826f24a23 None None] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:22:35.974267 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8614c997-878b-4539-b963-458826f24a23 None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:22:35.974533 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8614c997-878b-4539-b963-458826f24a23 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:22:35.994230 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.auth.core [None req-8614c997-878b-4539-b963-458826f24a23 None None] MFA Rules not processed for user `49f280ab086b4610855c4ffa378517d9`. Rule list: `[]` (Enabled: `True`). {{(pid=81400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 03:22:35.995442 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-8614c997-878b-4539-b963-458826f24a23 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:22:36.003235 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 1035/2076] 10.222.0.18 () {58 vars in 958 bytes} [Tue Sep 13 03:22:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 03:22:36.007375 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2e83f91-c698-4af9-9305-eeb980a850fe None None] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:22:36.016956 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2e83f91-c698-4af9-9305-eeb980a850fe None None] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:22:36.021573 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5242e0a9-7489-40ab-824b-0cd712e09db1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76c5181fc7be45279ac00017bb4640b3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:22:36.022162 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5242e0a9-7489-40ab-824b-0cd712e09db1 admin admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:22:36.022437 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5242e0a9-7489-40ab-824b-0cd712e09db1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:22:36.022753 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5242e0a9-7489-40ab-824b-0cd712e09db1 admin admin] PATH_INFO: `/v3/domains` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:22:36.023406 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5242e0a9-7489-40ab-824b-0cd712e09db1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 03:22:36.023727 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5242e0a9-7489-40ab-824b-0cd712e09db1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:22:36.025617 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5242e0a9-7489-40ab-824b-0cd712e09db1 admin admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:22:36.037642 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 1042/2077] 10.222.0.18 () {58 vars in 1153 bytes} [Tue Sep 13 03:22:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 03:22:36.041942 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8614c997-878b-4539-b963-458826f24a23 None None] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:22:36.052188 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8614c997-878b-4539-b963-458826f24a23 None None] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:22:36.056802 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56bc4927-efd0-423a-9b2e-9f7816cbbe6e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76c5181fc7be45279ac00017bb4640b3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:22:36.057410 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56bc4927-efd0-423a-9b2e-9f7816cbbe6e admin admin] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:22:36.057690 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56bc4927-efd0-423a-9b2e-9f7816cbbe6e admin admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:22:36.057961 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56bc4927-efd0-423a-9b2e-9f7816cbbe6e admin admin] PATH_INFO: `/v3/projects` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:22:36.058584 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56bc4927-efd0-423a-9b2e-9f7816cbbe6e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:22:36.060515 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56bc4927-efd0-423a-9b2e-9f7816cbbe6e admin admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:22:36.087511 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 1036/2078] 10.222.0.18 () {60 vars in 1153 bytes} [Tue Sep 13 03:22:36 2022] POST /identity/v3/projects => generated 371 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 03:22:36.091611 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5242e0a9-7489-40ab-824b-0cd712e09db1 admin admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:22:36.101923 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5242e0a9-7489-40ab-824b-0cd712e09db1 admin admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:22:36.155817 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cd934cc-75d4-4c9f-88b4-d557d61003ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76c5181fc7be45279ac00017bb4640b3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:22:36.156422 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd934cc-75d4-4c9f-88b4-d557d61003ba admin admin] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:22:36.156678 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd934cc-75d4-4c9f-88b4-d557d61003ba admin admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:22:36.156948 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd934cc-75d4-4c9f-88b4-d557d61003ba admin admin] PATH_INFO: `/v3/users` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:22:36.157566 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cd934cc-75d4-4c9f-88b4-d557d61003ba admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:22:36.159558 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cd934cc-75d4-4c9f-88b4-d557d61003ba admin admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:22:36.181704 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 1043/2079] 10.222.0.18 () {60 vars in 1144 bytes} [Tue Sep 13 03:22:36 2022] POST /identity/v3/users => generated 286 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 03:22:36.185846 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56bc4927-efd0-423a-9b2e-9f7816cbbe6e admin admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:22:36.195957 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56bc4927-efd0-423a-9b2e-9f7816cbbe6e admin admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:22:36.200446 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1b95b4f-5bf6-4d5b-b6ef-d5e0426d1312 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76c5181fc7be45279ac00017bb4640b3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:22:36.201033 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b95b4f-5bf6-4d5b-b6ef-d5e0426d1312 admin admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:22:36.201305 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b95b4f-5bf6-4d5b-b6ef-d5e0426d1312 admin admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:22:36.201571 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b95b4f-5bf6-4d5b-b6ef-d5e0426d1312 admin admin] PATH_INFO: `/v3/roles` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:22:36.202206 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1b95b4f-5bf6-4d5b-b6ef-d5e0426d1312 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:22:36.203807 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1b95b4f-5bf6-4d5b-b6ef-d5e0426d1312 admin admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:22:36.213910 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 1037/2080] 10.222.0.18 () {58 vars in 1122 bytes} [Tue Sep 13 03:22:36 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 03:22:36.217584 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cd934cc-75d4-4c9f-88b4-d557d61003ba admin admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:22:36.228020 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cd934cc-75d4-4c9f-88b4-d557d61003ba admin admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:22:36.232738 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19fefd4e-9103-4434-837e-9ae694c6df5c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76c5181fc7be45279ac00017bb4640b3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:22:36.233321 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19fefd4e-9103-4434-837e-9ae694c6df5c admin admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:22:36.233595 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19fefd4e-9103-4434-837e-9ae694c6df5c admin admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:22:36.233860 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19fefd4e-9103-4434-837e-9ae694c6df5c admin admin] PATH_INFO: `/v3/roles` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:22:36.234480 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19fefd4e-9103-4434-837e-9ae694c6df5c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:22:36.236102 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19fefd4e-9103-4434-837e-9ae694c6df5c admin admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:22:36.246380 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 1044/2081] 10.222.0.18 () {58 vars in 1122 bytes} [Tue Sep 13 03:22:36 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 03:22:36.250223 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1b95b4f-5bf6-4d5b-b6ef-d5e0426d1312 admin admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:22:36.259918 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1b95b4f-5bf6-4d5b-b6ef-d5e0426d1312 admin admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:22:36.264455 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-579e1465-e745-4ddb-9043-e15e469a373d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76c5181fc7be45279ac00017bb4640b3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:22:36.265079 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-579e1465-e745-4ddb-9043-e15e469a373d admin admin] REQUEST_METHOD: `PUT` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:22:36.265352 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-579e1465-e745-4ddb-9043-e15e469a373d admin admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:22:36.265621 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-579e1465-e745-4ddb-9043-e15e469a373d admin admin] PATH_INFO: `/v3/projects/0d4a92c7151540778f6728ff90fb7114/users/e57c764c47e64e55b0e515f91973a414/roles/1eb7a5b5203945c29e6d22d653f47847` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:22:36.274534 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=85,keystone:DELETE=12,keystone:INSERT=3 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:22:36.282517 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-579e1465-e745-4ddb-9043-e15e469a373d admin admin] RBAC: Authorizing `identity:create_grant(project_id=0d4a92c7151540778f6728ff90fb7114, user_id=e57c764c47e64e55b0e515f91973a414, role_id=1eb7a5b5203945c29e6d22d653f47847)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:22:36.286046 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-579e1465-e745-4ddb-9043-e15e469a373d admin admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:22:36.301865 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 1038/2082] 10.222.0.18 () {60 vars in 1483 bytes} [Tue Sep 13 03:22:36 2022] PUT /identity/v3/projects/0d4a92c7151540778f6728ff90fb7114/users/e57c764c47e64e55b0e515f91973a414/roles/1eb7a5b5203945c29e6d22d653f47847 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 03:22:36.312853 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f39d88-f487-4189-b00e-03f7a19e5b60 None None] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:22:36.313142 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f39d88-f487-4189-b00e-03f7a19e5b60 None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:22:36.313464 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f39d88-f487-4189-b00e-03f7a19e5b60 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:22:36.326472 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=90,keystone:DELETE=6,keystone:INSERT=4 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:22:36.345593 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.auth.core [None req-37f39d88-f487-4189-b00e-03f7a19e5b60 None None] MFA Rules not processed for user `e57c764c47e64e55b0e515f91973a414`. Rule list: `[]` (Enabled: `True`). {{(pid=81399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 03:22:36.346829 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-37f39d88-f487-4189-b00e-03f7a19e5b60 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:22:36.413819 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 1045/2083] 10.222.0.18 () {58 vars in 958 bytes} [Tue Sep 13 03:22:36 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 03:22:36.425520 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-579e1465-e745-4ddb-9043-e15e469a373d admin admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:22:36.427429 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-579e1465-e745-4ddb-9043-e15e469a373d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:22:36.493018 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-579e1465-e745-4ddb-9043-e15e469a373d admin admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:22:36.496787 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-640b0e3c-1403-4329-ac75-16c8ea459db8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '924ee3f608d34a18b0ddc4695f102a56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b9d94f6a68f7470d9494f112175620dc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:22:36.497445 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-640b0e3c-1403-4329-ac75-16c8ea459db8 service cinder] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:22:36.497746 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-640b0e3c-1403-4329-ac75-16c8ea459db8 service cinder] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:22:36.498052 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-640b0e3c-1403-4329-ac75-16c8ea459db8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:22:36.507036 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-640b0e3c-1403-4329-ac75-16c8ea459db8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:22:36.508509 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-640b0e3c-1403-4329-ac75-16c8ea459db8 service cinder] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:22:36.517825 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 1039/2084] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Sep 13 03:22:36 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 03:22:36.544593 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37f39d88-f487-4189-b00e-03f7a19e5b60 None None] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:22:36.668216 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37f39d88-f487-4189-b00e-03f7a19e5b60 None None] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:22:36.675241 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-709ac97b-38ff-4830-b1e8-3dc54bbfd7e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76c5181fc7be45279ac00017bb4640b3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:22:36.677129 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-709ac97b-38ff-4830-b1e8-3dc54bbfd7e9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:22:36.677129 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-709ac97b-38ff-4830-b1e8-3dc54bbfd7e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:22:36.677409 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-709ac97b-38ff-4830-b1e8-3dc54bbfd7e9 admin admin] PATH_INFO: `/v3/users/e57c764c47e64e55b0e515f91973a414` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:22:36.678866 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-709ac97b-38ff-4830-b1e8-3dc54bbfd7e9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e57c764c47e64e55b0e515f91973a414)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:22:36.680892 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-709ac97b-38ff-4830-b1e8-3dc54bbfd7e9 admin admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:22:36.738903 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-709ac97b-38ff-4830-b1e8-3dc54bbfd7e9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e57c764c47e64e55b0e515f91973a414'} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:22:36.748595 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-709ac97b-38ff-4830-b1e8-3dc54bbfd7e9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e57c764c47e64e55b0e515f91973a414'} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:22:36.762011 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-709ac97b-38ff-4830-b1e8-3dc54bbfd7e9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e57c764c47e64e55b0e515f91973a414'} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:22:36.762710 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-709ac97b-38ff-4830-b1e8-3dc54bbfd7e9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e57c764c47e64e55b0e515f91973a414'} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:22:36.770441 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-709ac97b-38ff-4830-b1e8-3dc54bbfd7e9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e57c764c47e64e55b0e515f91973a414'} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:22:36.776438 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 1046/2085] 10.222.0.18 () {58 vars in 1224 bytes} [Tue Sep 13 03:22:36 2022] DELETE /identity/v3/users/e57c764c47e64e55b0e515f91973a414 => generated 0 bytes in 233 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 03:22:36.780959 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-836fae97-54cc-4ed6-bdf4-2f0c5405ae08 None None] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:22:36.781304 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-836fae97-54cc-4ed6-bdf4-2f0c5405ae08 None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:22:36.781641 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-836fae97-54cc-4ed6-bdf4-2f0c5405ae08 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:22:36.801287 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.auth.core [None req-836fae97-54cc-4ed6-bdf4-2f0c5405ae08 None None] MFA Rules not processed for user `49f280ab086b4610855c4ffa378517d9`. Rule list: `[]` (Enabled: `True`). {{(pid=81400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 03:22:36.802553 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-836fae97-54cc-4ed6-bdf4-2f0c5405ae08 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:22:36.862003 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 1040/2086] 10.222.0.18 () {58 vars in 958 bytes} [Tue Sep 13 03:22:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 13 03:22:36.871918 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-709ac97b-38ff-4830-b1e8-3dc54bbfd7e9 admin admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:22:36.873292 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-709ac97b-38ff-4830-b1e8-3dc54bbfd7e9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:22:36.892175 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:22:36.933030 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-709ac97b-38ff-4830-b1e8-3dc54bbfd7e9 admin admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:22:36.936728 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc2f545f-5f92-4980-bcea-9d1e699f6c94 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdffbc851086441ebd01022d1e4c2609', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b9d94f6a68f7470d9494f112175620dc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:22:36.937333 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc2f545f-5f92-4980-bcea-9d1e699f6c94 service neutron] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:22:36.937612 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc2f545f-5f92-4980-bcea-9d1e699f6c94 service neutron] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:22:36.937883 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc2f545f-5f92-4980-bcea-9d1e699f6c94 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:22:36.947546 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc2f545f-5f92-4980-bcea-9d1e699f6c94 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:22:36.948962 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc2f545f-5f92-4980-bcea-9d1e699f6c94 service neutron] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:22:36.958359 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 1047/2087] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Sep 13 03:22:36 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 03:22:37.225786 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-836fae97-54cc-4ed6-bdf4-2f0c5405ae08 None None] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:22:37.227241 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-836fae97-54cc-4ed6-bdf4-2f0c5405ae08 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:22:37.240671 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-836fae97-54cc-4ed6-bdf4-2f0c5405ae08 None None] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:22:37.245348 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40136513-7205-453c-80b6-17f25f7f0e39 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76c5181fc7be45279ac00017bb4640b3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:22:37.245981 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40136513-7205-453c-80b6-17f25f7f0e39 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:22:37.246259 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40136513-7205-453c-80b6-17f25f7f0e39 admin admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:22:37.246535 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40136513-7205-453c-80b6-17f25f7f0e39 admin admin] PATH_INFO: `/v3/projects/0d4a92c7151540778f6728ff90fb7114` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:22:37.247693 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40136513-7205-453c-80b6-17f25f7f0e39 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0d4a92c7151540778f6728ff90fb7114)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:22:37.249618 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40136513-7205-453c-80b6-17f25f7f0e39 admin admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:22:37.299171 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-40136513-7205-453c-80b6-17f25f7f0e39 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0d4a92c7151540778f6728ff90fb7114', 'request_id': 'req-40136513-7205-453c-80b6-17f25f7f0e39'} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:22:37.305449 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-40136513-7205-453c-80b6-17f25f7f0e39 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0d4a92c7151540778f6728ff90fb7114', 'request_id': 'req-40136513-7205-453c-80b6-17f25f7f0e39'} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:22:37.311652 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-40136513-7205-453c-80b6-17f25f7f0e39 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0d4a92c7151540778f6728ff90fb7114', 'request_id': 'req-40136513-7205-453c-80b6-17f25f7f0e39'} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:22:37.324394 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-40136513-7205-453c-80b6-17f25f7f0e39 admin admin] The token cache is being invalidate because project 0d4a92c7151540778f6728ff90fb7114 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81400) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 03:22:37.324701 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-40136513-7205-453c-80b6-17f25f7f0e39 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:22:37.325322 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-40136513-7205-453c-80b6-17f25f7f0e39 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:22:37.326822 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 1041/2088] 10.222.0.18 () {58 vars in 1233 bytes} [Tue Sep 13 03:22:37 2022] DELETE /identity/v3/projects/0d4a92c7151540778f6728ff90fb7114 => generated 0 bytes in 102 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 03:22:37.331772 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff4cefd-ae1c-40fc-b6cc-a18dc8544747 None None] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:22:37.332231 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff4cefd-ae1c-40fc-b6cc-a18dc8544747 None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:22:37.332354 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff4cefd-ae1c-40fc-b6cc-a18dc8544747 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:22:37.351893 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.auth.core [None req-0ff4cefd-ae1c-40fc-b6cc-a18dc8544747 None None] MFA Rules not processed for user `49f280ab086b4610855c4ffa378517d9`. Rule list: `[]` (Enabled: `True`). {{(pid=81399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 03:22:37.353085 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-0ff4cefd-ae1c-40fc-b6cc-a18dc8544747 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:22:37.411403 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 1048/2089] 10.222.0.18 () {58 vars in 958 bytes} [Tue Sep 13 03:22:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 03:22:37.419101 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd6ee667-e48a-4ba2-be60-631b7ce53d73 None None] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:22:37.419395 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd6ee667-e48a-4ba2-be60-631b7ce53d73 None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:22:37.419666 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd6ee667-e48a-4ba2-be60-631b7ce53d73 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:22:37.439427 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.auth.core [None req-cd6ee667-e48a-4ba2-be60-631b7ce53d73 None None] MFA Rules not processed for user `49f280ab086b4610855c4ffa378517d9`. Rule list: `[]` (Enabled: `True`). {{(pid=81400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 03:22:37.440611 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-cd6ee667-e48a-4ba2-be60-631b7ce53d73 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:22:37.448445 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 1042/2090] 10.222.0.18 () {58 vars in 958 bytes} [Tue Sep 13 03:22:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 03:22:37.452510 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ff4cefd-ae1c-40fc-b6cc-a18dc8544747 None None] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:22:37.462304 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ff4cefd-ae1c-40fc-b6cc-a18dc8544747 None None] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:22:37.466879 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb98a495-363e-40eb-8104-a892914e62a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76c5181fc7be45279ac00017bb4640b3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:22:37.467474 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb98a495-363e-40eb-8104-a892914e62a0 admin admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:22:37.467753 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb98a495-363e-40eb-8104-a892914e62a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:22:37.468025 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb98a495-363e-40eb-8104-a892914e62a0 admin admin] PATH_INFO: `/v3/domains` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:22:37.468689 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb98a495-363e-40eb-8104-a892914e62a0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 03:22:37.468988 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb98a495-363e-40eb-8104-a892914e62a0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:22:37.470579 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb98a495-363e-40eb-8104-a892914e62a0 admin admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:22:37.482390 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 1049/2091] 10.222.0.18 () {58 vars in 1153 bytes} [Tue Sep 13 03:22:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 03:22:37.486517 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd6ee667-e48a-4ba2-be60-631b7ce53d73 None None] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:22:37.496157 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd6ee667-e48a-4ba2-be60-631b7ce53d73 None None] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:22:37.500815 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4d2faad-4a46-4b48-805e-c9e31ba0cb54 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76c5181fc7be45279ac00017bb4640b3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:22:37.501393 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4d2faad-4a46-4b48-805e-c9e31ba0cb54 admin admin] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:22:37.501666 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4d2faad-4a46-4b48-805e-c9e31ba0cb54 admin admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:22:37.501928 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4d2faad-4a46-4b48-805e-c9e31ba0cb54 admin admin] PATH_INFO: `/v3/projects` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:22:37.502542 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4d2faad-4a46-4b48-805e-c9e31ba0cb54 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:22:37.504512 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4d2faad-4a46-4b48-805e-c9e31ba0cb54 admin admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:22:37.508869 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:22:37.531508 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 1043/2092] 10.222.0.18 () {60 vars in 1153 bytes} [Tue Sep 13 03:22:37 2022] POST /identity/v3/projects => generated 391 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 03:22:37.536153 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb98a495-363e-40eb-8104-a892914e62a0 admin admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:22:37.546372 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb98a495-363e-40eb-8104-a892914e62a0 admin admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:22:37.601648 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5d0e158-18ae-4148-9a06-ad341d6b2032 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76c5181fc7be45279ac00017bb4640b3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:22:37.602252 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d0e158-18ae-4148-9a06-ad341d6b2032 admin admin] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:22:37.602532 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d0e158-18ae-4148-9a06-ad341d6b2032 admin admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:22:37.602833 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d0e158-18ae-4148-9a06-ad341d6b2032 admin admin] PATH_INFO: `/v3/users` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:22:37.603452 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5d0e158-18ae-4148-9a06-ad341d6b2032 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:22:37.605386 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5d0e158-18ae-4148-9a06-ad341d6b2032 admin admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:22:37.626361 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 1050/2093] 10.222.0.18 () {60 vars in 1144 bytes} [Tue Sep 13 03:22:37 2022] POST /identity/v3/users => generated 296 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 13 03:22:37.630276 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4d2faad-4a46-4b48-805e-c9e31ba0cb54 admin admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:22:37.640481 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4d2faad-4a46-4b48-805e-c9e31ba0cb54 admin admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:22:37.645265 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4c5c3eb-46a6-4006-8586-79f60e8d265d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76c5181fc7be45279ac00017bb4640b3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:22:37.645931 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c5c3eb-46a6-4006-8586-79f60e8d265d admin admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:22:37.646276 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c5c3eb-46a6-4006-8586-79f60e8d265d admin admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:22:37.646632 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c5c3eb-46a6-4006-8586-79f60e8d265d admin admin] PATH_INFO: `/v3/roles` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:22:37.647394 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4c5c3eb-46a6-4006-8586-79f60e8d265d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:22:37.649042 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4c5c3eb-46a6-4006-8586-79f60e8d265d admin admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:22:37.658963 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 1044/2094] 10.222.0.18 () {58 vars in 1122 bytes} [Tue Sep 13 03:22:37 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 03:22:37.663489 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5d0e158-18ae-4148-9a06-ad341d6b2032 admin admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:22:37.673841 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5d0e158-18ae-4148-9a06-ad341d6b2032 admin admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:22:37.679409 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e6e1929-1b11-49cf-8e20-d0e24b3da690 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76c5181fc7be45279ac00017bb4640b3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:22:37.680033 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e6e1929-1b11-49cf-8e20-d0e24b3da690 admin admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:22:37.680337 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e6e1929-1b11-49cf-8e20-d0e24b3da690 admin admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:22:37.680656 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e6e1929-1b11-49cf-8e20-d0e24b3da690 admin admin] PATH_INFO: `/v3/roles` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:22:37.681355 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e6e1929-1b11-49cf-8e20-d0e24b3da690 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:22:37.682985 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e6e1929-1b11-49cf-8e20-d0e24b3da690 admin admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:22:37.693459 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 1051/2095] 10.222.0.18 () {58 vars in 1122 bytes} [Tue Sep 13 03:22:37 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 03:22:37.697396 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4c5c3eb-46a6-4006-8586-79f60e8d265d admin admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:22:37.707736 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4c5c3eb-46a6-4006-8586-79f60e8d265d admin admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:22:37.712262 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6da824d2-e793-4b47-9af7-770a98f59b15 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76c5181fc7be45279ac00017bb4640b3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:22:37.712893 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da824d2-e793-4b47-9af7-770a98f59b15 admin admin] REQUEST_METHOD: `PUT` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:22:37.713165 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da824d2-e793-4b47-9af7-770a98f59b15 admin admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:22:37.713430 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da824d2-e793-4b47-9af7-770a98f59b15 admin admin] PATH_INFO: `/v3/projects/4eefb4b6cab54c4aa25fa1928d3c8962/users/d009b5b585dc47c4b0bc5fd39e4b1cf1/roles/1eb7a5b5203945c29e6d22d653f47847` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:22:37.728836 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6da824d2-e793-4b47-9af7-770a98f59b15 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4eefb4b6cab54c4aa25fa1928d3c8962, user_id=d009b5b585dc47c4b0bc5fd39e4b1cf1, role_id=1eb7a5b5203945c29e6d22d653f47847)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:22:37.731784 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6da824d2-e793-4b47-9af7-770a98f59b15 admin admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:22:37.744983 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 1045/2096] 10.222.0.18 () {60 vars in 1483 bytes} [Tue Sep 13 03:22:37 2022] PUT /identity/v3/projects/4eefb4b6cab54c4aa25fa1928d3c8962/users/d009b5b585dc47c4b0bc5fd39e4b1cf1/roles/1eb7a5b5203945c29e6d22d653f47847 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 03:22:37.755142 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e02bbe5-3f63-4cd8-9c19-a279e48454b9 None None] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:22:37.755451 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e02bbe5-3f63-4cd8-9c19-a279e48454b9 None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:22:37.755723 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e02bbe5-3f63-4cd8-9c19-a279e48454b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:22:37.788124 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.auth.core [None req-7e02bbe5-3f63-4cd8-9c19-a279e48454b9 None None] MFA Rules not processed for user `d009b5b585dc47c4b0bc5fd39e4b1cf1`. Rule list: `[]` (Enabled: `True`). {{(pid=81399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 03:22:37.789337 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-7e02bbe5-3f63-4cd8-9c19-a279e48454b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:22:37.801939 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:22:37.860331 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 1052/2097] 10.222.0.18 () {58 vars in 958 bytes} [Tue Sep 13 03:22:37 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 03:22:37.867659 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6da824d2-e793-4b47-9af7-770a98f59b15 admin admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:22:37.869506 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-6da824d2-e793-4b47-9af7-770a98f59b15 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:22:37.928878 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6da824d2-e793-4b47-9af7-770a98f59b15 admin admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:22:37.932652 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b40c513c-07ed-4d5f-abfd-3963a17ea364 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '924ee3f608d34a18b0ddc4695f102a56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b9d94f6a68f7470d9494f112175620dc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:22:37.933253 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b40c513c-07ed-4d5f-abfd-3963a17ea364 service cinder] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:22:37.933528 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b40c513c-07ed-4d5f-abfd-3963a17ea364 service cinder] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:22:37.933792 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b40c513c-07ed-4d5f-abfd-3963a17ea364 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:22:37.942238 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b40c513c-07ed-4d5f-abfd-3963a17ea364 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:22:37.943744 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b40c513c-07ed-4d5f-abfd-3963a17ea364 service cinder] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:22:37.952465 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 1046/2098] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Sep 13 03:22:37 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 03:22:47.853899 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=21 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:22:47.948456 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=47,keystone:INSERT=2 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:23:05.783872 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e02bbe5-3f63-4cd8-9c19-a279e48454b9 None None] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:23:05.785470 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-7e02bbe5-3f63-4cd8-9c19-a279e48454b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:23:05.854881 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e02bbe5-3f63-4cd8-9c19-a279e48454b9 None None] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:23:05.859690 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82de5a31-c3dd-4482-837b-6d10dd757ed7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76c5181fc7be45279ac00017bb4640b3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:23:05.860320 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82de5a31-c3dd-4482-837b-6d10dd757ed7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:23:05.860598 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82de5a31-c3dd-4482-837b-6d10dd757ed7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:23:05.860868 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82de5a31-c3dd-4482-837b-6d10dd757ed7 admin admin] PATH_INFO: `/v3/users/0b030a8e1147430c83093300f5599fc7` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:23:05.862015 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82de5a31-c3dd-4482-837b-6d10dd757ed7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0b030a8e1147430c83093300f5599fc7)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:23:05.863932 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82de5a31-c3dd-4482-837b-6d10dd757ed7 admin admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:23:05.913653 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-82de5a31-c3dd-4482-837b-6d10dd757ed7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0b030a8e1147430c83093300f5599fc7'} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:23:05.922186 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-82de5a31-c3dd-4482-837b-6d10dd757ed7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0b030a8e1147430c83093300f5599fc7'} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:23:05.934998 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-82de5a31-c3dd-4482-837b-6d10dd757ed7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0b030a8e1147430c83093300f5599fc7'} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:23:05.935667 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-82de5a31-c3dd-4482-837b-6d10dd757ed7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0b030a8e1147430c83093300f5599fc7'} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:23:05.942701 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-82de5a31-c3dd-4482-837b-6d10dd757ed7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0b030a8e1147430c83093300f5599fc7'} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:23:05.948761 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 1053/2099] 10.222.0.18 () {58 vars in 1224 bytes} [Tue Sep 13 03:23:05 2022] DELETE /identity/v3/users/0b030a8e1147430c83093300f5599fc7 => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 03:23:05.953056 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9874064-bac0-4dd0-8812-b28810ef0728 None None] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:23:05.953526 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9874064-bac0-4dd0-8812-b28810ef0728 None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:23:05.953620 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9874064-bac0-4dd0-8812-b28810ef0728 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:23:05.973958 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.auth.core [None req-d9874064-bac0-4dd0-8812-b28810ef0728 None None] MFA Rules not processed for user `49f280ab086b4610855c4ffa378517d9`. Rule list: `[]` (Enabled: `True`). {{(pid=81400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 03:23:05.975752 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-d9874064-bac0-4dd0-8812-b28810ef0728 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:23:06.044718 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 1047/2100] 10.222.0.18 () {58 vars in 958 bytes} [Tue Sep 13 03:23:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 03:23:06.055173 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82de5a31-c3dd-4482-837b-6d10dd757ed7 admin admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:23:06.056639 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-82de5a31-c3dd-4482-837b-6d10dd757ed7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:23:06.160792 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82de5a31-c3dd-4482-837b-6d10dd757ed7 admin admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:23:06.163363 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fc6f85c-4b29-400e-82c7-8b2255e5b6a8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdffbc851086441ebd01022d1e4c2609', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b9d94f6a68f7470d9494f112175620dc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:23:06.164163 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc6f85c-4b29-400e-82c7-8b2255e5b6a8 service neutron] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:23:06.164467 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc6f85c-4b29-400e-82c7-8b2255e5b6a8 service neutron] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:23:06.164736 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc6f85c-4b29-400e-82c7-8b2255e5b6a8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:23:06.176688 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fc6f85c-4b29-400e-82c7-8b2255e5b6a8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:23:06.178269 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fc6f85c-4b29-400e-82c7-8b2255e5b6a8 service neutron] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:23:06.192470 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 1054/2101] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Sep 13 03:23:06 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 138 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 13 03:23:06.492740 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9874064-bac0-4dd0-8812-b28810ef0728 None None] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:23:06.494439 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-d9874064-bac0-4dd0-8812-b28810ef0728 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:23:06.509049 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9874064-bac0-4dd0-8812-b28810ef0728 None None] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:23:06.513121 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1854ca79-53b9-4d5f-9a25-6fc7fc115016 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76c5181fc7be45279ac00017bb4640b3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:23:06.513892 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1854ca79-53b9-4d5f-9a25-6fc7fc115016 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:23:06.514228 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1854ca79-53b9-4d5f-9a25-6fc7fc115016 admin admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:23:06.514563 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1854ca79-53b9-4d5f-9a25-6fc7fc115016 admin admin] PATH_INFO: `/v3/projects/f9cacf3bdc8f43fe9cd2ae60a8cb4e82` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:23:06.515956 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1854ca79-53b9-4d5f-9a25-6fc7fc115016 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f9cacf3bdc8f43fe9cd2ae60a8cb4e82)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:23:06.517908 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1854ca79-53b9-4d5f-9a25-6fc7fc115016 admin admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:23:06.573351 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-1854ca79-53b9-4d5f-9a25-6fc7fc115016 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f9cacf3bdc8f43fe9cd2ae60a8cb4e82', 'request_id': 'req-1854ca79-53b9-4d5f-9a25-6fc7fc115016'} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:23:06.579791 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-1854ca79-53b9-4d5f-9a25-6fc7fc115016 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f9cacf3bdc8f43fe9cd2ae60a8cb4e82', 'request_id': 'req-1854ca79-53b9-4d5f-9a25-6fc7fc115016'} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:23:06.587111 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-1854ca79-53b9-4d5f-9a25-6fc7fc115016 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f9cacf3bdc8f43fe9cd2ae60a8cb4e82', 'request_id': 'req-1854ca79-53b9-4d5f-9a25-6fc7fc115016'} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:23:06.602217 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-1854ca79-53b9-4d5f-9a25-6fc7fc115016 admin admin] The token cache is being invalidate because project f9cacf3bdc8f43fe9cd2ae60a8cb4e82 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81400) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 03:23:06.602499 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-1854ca79-53b9-4d5f-9a25-6fc7fc115016 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:23:06.603277 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-1854ca79-53b9-4d5f-9a25-6fc7fc115016 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:23:06.604839 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 1048/2102] 10.222.0.18 () {58 vars in 1233 bytes} [Tue Sep 13 03:23:06 2022] DELETE /identity/v3/projects/f9cacf3bdc8f43fe9cd2ae60a8cb4e82 => generated 0 bytes in 113 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 03:23:06.609745 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fc6f85c-4b29-400e-82c7-8b2255e5b6a8 service neutron] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:23:06.621816 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fc6f85c-4b29-400e-82c7-8b2255e5b6a8 service neutron] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:23:06.643972 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:23:06.698536 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db4b78eb-0925-482a-b081-ac6247e13d8b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76c5181fc7be45279ac00017bb4640b3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:23:06.699762 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db4b78eb-0925-482a-b081-ac6247e13d8b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:23:06.699762 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db4b78eb-0925-482a-b081-ac6247e13d8b admin admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:23:06.699992 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db4b78eb-0925-482a-b081-ac6247e13d8b admin admin] PATH_INFO: `/v3/users/e20dec95f803485bb1e04a61e456d24a` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:23:06.701132 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db4b78eb-0925-482a-b081-ac6247e13d8b admin admin] RBAC: Authorizing `identity:delete_user(user_id=e20dec95f803485bb1e04a61e456d24a)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:23:06.703735 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db4b78eb-0925-482a-b081-ac6247e13d8b admin admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:23:06.755779 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-db4b78eb-0925-482a-b081-ac6247e13d8b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e20dec95f803485bb1e04a61e456d24a'} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:23:06.764745 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-db4b78eb-0925-482a-b081-ac6247e13d8b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e20dec95f803485bb1e04a61e456d24a'} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:23:06.778847 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-db4b78eb-0925-482a-b081-ac6247e13d8b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e20dec95f803485bb1e04a61e456d24a'} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:23:06.779594 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-db4b78eb-0925-482a-b081-ac6247e13d8b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e20dec95f803485bb1e04a61e456d24a'} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:23:06.786602 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-db4b78eb-0925-482a-b081-ac6247e13d8b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e20dec95f803485bb1e04a61e456d24a'} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:23:06.792550 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 1055/2103] 10.222.0.18 () {58 vars in 1224 bytes} [Tue Sep 13 03:23:06 2022] DELETE /identity/v3/users/e20dec95f803485bb1e04a61e456d24a => generated 0 bytes in 184 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 03:23:07.118454 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1854ca79-53b9-4d5f-9a25-6fc7fc115016 admin admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:23:07.119918 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-1854ca79-53b9-4d5f-9a25-6fc7fc115016 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:23:07.133648 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1854ca79-53b9-4d5f-9a25-6fc7fc115016 admin admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:23:07.190068 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e96ccb5e-2c6f-4e55-8b56-3bcfb95ab5df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76c5181fc7be45279ac00017bb4640b3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:23:07.190717 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e96ccb5e-2c6f-4e55-8b56-3bcfb95ab5df admin admin] REQUEST_METHOD: `DELETE` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:23:07.190994 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e96ccb5e-2c6f-4e55-8b56-3bcfb95ab5df admin admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:23:07.191263 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e96ccb5e-2c6f-4e55-8b56-3bcfb95ab5df admin admin] PATH_INFO: `/v3/projects/19bbaf664bef43129e553015baad9ca0` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:23:07.192359 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e96ccb5e-2c6f-4e55-8b56-3bcfb95ab5df admin admin] RBAC: Authorizing `identity:delete_project(project_id=19bbaf664bef43129e553015baad9ca0)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:23:07.194847 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e96ccb5e-2c6f-4e55-8b56-3bcfb95ab5df admin admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:23:07.223964 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:23:07.243392 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-e96ccb5e-2c6f-4e55-8b56-3bcfb95ab5df admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '19bbaf664bef43129e553015baad9ca0', 'request_id': 'req-e96ccb5e-2c6f-4e55-8b56-3bcfb95ab5df'} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:23:07.249073 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-e96ccb5e-2c6f-4e55-8b56-3bcfb95ab5df admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '19bbaf664bef43129e553015baad9ca0', 'request_id': 'req-e96ccb5e-2c6f-4e55-8b56-3bcfb95ab5df'} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:23:07.255339 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-e96ccb5e-2c6f-4e55-8b56-3bcfb95ab5df admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '19bbaf664bef43129e553015baad9ca0', 'request_id': 'req-e96ccb5e-2c6f-4e55-8b56-3bcfb95ab5df'} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:23:07.268470 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-e96ccb5e-2c6f-4e55-8b56-3bcfb95ab5df admin admin] The token cache is being invalidate because project 19bbaf664bef43129e553015baad9ca0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81400) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 03:23:07.268746 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-e96ccb5e-2c6f-4e55-8b56-3bcfb95ab5df admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:23:07.269357 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-e96ccb5e-2c6f-4e55-8b56-3bcfb95ab5df admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:23:07.270863 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 1049/2104] 10.222.0.18 () {58 vars in 1233 bytes} [Tue Sep 13 03:23:07 2022] DELETE /identity/v3/projects/19bbaf664bef43129e553015baad9ca0 => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 13 03:23:07.275697 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eecb0678-3f9d-43a7-83c0-59c20c20af61 None None] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:23:07.275987 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eecb0678-3f9d-43a7-83c0-59c20c20af61 None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:23:07.276261 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eecb0678-3f9d-43a7-83c0-59c20c20af61 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:23:07.296535 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.auth.core [None req-eecb0678-3f9d-43a7-83c0-59c20c20af61 None None] MFA Rules not processed for user `49f280ab086b4610855c4ffa378517d9`. Rule list: `[]` (Enabled: `True`). {{(pid=81399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 03:23:07.297656 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-eecb0678-3f9d-43a7-83c0-59c20c20af61 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:23:07.372356 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 1056/2105] 10.222.0.18 () {58 vars in 958 bytes} [Tue Sep 13 03:23:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 03:23:07.374235 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c0e6c1-9aa2-40a9-b166-8aef8f11a4d3 None None] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:23:07.374971 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c0e6c1-9aa2-40a9-b166-8aef8f11a4d3 None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:23:07.375460 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c0e6c1-9aa2-40a9-b166-8aef8f11a4d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:23:07.403362 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.auth.core [None req-e5c0e6c1-9aa2-40a9-b166-8aef8f11a4d3 None None] MFA Rules not processed for user `49f280ab086b4610855c4ffa378517d9`. Rule list: `[]` (Enabled: `True`). {{(pid=81400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 03:23:07.404762 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-e5c0e6c1-9aa2-40a9-b166-8aef8f11a4d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:23:07.425003 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 1050/2106] 10.222.0.18 () {58 vars in 958 bytes} [Tue Sep 13 03:23:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 52 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 13 03:23:07.429699 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eecb0678-3f9d-43a7-83c0-59c20c20af61 None None] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:23:07.439339 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eecb0678-3f9d-43a7-83c0-59c20c20af61 None None] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:23:07.444292 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b75032e0-6da8-4c94-ae4f-6d502645f0db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76c5181fc7be45279ac00017bb4640b3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:23:07.444879 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b75032e0-6da8-4c94-ae4f-6d502645f0db admin admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:23:07.445153 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b75032e0-6da8-4c94-ae4f-6d502645f0db admin admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:23:07.445420 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b75032e0-6da8-4c94-ae4f-6d502645f0db admin admin] PATH_INFO: `/v3/domains` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:23:07.446052 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b75032e0-6da8-4c94-ae4f-6d502645f0db admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 03:23:07.446372 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b75032e0-6da8-4c94-ae4f-6d502645f0db admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:23:07.447986 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b75032e0-6da8-4c94-ae4f-6d502645f0db admin admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:23:07.459992 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 1057/2107] 10.222.0.18 () {58 vars in 1153 bytes} [Tue Sep 13 03:23:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 03:23:07.464095 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5c0e6c1-9aa2-40a9-b166-8aef8f11a4d3 None None] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:23:07.473229 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5c0e6c1-9aa2-40a9-b166-8aef8f11a4d3 None None] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:23:07.477830 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cd131c2-f1b3-49f0-b1bb-8e5cf61fd2a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76c5181fc7be45279ac00017bb4640b3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:23:07.478409 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd131c2-f1b3-49f0-b1bb-8e5cf61fd2a7 admin admin] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:23:07.478726 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd131c2-f1b3-49f0-b1bb-8e5cf61fd2a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:23:07.478997 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd131c2-f1b3-49f0-b1bb-8e5cf61fd2a7 admin admin] PATH_INFO: `/v3/projects` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:23:07.479618 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cd131c2-f1b3-49f0-b1bb-8e5cf61fd2a7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:23:07.481552 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cd131c2-f1b3-49f0-b1bb-8e5cf61fd2a7 admin admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:23:07.508991 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 1051/2108] 10.222.0.18 () {60 vars in 1153 bytes} [Tue Sep 13 03:23:07 2022] POST /identity/v3/projects => generated 379 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 03:23:07.513179 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b75032e0-6da8-4c94-ae4f-6d502645f0db admin admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:23:07.523603 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b75032e0-6da8-4c94-ae4f-6d502645f0db admin admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:23:07.548653 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:23:07.579304 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0359185d-a284-491d-8d2e-e65e10a47cf5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76c5181fc7be45279ac00017bb4640b3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:23:07.579896 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0359185d-a284-491d-8d2e-e65e10a47cf5 admin admin] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:23:07.580171 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0359185d-a284-491d-8d2e-e65e10a47cf5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:23:07.580437 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0359185d-a284-491d-8d2e-e65e10a47cf5 admin admin] PATH_INFO: `/v3/users` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:23:07.581050 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0359185d-a284-491d-8d2e-e65e10a47cf5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:23:07.582996 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0359185d-a284-491d-8d2e-e65e10a47cf5 admin admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:23:07.604925 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 1058/2109] 10.222.0.18 () {60 vars in 1144 bytes} [Tue Sep 13 03:23:07 2022] POST /identity/v3/users => generated 290 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 03:23:07.609591 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cd131c2-f1b3-49f0-b1bb-8e5cf61fd2a7 admin admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:23:07.620375 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cd131c2-f1b3-49f0-b1bb-8e5cf61fd2a7 admin admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:23:07.625008 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9146ce9-bfc0-425c-bd8e-fa7a3b7d080a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76c5181fc7be45279ac00017bb4640b3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:23:07.625647 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9146ce9-bfc0-425c-bd8e-fa7a3b7d080a admin admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:23:07.625935 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9146ce9-bfc0-425c-bd8e-fa7a3b7d080a admin admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:23:07.626200 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9146ce9-bfc0-425c-bd8e-fa7a3b7d080a admin admin] PATH_INFO: `/v3/roles` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:23:07.626867 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9146ce9-bfc0-425c-bd8e-fa7a3b7d080a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:23:07.628641 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9146ce9-bfc0-425c-bd8e-fa7a3b7d080a admin admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:23:07.640418 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 1052/2110] 10.222.0.18 () {58 vars in 1122 bytes} [Tue Sep 13 03:23:07 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 03:23:07.647841 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0359185d-a284-491d-8d2e-e65e10a47cf5 admin admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:23:07.660063 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0359185d-a284-491d-8d2e-e65e10a47cf5 admin admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:23:07.667074 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11f156d7-f42f-4676-b294-0db8c77eafd7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76c5181fc7be45279ac00017bb4640b3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:23:07.667686 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f156d7-f42f-4676-b294-0db8c77eafd7 admin admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:23:07.668002 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f156d7-f42f-4676-b294-0db8c77eafd7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:23:07.668276 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f156d7-f42f-4676-b294-0db8c77eafd7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:23:07.669171 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11f156d7-f42f-4676-b294-0db8c77eafd7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:23:07.671009 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11f156d7-f42f-4676-b294-0db8c77eafd7 admin admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:23:07.687516 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 1059/2111] 10.222.0.18 () {58 vars in 1122 bytes} [Tue Sep 13 03:23:07 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 03:23:07.692023 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9146ce9-bfc0-425c-bd8e-fa7a3b7d080a admin admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:23:07.706285 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9146ce9-bfc0-425c-bd8e-fa7a3b7d080a admin admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:23:07.715137 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55b212a0-5167-4224-9943-7a8d89740174 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76c5181fc7be45279ac00017bb4640b3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:23:07.715830 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b212a0-5167-4224-9943-7a8d89740174 admin admin] REQUEST_METHOD: `PUT` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:23:07.716111 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b212a0-5167-4224-9943-7a8d89740174 admin admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:23:07.716383 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b212a0-5167-4224-9943-7a8d89740174 admin admin] PATH_INFO: `/v3/projects/fe45a82144ee4028883d2826bdf8866c/users/711e6da793574b979ec0cd2c7e343456/roles/1eb7a5b5203945c29e6d22d653f47847` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:23:07.732100 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55b212a0-5167-4224-9943-7a8d89740174 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fe45a82144ee4028883d2826bdf8866c, user_id=711e6da793574b979ec0cd2c7e343456, role_id=1eb7a5b5203945c29e6d22d653f47847)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:23:07.734821 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55b212a0-5167-4224-9943-7a8d89740174 admin admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:23:07.748154 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 1053/2112] 10.222.0.18 () {60 vars in 1483 bytes} [Tue Sep 13 03:23:07 2022] PUT /identity/v3/projects/fe45a82144ee4028883d2826bdf8866c/users/711e6da793574b979ec0cd2c7e343456/roles/1eb7a5b5203945c29e6d22d653f47847 => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 03:23:07.760234 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6433906-c547-42fe-8ae5-81bd931a8f18 None None] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:23:07.760549 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6433906-c547-42fe-8ae5-81bd931a8f18 None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:23:07.760828 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6433906-c547-42fe-8ae5-81bd931a8f18 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:23:07.787912 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.auth.core [None req-a6433906-c547-42fe-8ae5-81bd931a8f18 None None] MFA Rules not processed for user `711e6da793574b979ec0cd2c7e343456`. Rule list: `[]` (Enabled: `True`). {{(pid=81399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 03:23:07.789139 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-a6433906-c547-42fe-8ae5-81bd931a8f18 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:23:07.855096 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 1060/2113] 10.222.0.18 () {58 vars in 958 bytes} [Tue Sep 13 03:23:07 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 03:23:07.859500 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55b212a0-5167-4224-9943-7a8d89740174 admin admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:23:07.922123 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55b212a0-5167-4224-9943-7a8d89740174 admin admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:23:07.926884 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef45948e-841c-4b2e-b2c8-6ac7548bc132 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76c5181fc7be45279ac00017bb4640b3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:23:07.927541 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef45948e-841c-4b2e-b2c8-6ac7548bc132 admin admin] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:23:07.928334 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef45948e-841c-4b2e-b2c8-6ac7548bc132 admin admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:23:07.928608 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef45948e-841c-4b2e-b2c8-6ac7548bc132 admin admin] PATH_INFO: `/v3/projects` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:23:07.929249 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef45948e-841c-4b2e-b2c8-6ac7548bc132 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:23:07.931203 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef45948e-841c-4b2e-b2c8-6ac7548bc132 admin admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:23:07.957543 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 1054/2114] 10.222.0.18 () {60 vars in 1153 bytes} [Tue Sep 13 03:23:07 2022] POST /identity/v3/projects => generated 377 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 03:23:07.961623 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6433906-c547-42fe-8ae5-81bd931a8f18 None None] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:23:07.972393 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6433906-c547-42fe-8ae5-81bd931a8f18 None None] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:23:08.031188 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61b33764-646d-48f7-aacc-02539b6b1873 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76c5181fc7be45279ac00017bb4640b3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:23:08.031786 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b33764-646d-48f7-aacc-02539b6b1873 admin admin] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:23:08.032064 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b33764-646d-48f7-aacc-02539b6b1873 admin admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:23:08.032332 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b33764-646d-48f7-aacc-02539b6b1873 admin admin] PATH_INFO: `/v3/users` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:23:08.032947 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61b33764-646d-48f7-aacc-02539b6b1873 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:23:08.034907 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61b33764-646d-48f7-aacc-02539b6b1873 admin admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:23:08.056400 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 1061/2115] 10.222.0.18 () {60 vars in 1144 bytes} [Tue Sep 13 03:23:07 2022] POST /identity/v3/users => generated 289 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 03:23:08.060569 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef45948e-841c-4b2e-b2c8-6ac7548bc132 admin admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:23:08.071146 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef45948e-841c-4b2e-b2c8-6ac7548bc132 admin admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:23:08.075814 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b99bdb36-53df-4feb-8c43-a52bbc12047e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76c5181fc7be45279ac00017bb4640b3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:23:08.076416 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b99bdb36-53df-4feb-8c43-a52bbc12047e admin admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:23:08.076695 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b99bdb36-53df-4feb-8c43-a52bbc12047e admin admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:23:08.076967 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b99bdb36-53df-4feb-8c43-a52bbc12047e admin admin] PATH_INFO: `/v3/roles` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:23:08.077601 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b99bdb36-53df-4feb-8c43-a52bbc12047e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:23:08.079280 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b99bdb36-53df-4feb-8c43-a52bbc12047e admin admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:23:08.089795 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 1055/2116] 10.222.0.18 () {58 vars in 1122 bytes} [Tue Sep 13 03:23:08 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 03:23:08.093835 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61b33764-646d-48f7-aacc-02539b6b1873 admin admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:23:08.104261 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61b33764-646d-48f7-aacc-02539b6b1873 admin admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:23:08.108904 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea083fa9-c437-48c5-8a91-60e7a3a31a6f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76c5181fc7be45279ac00017bb4640b3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:23:08.109475 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea083fa9-c437-48c5-8a91-60e7a3a31a6f admin admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:23:08.109751 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea083fa9-c437-48c5-8a91-60e7a3a31a6f admin admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:23:08.110065 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea083fa9-c437-48c5-8a91-60e7a3a31a6f admin admin] PATH_INFO: `/v3/roles` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:23:08.110950 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea083fa9-c437-48c5-8a91-60e7a3a31a6f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:23:08.112560 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea083fa9-c437-48c5-8a91-60e7a3a31a6f admin admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:23:08.123975 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 1062/2117] 10.222.0.18 () {58 vars in 1122 bytes} [Tue Sep 13 03:23:08 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 03:23:08.127947 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b99bdb36-53df-4feb-8c43-a52bbc12047e admin admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:23:08.138110 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b99bdb36-53df-4feb-8c43-a52bbc12047e admin admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:23:08.142800 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dc45bb6-ee8f-468a-95f6-a8305b06092c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76c5181fc7be45279ac00017bb4640b3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:23:08.143443 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dc45bb6-ee8f-468a-95f6-a8305b06092c admin admin] REQUEST_METHOD: `PUT` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:23:08.143722 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dc45bb6-ee8f-468a-95f6-a8305b06092c admin admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:23:08.143992 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dc45bb6-ee8f-468a-95f6-a8305b06092c admin admin] PATH_INFO: `/v3/projects/f7e427a44973492ca0afd6f53f868eb0/users/b3dbf60e414b4aac8811d223b40f0772/roles/1eb7a5b5203945c29e6d22d653f47847` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:23:08.158450 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dc45bb6-ee8f-468a-95f6-a8305b06092c admin admin] RBAC: Authorizing `identity:create_grant(project_id=f7e427a44973492ca0afd6f53f868eb0, user_id=b3dbf60e414b4aac8811d223b40f0772, role_id=1eb7a5b5203945c29e6d22d653f47847)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:23:08.161203 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dc45bb6-ee8f-468a-95f6-a8305b06092c admin admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:23:08.171165 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:DELETE=6,keystone:SELECT=89,keystone:INSERT=5 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:23:08.176454 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 1056/2118] 10.222.0.18 () {60 vars in 1483 bytes} [Tue Sep 13 03:23:08 2022] PUT /identity/v3/projects/f7e427a44973492ca0afd6f53f868eb0/users/b3dbf60e414b4aac8811d223b40f0772/roles/1eb7a5b5203945c29e6d22d653f47847 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 03:23:08.187210 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-604ccb4f-7472-4271-a124-0804dbe3f955 None None] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:23:08.187501 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-604ccb4f-7472-4271-a124-0804dbe3f955 None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:23:08.187777 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-604ccb4f-7472-4271-a124-0804dbe3f955 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:23:08.215747 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.auth.core [None req-604ccb4f-7472-4271-a124-0804dbe3f955 None None] MFA Rules not processed for user `b3dbf60e414b4aac8811d223b40f0772`. Rule list: `[]` (Enabled: `True`). {{(pid=81399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 03:23:08.217031 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-604ccb4f-7472-4271-a124-0804dbe3f955 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:23:08.220988 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:23:08.282870 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 1063/2119] 10.222.0.18 () {58 vars in 958 bytes} [Tue Sep 13 03:23:08 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 03:23:08.287969 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dc45bb6-ee8f-468a-95f6-a8305b06092c admin admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:23:08.351896 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dc45bb6-ee8f-468a-95f6-a8305b06092c admin admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:23:08.356564 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-703e4892-0787-47c5-b635-a39030f3fb47 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76c5181fc7be45279ac00017bb4640b3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:23:08.357169 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-703e4892-0787-47c5-b635-a39030f3fb47 admin admin] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:23:08.357447 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-703e4892-0787-47c5-b635-a39030f3fb47 admin admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:23:08.357719 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-703e4892-0787-47c5-b635-a39030f3fb47 admin admin] PATH_INFO: `/v3/projects` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:23:08.358357 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-703e4892-0787-47c5-b635-a39030f3fb47 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:23:08.360514 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-703e4892-0787-47c5-b635-a39030f3fb47 admin admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:23:08.386833 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 1057/2120] 10.222.0.18 () {60 vars in 1153 bytes} [Tue Sep 13 03:23:08 2022] POST /identity/v3/projects => generated 379 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 03:23:08.390848 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-604ccb4f-7472-4271-a124-0804dbe3f955 None None] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:23:08.401746 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-604ccb4f-7472-4271-a124-0804dbe3f955 None None] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:23:08.475824 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cfb97e7-2dcd-45cd-bee5-3e2add06aca6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76c5181fc7be45279ac00017bb4640b3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:23:08.476590 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cfb97e7-2dcd-45cd-bee5-3e2add06aca6 admin admin] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:23:08.477111 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cfb97e7-2dcd-45cd-bee5-3e2add06aca6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:23:08.477431 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cfb97e7-2dcd-45cd-bee5-3e2add06aca6 admin admin] PATH_INFO: `/v3/users` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:23:08.478174 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cfb97e7-2dcd-45cd-bee5-3e2add06aca6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:23:08.480316 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cfb97e7-2dcd-45cd-bee5-3e2add06aca6 admin admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:23:08.506785 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 1064/2121] 10.222.0.18 () {60 vars in 1144 bytes} [Tue Sep 13 03:23:08 2022] POST /identity/v3/users => generated 296 bytes in 116 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 03:23:08.510977 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-703e4892-0787-47c5-b635-a39030f3fb47 admin admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:23:08.526072 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-703e4892-0787-47c5-b635-a39030f3fb47 admin admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:23:08.532561 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a569ca26-6e05-4734-b145-f8159557c0ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76c5181fc7be45279ac00017bb4640b3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:23:08.533166 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a569ca26-6e05-4734-b145-f8159557c0ec admin admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:23:08.533438 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a569ca26-6e05-4734-b145-f8159557c0ec admin admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:23:08.533757 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a569ca26-6e05-4734-b145-f8159557c0ec admin admin] PATH_INFO: `/v3/roles` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:23:08.534468 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a569ca26-6e05-4734-b145-f8159557c0ec admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:23:08.535969 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a569ca26-6e05-4734-b145-f8159557c0ec admin admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:23:08.549315 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 1058/2122] 10.222.0.18 () {58 vars in 1122 bytes} [Tue Sep 13 03:23:08 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 13 03:23:08.554327 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cfb97e7-2dcd-45cd-bee5-3e2add06aca6 admin admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:23:08.564372 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cfb97e7-2dcd-45cd-bee5-3e2add06aca6 admin admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:23:08.568988 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62abde1d-2112-429d-9be7-166c7d724733 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76c5181fc7be45279ac00017bb4640b3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:23:08.569635 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62abde1d-2112-429d-9be7-166c7d724733 admin admin] REQUEST_METHOD: `PUT` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:23:08.570019 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62abde1d-2112-429d-9be7-166c7d724733 admin admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:23:08.570297 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62abde1d-2112-429d-9be7-166c7d724733 admin admin] PATH_INFO: `/v3/projects/c73c328b3bf34d65b32928e78e7e4a43/users/ca617ad9615f47f99a41ba0e6c4a16b8/roles/425d4b481c7846e0bc07e7e805d5b15f` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:23:08.586076 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62abde1d-2112-429d-9be7-166c7d724733 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c73c328b3bf34d65b32928e78e7e4a43, user_id=ca617ad9615f47f99a41ba0e6c4a16b8, role_id=425d4b481c7846e0bc07e7e805d5b15f)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:23:08.588779 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62abde1d-2112-429d-9be7-166c7d724733 admin admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:23:08.604102 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 1065/2123] 10.222.0.18 () {60 vars in 1483 bytes} [Tue Sep 13 03:23:08 2022] PUT /identity/v3/projects/c73c328b3bf34d65b32928e78e7e4a43/users/ca617ad9615f47f99a41ba0e6c4a16b8/roles/425d4b481c7846e0bc07e7e805d5b15f => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 13 03:23:08.608362 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a569ca26-6e05-4734-b145-f8159557c0ec admin admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:23:08.618605 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a569ca26-6e05-4734-b145-f8159557c0ec admin admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:23:08.676229 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94aade83-caa0-45ec-9ddc-a8dcddef6bce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76c5181fc7be45279ac00017bb4640b3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:23:08.677532 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94aade83-caa0-45ec-9ddc-a8dcddef6bce admin admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:23:08.678019 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94aade83-caa0-45ec-9ddc-a8dcddef6bce admin admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:23:08.678346 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94aade83-caa0-45ec-9ddc-a8dcddef6bce admin admin] PATH_INFO: `/v3/roles` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:23:08.679096 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94aade83-caa0-45ec-9ddc-a8dcddef6bce admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:23:08.680931 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94aade83-caa0-45ec-9ddc-a8dcddef6bce admin admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:23:08.694722 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 1059/2124] 10.222.0.18 () {58 vars in 1122 bytes} [Tue Sep 13 03:23:08 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 03:23:08.696447 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62abde1d-2112-429d-9be7-166c7d724733 admin admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:23:08.706827 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62abde1d-2112-429d-9be7-166c7d724733 admin admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:23:08.711981 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e01513b-9e7f-4b75-9cb7-188647ddd55f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76c5181fc7be45279ac00017bb4640b3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:23:08.712630 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e01513b-9e7f-4b75-9cb7-188647ddd55f admin admin] REQUEST_METHOD: `PUT` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:23:08.712910 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e01513b-9e7f-4b75-9cb7-188647ddd55f admin admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:23:08.713198 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e01513b-9e7f-4b75-9cb7-188647ddd55f admin admin] PATH_INFO: `/v3/projects/c73c328b3bf34d65b32928e78e7e4a43/users/ca617ad9615f47f99a41ba0e6c4a16b8/roles/425d4b481c7846e0bc07e7e805d5b15f` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:23:08.715147 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e01513b-9e7f-4b75-9cb7-188647ddd55f admin admin] RBAC: Authorizing `identity:create_grant(project_id=c73c328b3bf34d65b32928e78e7e4a43, user_id=ca617ad9615f47f99a41ba0e6c4a16b8, role_id=425d4b481c7846e0bc07e7e805d5b15f)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:23:08.717779 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e01513b-9e7f-4b75-9cb7-188647ddd55f admin admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:23:08.728674 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 1066/2125] 10.222.0.18 () {60 vars in 1483 bytes} [Tue Sep 13 03:23:08 2022] PUT /identity/v3/projects/c73c328b3bf34d65b32928e78e7e4a43/users/ca617ad9615f47f99a41ba0e6c4a16b8/roles/425d4b481c7846e0bc07e7e805d5b15f => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 03:23:08.739063 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa08541-d389-436b-a3e2-5a73b231c590 None None] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:23:08.739347 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa08541-d389-436b-a3e2-5a73b231c590 None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:23:08.739620 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa08541-d389-436b-a3e2-5a73b231c590 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:23:08.767985 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.auth.core [None req-3fa08541-d389-436b-a3e2-5a73b231c590 None None] MFA Rules not processed for user `ca617ad9615f47f99a41ba0e6c4a16b8`. Rule list: `[]` (Enabled: `True`). {{(pid=81400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 03:23:08.769358 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-3fa08541-d389-436b-a3e2-5a73b231c590 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:23:08.834845 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:23:08.849097 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 1060/2126] 10.222.0.18 () {58 vars in 958 bytes} [Tue Sep 13 03:23:08 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 03:23:08.856542 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e01513b-9e7f-4b75-9cb7-188647ddd55f admin admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:23:08.858371 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-4e01513b-9e7f-4b75-9cb7-188647ddd55f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:23:08.915112 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e01513b-9e7f-4b75-9cb7-188647ddd55f admin admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:23:08.918895 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c81b49ee-5091-4f5b-a595-6906f897aee2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '924ee3f608d34a18b0ddc4695f102a56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b9d94f6a68f7470d9494f112175620dc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:23:08.919488 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c81b49ee-5091-4f5b-a595-6906f897aee2 service cinder] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:23:08.919763 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c81b49ee-5091-4f5b-a595-6906f897aee2 service cinder] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:23:08.920031 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c81b49ee-5091-4f5b-a595-6906f897aee2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:23:08.928809 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c81b49ee-5091-4f5b-a595-6906f897aee2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:23:08.930237 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c81b49ee-5091-4f5b-a595-6906f897aee2 service cinder] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:23:08.935205 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:23:08.941777 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 1067/2127] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Sep 13 03:23:08 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 03:23:08.987769 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fa08541-d389-436b-a3e2-5a73b231c590 None None] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:23:08.999415 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fa08541-d389-436b-a3e2-5a73b231c590 None None] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:23:09.003003 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7370ea65-901a-4ab3-8539-b1696dfca2ab service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '924ee3f608d34a18b0ddc4695f102a56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b9d94f6a68f7470d9494f112175620dc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:23:09.003593 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7370ea65-901a-4ab3-8539-b1696dfca2ab service cinder] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:23:09.003868 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7370ea65-901a-4ab3-8539-b1696dfca2ab service cinder] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:23:09.004135 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7370ea65-901a-4ab3-8539-b1696dfca2ab service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:23:09.065543 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7370ea65-901a-4ab3-8539-b1696dfca2ab service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:23:09.067128 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7370ea65-901a-4ab3-8539-b1696dfca2ab service cinder] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:23:09.076559 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 1061/2128] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Sep 13 03:23:08 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 03:23:11.767410 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c81b49ee-5091-4f5b-a595-6906f897aee2 service cinder] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:23:11.768827 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-c81b49ee-5091-4f5b-a595-6906f897aee2 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:23:11.845356 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c81b49ee-5091-4f5b-a595-6906f897aee2 service cinder] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:23:11.851155 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58950036-9578-4a24-aa1e-85df2645ff5e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76c5181fc7be45279ac00017bb4640b3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:23:11.851855 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58950036-9578-4a24-aa1e-85df2645ff5e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:23:11.852159 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58950036-9578-4a24-aa1e-85df2645ff5e admin admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:23:11.852473 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58950036-9578-4a24-aa1e-85df2645ff5e admin admin] PATH_INFO: `/v3/users/d009b5b585dc47c4b0bc5fd39e4b1cf1` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:23:11.854078 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58950036-9578-4a24-aa1e-85df2645ff5e admin admin] RBAC: Authorizing `identity:delete_user(user_id=d009b5b585dc47c4b0bc5fd39e4b1cf1)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:23:11.856721 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58950036-9578-4a24-aa1e-85df2645ff5e admin admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:23:11.914062 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-58950036-9578-4a24-aa1e-85df2645ff5e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd009b5b585dc47c4b0bc5fd39e4b1cf1'} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:23:11.921747 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-58950036-9578-4a24-aa1e-85df2645ff5e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd009b5b585dc47c4b0bc5fd39e4b1cf1'} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:23:11.933731 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-58950036-9578-4a24-aa1e-85df2645ff5e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd009b5b585dc47c4b0bc5fd39e4b1cf1'} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:23:11.934360 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-58950036-9578-4a24-aa1e-85df2645ff5e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd009b5b585dc47c4b0bc5fd39e4b1cf1'} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:23:11.941021 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-58950036-9578-4a24-aa1e-85df2645ff5e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd009b5b585dc47c4b0bc5fd39e4b1cf1'} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:23:11.947380 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 1068/2129] 10.222.0.18 () {58 vars in 1224 bytes} [Tue Sep 13 03:23:11 2022] DELETE /identity/v3/users/d009b5b585dc47c4b0bc5fd39e4b1cf1 => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 03:23:11.951757 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-025abe6d-f3c8-454a-aca9-e487e6c8db8c None None] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:23:11.952207 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-025abe6d-f3c8-454a-aca9-e487e6c8db8c None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:23:11.952299 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-025abe6d-f3c8-454a-aca9-e487e6c8db8c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:23:11.971560 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.auth.core [None req-025abe6d-f3c8-454a-aca9-e487e6c8db8c None None] MFA Rules not processed for user `49f280ab086b4610855c4ffa378517d9`. Rule list: `[]` (Enabled: `True`). {{(pid=81400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 03:23:11.972739 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-025abe6d-f3c8-454a-aca9-e487e6c8db8c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:23:12.030405 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 1062/2130] 10.222.0.18 () {58 vars in 958 bytes} [Tue Sep 13 03:23:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 03:23:12.040078 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58950036-9578-4a24-aa1e-85df2645ff5e admin admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:23:12.041461 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-58950036-9578-4a24-aa1e-85df2645ff5e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:23:12.098035 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58950036-9578-4a24-aa1e-85df2645ff5e admin admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:23:12.101662 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da5ea573-dbde-4f70-800f-c3e38ed4661d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdffbc851086441ebd01022d1e4c2609', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b9d94f6a68f7470d9494f112175620dc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:23:12.102255 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da5ea573-dbde-4f70-800f-c3e38ed4661d service neutron] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:23:12.102530 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da5ea573-dbde-4f70-800f-c3e38ed4661d service neutron] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:23:12.102864 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da5ea573-dbde-4f70-800f-c3e38ed4661d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:23:12.111711 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da5ea573-dbde-4f70-800f-c3e38ed4661d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:23:12.113130 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da5ea573-dbde-4f70-800f-c3e38ed4661d service neutron] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:23:12.123039 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 1069/2131] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Sep 13 03:23:12 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 03:23:12.399436 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-025abe6d-f3c8-454a-aca9-e487e6c8db8c None None] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:23:12.401356 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-025abe6d-f3c8-454a-aca9-e487e6c8db8c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:23:12.414192 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-025abe6d-f3c8-454a-aca9-e487e6c8db8c None None] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:23:12.438988 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a93e2e4-843c-4ba4-96b9-1e0df0360f9b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76c5181fc7be45279ac00017bb4640b3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:23:12.439626 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a93e2e4-843c-4ba4-96b9-1e0df0360f9b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:23:12.439954 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a93e2e4-843c-4ba4-96b9-1e0df0360f9b admin admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:23:12.440160 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a93e2e4-843c-4ba4-96b9-1e0df0360f9b admin admin] PATH_INFO: `/v3/projects/4eefb4b6cab54c4aa25fa1928d3c8962` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:23:12.441268 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a93e2e4-843c-4ba4-96b9-1e0df0360f9b admin admin] RBAC: Authorizing `identity:delete_project(project_id=4eefb4b6cab54c4aa25fa1928d3c8962)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:23:12.443205 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a93e2e4-843c-4ba4-96b9-1e0df0360f9b admin admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:23:12.493688 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-6a93e2e4-843c-4ba4-96b9-1e0df0360f9b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4eefb4b6cab54c4aa25fa1928d3c8962', 'request_id': 'req-6a93e2e4-843c-4ba4-96b9-1e0df0360f9b'} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:23:12.499770 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-6a93e2e4-843c-4ba4-96b9-1e0df0360f9b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4eefb4b6cab54c4aa25fa1928d3c8962', 'request_id': 'req-6a93e2e4-843c-4ba4-96b9-1e0df0360f9b'} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:23:12.506764 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-6a93e2e4-843c-4ba4-96b9-1e0df0360f9b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4eefb4b6cab54c4aa25fa1928d3c8962', 'request_id': 'req-6a93e2e4-843c-4ba4-96b9-1e0df0360f9b'} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:23:12.520499 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-6a93e2e4-843c-4ba4-96b9-1e0df0360f9b admin admin] The token cache is being invalidate because project 4eefb4b6cab54c4aa25fa1928d3c8962 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81400) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 03:23:12.520793 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-6a93e2e4-843c-4ba4-96b9-1e0df0360f9b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:23:12.521532 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-6a93e2e4-843c-4ba4-96b9-1e0df0360f9b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:23:12.527042 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 1063/2132] 10.222.0.18 () {58 vars in 1233 bytes} [Tue Sep 13 03:23:12 2022] DELETE /identity/v3/projects/4eefb4b6cab54c4aa25fa1928d3c8962 => generated 0 bytes in 124 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 03:23:12.528995 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-581e7d10-ef7b-480d-8e2e-e62a62c1d838 None None] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:23:12.529278 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-581e7d10-ef7b-480d-8e2e-e62a62c1d838 None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:23:12.529549 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-581e7d10-ef7b-480d-8e2e-e62a62c1d838 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:23:12.552174 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.auth.core [None req-581e7d10-ef7b-480d-8e2e-e62a62c1d838 None None] MFA Rules not processed for user `49f280ab086b4610855c4ffa378517d9`. Rule list: `[]` (Enabled: `True`). {{(pid=81399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 03:23:12.554200 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-581e7d10-ef7b-480d-8e2e-e62a62c1d838 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:23:12.573576 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:23:12.625388 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 1070/2133] 10.222.0.18 () {58 vars in 958 bytes} [Tue Sep 13 03:23:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 13 03:23:12.633198 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb016d8-b4fc-479f-9164-2f26465e8afd None None] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:23:12.633485 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb016d8-b4fc-479f-9164-2f26465e8afd None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:23:12.633755 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb016d8-b4fc-479f-9164-2f26465e8afd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:23:12.653395 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.auth.core [None req-adb016d8-b4fc-479f-9164-2f26465e8afd None None] MFA Rules not processed for user `49f280ab086b4610855c4ffa378517d9`. Rule list: `[]` (Enabled: `True`). {{(pid=81400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 03:23:12.654563 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-adb016d8-b4fc-479f-9164-2f26465e8afd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:23:12.663390 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 1064/2134] 10.222.0.18 () {58 vars in 958 bytes} [Tue Sep 13 03:23:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 03:23:12.667715 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-581e7d10-ef7b-480d-8e2e-e62a62c1d838 None None] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:23:12.680962 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-581e7d10-ef7b-480d-8e2e-e62a62c1d838 None None] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:23:12.685780 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51982c73-3489-437c-b06e-ad6c58b9c722 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76c5181fc7be45279ac00017bb4640b3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:23:12.686525 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51982c73-3489-437c-b06e-ad6c58b9c722 admin admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:23:12.686846 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51982c73-3489-437c-b06e-ad6c58b9c722 admin admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:23:12.687121 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51982c73-3489-437c-b06e-ad6c58b9c722 admin admin] PATH_INFO: `/v3/domains` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:23:12.687774 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51982c73-3489-437c-b06e-ad6c58b9c722 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 03:23:12.688074 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51982c73-3489-437c-b06e-ad6c58b9c722 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:23:12.689702 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51982c73-3489-437c-b06e-ad6c58b9c722 admin admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:23:12.705317 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 1071/2135] 10.222.0.18 () {58 vars in 1153 bytes} [Tue Sep 13 03:23:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 03:23:12.706439 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adb016d8-b4fc-479f-9164-2f26465e8afd None None] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:23:12.715908 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adb016d8-b4fc-479f-9164-2f26465e8afd None None] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:23:12.721367 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c913989b-af7e-4b01-8c37-b4f25ece70a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76c5181fc7be45279ac00017bb4640b3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:23:12.721970 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c913989b-af7e-4b01-8c37-b4f25ece70a9 admin admin] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:23:12.722242 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c913989b-af7e-4b01-8c37-b4f25ece70a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:23:12.722541 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c913989b-af7e-4b01-8c37-b4f25ece70a9 admin admin] PATH_INFO: `/v3/projects` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:23:12.723176 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c913989b-af7e-4b01-8c37-b4f25ece70a9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:23:12.725607 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c913989b-af7e-4b01-8c37-b4f25ece70a9 admin admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:23:12.735617 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:23:12.756138 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 1065/2136] 10.222.0.18 () {60 vars in 1153 bytes} [Tue Sep 13 03:23:12 2022] POST /identity/v3/projects => generated 371 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 03:23:12.760057 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51982c73-3489-437c-b06e-ad6c58b9c722 admin admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:23:12.770238 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51982c73-3489-437c-b06e-ad6c58b9c722 admin admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:23:12.842467 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e35a5cad-146b-453d-b98e-d94a764eabb0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76c5181fc7be45279ac00017bb4640b3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:23:12.843113 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e35a5cad-146b-453d-b98e-d94a764eabb0 admin admin] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:23:12.843390 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e35a5cad-146b-453d-b98e-d94a764eabb0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:23:12.843660 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e35a5cad-146b-453d-b98e-d94a764eabb0 admin admin] PATH_INFO: `/v3/users` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:23:12.844282 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e35a5cad-146b-453d-b98e-d94a764eabb0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:23:12.846197 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e35a5cad-146b-453d-b98e-d94a764eabb0 admin admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:23:12.868459 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 1072/2137] 10.222.0.18 () {60 vars in 1144 bytes} [Tue Sep 13 03:23:12 2022] POST /identity/v3/users => generated 286 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 13 03:23:12.872534 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c913989b-af7e-4b01-8c37-b4f25ece70a9 admin admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:23:12.883079 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c913989b-af7e-4b01-8c37-b4f25ece70a9 admin admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:23:12.887780 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fe46835-cd20-4d03-ab66-60a1c66af188 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76c5181fc7be45279ac00017bb4640b3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:23:12.888379 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fe46835-cd20-4d03-ab66-60a1c66af188 admin admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:23:12.888656 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fe46835-cd20-4d03-ab66-60a1c66af188 admin admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:23:12.888925 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fe46835-cd20-4d03-ab66-60a1c66af188 admin admin] PATH_INFO: `/v3/roles` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:23:12.889562 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fe46835-cd20-4d03-ab66-60a1c66af188 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:23:12.891191 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fe46835-cd20-4d03-ab66-60a1c66af188 admin admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:23:12.901661 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 1066/2138] 10.222.0.18 () {58 vars in 1122 bytes} [Tue Sep 13 03:23:12 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 03:23:12.905602 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e35a5cad-146b-453d-b98e-d94a764eabb0 admin admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:23:12.916393 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e35a5cad-146b-453d-b98e-d94a764eabb0 admin admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:23:12.922159 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-219f5cae-7bc3-41e6-aceb-7789664bdfd4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76c5181fc7be45279ac00017bb4640b3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:23:12.922802 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-219f5cae-7bc3-41e6-aceb-7789664bdfd4 admin admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:23:12.923065 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-219f5cae-7bc3-41e6-aceb-7789664bdfd4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:23:12.923340 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-219f5cae-7bc3-41e6-aceb-7789664bdfd4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:23:12.923977 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-219f5cae-7bc3-41e6-aceb-7789664bdfd4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:23:12.925573 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-219f5cae-7bc3-41e6-aceb-7789664bdfd4 admin admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:23:12.936838 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 1073/2139] 10.222.0.18 () {58 vars in 1122 bytes} [Tue Sep 13 03:23:12 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 03:23:12.940803 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fe46835-cd20-4d03-ab66-60a1c66af188 admin admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:23:12.951185 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fe46835-cd20-4d03-ab66-60a1c66af188 admin admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:23:12.955910 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-047b3e3d-054b-4b30-a88d-7e9f06e71742 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76c5181fc7be45279ac00017bb4640b3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:23:12.956560 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-047b3e3d-054b-4b30-a88d-7e9f06e71742 admin admin] REQUEST_METHOD: `PUT` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:23:12.956835 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-047b3e3d-054b-4b30-a88d-7e9f06e71742 admin admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:23:12.957108 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-047b3e3d-054b-4b30-a88d-7e9f06e71742 admin admin] PATH_INFO: `/v3/projects/fb8fff2dfb49442492296d4c1701abb1/users/7ad4d86f4c8f450986870292477c1182/roles/1eb7a5b5203945c29e6d22d653f47847` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:23:12.975232 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-047b3e3d-054b-4b30-a88d-7e9f06e71742 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fb8fff2dfb49442492296d4c1701abb1, user_id=7ad4d86f4c8f450986870292477c1182, role_id=1eb7a5b5203945c29e6d22d653f47847)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:23:12.978572 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-047b3e3d-054b-4b30-a88d-7e9f06e71742 admin admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:23:12.994997 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 1067/2140] 10.222.0.18 () {60 vars in 1483 bytes} [Tue Sep 13 03:23:12 2022] PUT /identity/v3/projects/fb8fff2dfb49442492296d4c1701abb1/users/7ad4d86f4c8f450986870292477c1182/roles/1eb7a5b5203945c29e6d22d653f47847 => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 03:23:13.000492 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa1fc072-8325-480c-8c15-461cf69ec282 None None] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:23:13.000830 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa1fc072-8325-480c-8c15-461cf69ec282 None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:23:13.001111 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa1fc072-8325-480c-8c15-461cf69ec282 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:23:13.022900 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.auth.core [None req-aa1fc072-8325-480c-8c15-461cf69ec282 None None] MFA Rules not processed for user `49f280ab086b4610855c4ffa378517d9`. Rule list: `[]` (Enabled: `True`). {{(pid=81399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 03:23:13.024169 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-aa1fc072-8325-480c-8c15-461cf69ec282 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:23:13.036038 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-047b3e3d-054b-4b30-a88d-7e9f06e71742 admin admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:23:13.038453 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-047b3e3d-054b-4b30-a88d-7e9f06e71742 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:23:13.107395 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 1074/2141] 10.222.0.18 () {58 vars in 958 bytes} [Tue Sep 13 03:23:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 03:23:13.116091 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa1fc072-8325-480c-8c15-461cf69ec282 None None] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:23:13.117482 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-aa1fc072-8325-480c-8c15-461cf69ec282 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:23:13.118725 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-047b3e3d-054b-4b30-a88d-7e9f06e71742 admin admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:23:13.123670 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-216803cb-45d2-46e5-ae74-b2be95c84977 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76c5181fc7be45279ac00017bb4640b3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:23:13.124424 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-216803cb-45d2-46e5-ae74-b2be95c84977 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:23:13.124761 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-216803cb-45d2-46e5-ae74-b2be95c84977 admin admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:23:13.125096 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-216803cb-45d2-46e5-ae74-b2be95c84977 admin admin] PATH_INFO: `/v3/users/711e6da793574b979ec0cd2c7e343456` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:23:13.126448 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-216803cb-45d2-46e5-ae74-b2be95c84977 admin admin] RBAC: Authorizing `identity:delete_user(user_id=711e6da793574b979ec0cd2c7e343456)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:23:13.128558 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-216803cb-45d2-46e5-ae74-b2be95c84977 admin admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:23:13.154969 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:23:13.184820 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-216803cb-45d2-46e5-ae74-b2be95c84977 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '711e6da793574b979ec0cd2c7e343456'} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:23:13.189176 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa1fc072-8325-480c-8c15-461cf69ec282 None None] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:23:13.195244 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-216803cb-45d2-46e5-ae74-b2be95c84977 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '711e6da793574b979ec0cd2c7e343456'} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:23:13.195753 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86455af4-4a93-4ba4-8d82-236d7f9f9064 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdffbc851086441ebd01022d1e4c2609', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b9d94f6a68f7470d9494f112175620dc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:23:13.196289 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-216803cb-45d2-46e5-ae74-b2be95c84977 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '711e6da793574b979ec0cd2c7e343456'} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:23:13.196569 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86455af4-4a93-4ba4-8d82-236d7f9f9064 service neutron] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:23:13.197154 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86455af4-4a93-4ba4-8d82-236d7f9f9064 service neutron] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:23:13.197529 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86455af4-4a93-4ba4-8d82-236d7f9f9064 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:23:13.200356 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-86455af4-4a93-4ba4-8d82-236d7f9f9064 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:23:13.213976 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-216803cb-45d2-46e5-ae74-b2be95c84977 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '711e6da793574b979ec0cd2c7e343456'} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:23:13.218056 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86455af4-4a93-4ba4-8d82-236d7f9f9064 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:23:13.220088 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86455af4-4a93-4ba4-8d82-236d7f9f9064 service neutron] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:23:13.223650 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-216803cb-45d2-46e5-ae74-b2be95c84977 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '711e6da793574b979ec0cd2c7e343456'} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:23:13.234544 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 1068/2142] 10.222.0.18 () {58 vars in 1224 bytes} [Tue Sep 13 03:23:13 2022] DELETE /identity/v3/users/711e6da793574b979ec0cd2c7e343456 => generated 0 bytes in 197 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 13 03:23:13.234778 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 1075/2143] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Sep 13 03:23:13 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 119 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 13 03:23:13.236377 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef5797c-77dc-4980-93f6-de4c33e0ec88 None None] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:23:13.236927 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef5797c-77dc-4980-93f6-de4c33e0ec88 None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:23:13.237222 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef5797c-77dc-4980-93f6-de4c33e0ec88 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:23:13.260729 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.auth.core [None req-2ef5797c-77dc-4980-93f6-de4c33e0ec88 None None] MFA Rules not processed for user `49f280ab086b4610855c4ffa378517d9`. Rule list: `[]` (Enabled: `True`). {{(pid=81400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 03:23:13.262304 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-2ef5797c-77dc-4980-93f6-de4c33e0ec88 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:23:13.300659 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=87,keystone:INSERT=3,keystone:DELETE=10 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:23:13.344769 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 1069/2144] 10.222.0.18 () {58 vars in 958 bytes} [Tue Sep 13 03:23:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 03:23:13.356213 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86455af4-4a93-4ba4-8d82-236d7f9f9064 service neutron] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:23:13.357772 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-86455af4-4a93-4ba4-8d82-236d7f9f9064 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:23:13.370188 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86455af4-4a93-4ba4-8d82-236d7f9f9064 service neutron] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:23:13.432924 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef6a8d9d-a99d-4724-a93f-e1d6580f8b7b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdffbc851086441ebd01022d1e4c2609', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b9d94f6a68f7470d9494f112175620dc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:23:13.433540 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef6a8d9d-a99d-4724-a93f-e1d6580f8b7b service neutron] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:23:13.433820 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef6a8d9d-a99d-4724-a93f-e1d6580f8b7b service neutron] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:23:13.434088 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef6a8d9d-a99d-4724-a93f-e1d6580f8b7b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:23:13.444278 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef6a8d9d-a99d-4724-a93f-e1d6580f8b7b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:23:13.445708 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef6a8d9d-a99d-4724-a93f-e1d6580f8b7b service neutron] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:23:13.455854 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 1076/2145] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Sep 13 03:23:13 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 03:23:13.895145 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ef5797c-77dc-4980-93f6-de4c33e0ec88 None None] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:23:13.896724 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-2ef5797c-77dc-4980-93f6-de4c33e0ec88 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:23:13.908451 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ef5797c-77dc-4980-93f6-de4c33e0ec88 None None] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:23:13.913379 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13a0c760-8a14-4286-b710-da5dd8997644 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76c5181fc7be45279ac00017bb4640b3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:23:13.914298 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a0c760-8a14-4286-b710-da5dd8997644 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:23:13.914422 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a0c760-8a14-4286-b710-da5dd8997644 admin admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:23:13.914749 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a0c760-8a14-4286-b710-da5dd8997644 admin admin] PATH_INFO: `/v3/projects/fe45a82144ee4028883d2826bdf8866c` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:23:13.915985 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13a0c760-8a14-4286-b710-da5dd8997644 admin admin] RBAC: Authorizing `identity:delete_project(project_id=fe45a82144ee4028883d2826bdf8866c)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:23:13.918053 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13a0c760-8a14-4286-b710-da5dd8997644 admin admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:23:13.976734 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-13a0c760-8a14-4286-b710-da5dd8997644 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fe45a82144ee4028883d2826bdf8866c', 'request_id': 'req-13a0c760-8a14-4286-b710-da5dd8997644'} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:23:13.985559 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-13a0c760-8a14-4286-b710-da5dd8997644 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fe45a82144ee4028883d2826bdf8866c', 'request_id': 'req-13a0c760-8a14-4286-b710-da5dd8997644'} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:23:13.992925 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-13a0c760-8a14-4286-b710-da5dd8997644 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fe45a82144ee4028883d2826bdf8866c', 'request_id': 'req-13a0c760-8a14-4286-b710-da5dd8997644'} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:23:14.009457 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-13a0c760-8a14-4286-b710-da5dd8997644 admin admin] The token cache is being invalidate because project fe45a82144ee4028883d2826bdf8866c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81400) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 03:23:14.009776 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-13a0c760-8a14-4286-b710-da5dd8997644 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:23:14.010502 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-13a0c760-8a14-4286-b710-da5dd8997644 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:23:14.016624 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef6a8d9d-a99d-4724-a93f-e1d6580f8b7b service neutron] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:23:14.018676 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 1070/2146] 10.222.0.18 () {58 vars in 1233 bytes} [Tue Sep 13 03:23:13 2022] DELETE /identity/v3/projects/fe45a82144ee4028883d2826bdf8866c => generated 0 bytes in 118 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 03:23:14.027827 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef6a8d9d-a99d-4724-a93f-e1d6580f8b7b service neutron] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:23:14.133411 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee4e8fc2-63ca-4d4c-9574-b53d64f6f50f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76c5181fc7be45279ac00017bb4640b3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:23:14.134650 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee4e8fc2-63ca-4d4c-9574-b53d64f6f50f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:23:14.134650 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee4e8fc2-63ca-4d4c-9574-b53d64f6f50f admin admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:23:14.134898 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee4e8fc2-63ca-4d4c-9574-b53d64f6f50f admin admin] PATH_INFO: `/v3/users/b3dbf60e414b4aac8811d223b40f0772` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:23:14.135960 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee4e8fc2-63ca-4d4c-9574-b53d64f6f50f admin admin] RBAC: Authorizing `identity:delete_user(user_id=b3dbf60e414b4aac8811d223b40f0772)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:23:14.138513 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee4e8fc2-63ca-4d4c-9574-b53d64f6f50f admin admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:23:14.193686 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-ee4e8fc2-63ca-4d4c-9574-b53d64f6f50f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b3dbf60e414b4aac8811d223b40f0772'} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:23:14.201855 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-ee4e8fc2-63ca-4d4c-9574-b53d64f6f50f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b3dbf60e414b4aac8811d223b40f0772'} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:23:14.217073 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-ee4e8fc2-63ca-4d4c-9574-b53d64f6f50f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b3dbf60e414b4aac8811d223b40f0772'} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:23:14.217783 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-ee4e8fc2-63ca-4d4c-9574-b53d64f6f50f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b3dbf60e414b4aac8811d223b40f0772'} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:23:14.226048 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-ee4e8fc2-63ca-4d4c-9574-b53d64f6f50f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b3dbf60e414b4aac8811d223b40f0772'} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:23:14.228688 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:23:14.234931 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 1077/2147] 10.222.0.18 () {58 vars in 1224 bytes} [Tue Sep 13 03:23:14 2022] DELETE /identity/v3/users/b3dbf60e414b4aac8811d223b40f0772 => generated 0 bytes in 219 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 13 03:23:14.576377 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13a0c760-8a14-4286-b710-da5dd8997644 admin admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:23:14.577754 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-13a0c760-8a14-4286-b710-da5dd8997644 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:23:14.592781 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13a0c760-8a14-4286-b710-da5dd8997644 admin admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:23:14.660681 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5ab12f2-9542-4aa2-9b44-1fbbfcdbe918 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76c5181fc7be45279ac00017bb4640b3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:23:14.661562 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ab12f2-9542-4aa2-9b44-1fbbfcdbe918 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:23:14.661978 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ab12f2-9542-4aa2-9b44-1fbbfcdbe918 admin admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:23:14.662455 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ab12f2-9542-4aa2-9b44-1fbbfcdbe918 admin admin] PATH_INFO: `/v3/projects/f7e427a44973492ca0afd6f53f868eb0` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:23:14.663823 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5ab12f2-9542-4aa2-9b44-1fbbfcdbe918 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f7e427a44973492ca0afd6f53f868eb0)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:23:14.665828 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5ab12f2-9542-4aa2-9b44-1fbbfcdbe918 admin admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:23:14.757320 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-f5ab12f2-9542-4aa2-9b44-1fbbfcdbe918 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f7e427a44973492ca0afd6f53f868eb0', 'request_id': 'req-f5ab12f2-9542-4aa2-9b44-1fbbfcdbe918'} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:23:14.765482 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-f5ab12f2-9542-4aa2-9b44-1fbbfcdbe918 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f7e427a44973492ca0afd6f53f868eb0', 'request_id': 'req-f5ab12f2-9542-4aa2-9b44-1fbbfcdbe918'} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:23:14.767418 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:23:14.774766 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-f5ab12f2-9542-4aa2-9b44-1fbbfcdbe918 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f7e427a44973492ca0afd6f53f868eb0', 'request_id': 'req-f5ab12f2-9542-4aa2-9b44-1fbbfcdbe918'} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:23:14.796580 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-f5ab12f2-9542-4aa2-9b44-1fbbfcdbe918 admin admin] The token cache is being invalidate because project f7e427a44973492ca0afd6f53f868eb0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81400) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 03:23:14.797121 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-f5ab12f2-9542-4aa2-9b44-1fbbfcdbe918 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:23:14.798379 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-f5ab12f2-9542-4aa2-9b44-1fbbfcdbe918 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:23:14.804894 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 1071/2148] 10.222.0.18 () {58 vars in 1233 bytes} [Tue Sep 13 03:23:14 2022] DELETE /identity/v3/projects/f7e427a44973492ca0afd6f53f868eb0 => generated 0 bytes in 225 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 03:23:14.806220 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee4e8fc2-63ca-4d4c-9574-b53d64f6f50f admin admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:23:14.808139 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-ee4e8fc2-63ca-4d4c-9574-b53d64f6f50f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:23:14.825007 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee4e8fc2-63ca-4d4c-9574-b53d64f6f50f admin admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:23:14.933698 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0019404-3729-4709-a3b8-103c79da1553 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76c5181fc7be45279ac00017bb4640b3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:23:14.934776 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0019404-3729-4709-a3b8-103c79da1553 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:23:14.935054 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0019404-3729-4709-a3b8-103c79da1553 admin admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:23:14.935322 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0019404-3729-4709-a3b8-103c79da1553 admin admin] PATH_INFO: `/v3/users/ca617ad9615f47f99a41ba0e6c4a16b8` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:23:14.936533 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0019404-3729-4709-a3b8-103c79da1553 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ca617ad9615f47f99a41ba0e6c4a16b8)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:23:14.938482 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0019404-3729-4709-a3b8-103c79da1553 admin admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:23:15.000999 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-f0019404-3729-4709-a3b8-103c79da1553 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ca617ad9615f47f99a41ba0e6c4a16b8'} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:23:15.010975 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-f0019404-3729-4709-a3b8-103c79da1553 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ca617ad9615f47f99a41ba0e6c4a16b8'} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:23:15.048053 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-f0019404-3729-4709-a3b8-103c79da1553 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ca617ad9615f47f99a41ba0e6c4a16b8'} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:23:15.049118 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-f0019404-3729-4709-a3b8-103c79da1553 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ca617ad9615f47f99a41ba0e6c4a16b8'} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:23:15.085485 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-f0019404-3729-4709-a3b8-103c79da1553 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ca617ad9615f47f99a41ba0e6c4a16b8'} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:23:15.099065 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 1078/2149] 10.222.0.18 () {58 vars in 1224 bytes} [Tue Sep 13 03:23:14 2022] DELETE /identity/v3/users/ca617ad9615f47f99a41ba0e6c4a16b8 => generated 0 bytes in 293 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 03:23:15.525218 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5ab12f2-9542-4aa2-9b44-1fbbfcdbe918 admin admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:23:15.528481 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-f5ab12f2-9542-4aa2-9b44-1fbbfcdbe918 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:23:15.552633 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5ab12f2-9542-4aa2-9b44-1fbbfcdbe918 admin admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:23:15.652911 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eaabf97-3a75-442b-83be-65901abf3413 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76c5181fc7be45279ac00017bb4640b3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:23:15.653677 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eaabf97-3a75-442b-83be-65901abf3413 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:23:15.654071 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eaabf97-3a75-442b-83be-65901abf3413 admin admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:23:15.654411 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eaabf97-3a75-442b-83be-65901abf3413 admin admin] PATH_INFO: `/v3/projects/c73c328b3bf34d65b32928e78e7e4a43` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:23:15.657017 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eaabf97-3a75-442b-83be-65901abf3413 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c73c328b3bf34d65b32928e78e7e4a43)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:23:15.659293 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eaabf97-3a75-442b-83be-65901abf3413 admin admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:23:15.727578 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-4eaabf97-3a75-442b-83be-65901abf3413 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c73c328b3bf34d65b32928e78e7e4a43', 'request_id': 'req-4eaabf97-3a75-442b-83be-65901abf3413'} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:23:15.733781 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-4eaabf97-3a75-442b-83be-65901abf3413 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c73c328b3bf34d65b32928e78e7e4a43', 'request_id': 'req-4eaabf97-3a75-442b-83be-65901abf3413'} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:23:15.740396 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-4eaabf97-3a75-442b-83be-65901abf3413 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c73c328b3bf34d65b32928e78e7e4a43', 'request_id': 'req-4eaabf97-3a75-442b-83be-65901abf3413'} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:23:15.755233 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-4eaabf97-3a75-442b-83be-65901abf3413 admin admin] The token cache is being invalidate because project c73c328b3bf34d65b32928e78e7e4a43 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81400) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 03:23:15.756277 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-4eaabf97-3a75-442b-83be-65901abf3413 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:23:15.757118 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-4eaabf97-3a75-442b-83be-65901abf3413 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:23:15.759337 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 1072/2150] 10.222.0.18 () {58 vars in 1233 bytes} [Tue Sep 13 03:23:15 2022] DELETE /identity/v3/projects/c73c328b3bf34d65b32928e78e7e4a43 => generated 0 bytes in 234 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 03:23:15.765116 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea115caa-ce91-468d-8446-0c8f94284e00 None None] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:23:15.765717 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea115caa-ce91-468d-8446-0c8f94284e00 None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:23:15.766157 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea115caa-ce91-468d-8446-0c8f94284e00 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:23:15.788712 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.auth.core [None req-ea115caa-ce91-468d-8446-0c8f94284e00 None None] MFA Rules not processed for user `49f280ab086b4610855c4ffa378517d9`. Rule list: `[]` (Enabled: `True`). {{(pid=81399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 03:23:15.791219 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-ea115caa-ce91-468d-8446-0c8f94284e00 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:23:15.875898 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 1079/2151] 10.222.0.18 () {58 vars in 958 bytes} [Tue Sep 13 03:23:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 13 03:23:15.884976 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff2e1a8-8a2f-4a24-80be-4b9b6b04f856 None None] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:23:15.885267 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff2e1a8-8a2f-4a24-80be-4b9b6b04f856 None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:23:15.885603 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff2e1a8-8a2f-4a24-80be-4b9b6b04f856 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:23:15.908421 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.auth.core [None req-2ff2e1a8-8a2f-4a24-80be-4b9b6b04f856 None None] MFA Rules not processed for user `49f280ab086b4610855c4ffa378517d9`. Rule list: `[]` (Enabled: `True`). {{(pid=81400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 03:23:15.909928 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-2ff2e1a8-8a2f-4a24-80be-4b9b6b04f856 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:23:15.925132 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 1073/2152] 10.222.0.18 () {58 vars in 958 bytes} [Tue Sep 13 03:23:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 03:23:15.930111 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea115caa-ce91-468d-8446-0c8f94284e00 None None] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:23:15.941282 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea115caa-ce91-468d-8446-0c8f94284e00 None None] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:23:15.946923 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd4ab629-f73d-4153-bf67-b4464c93861a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76c5181fc7be45279ac00017bb4640b3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:23:15.947686 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd4ab629-f73d-4153-bf67-b4464c93861a admin admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:23:15.947969 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd4ab629-f73d-4153-bf67-b4464c93861a admin admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:23:15.948245 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd4ab629-f73d-4153-bf67-b4464c93861a admin admin] PATH_INFO: `/v3/domains` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:23:15.948894 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd4ab629-f73d-4153-bf67-b4464c93861a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 03:23:15.949199 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd4ab629-f73d-4153-bf67-b4464c93861a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:23:15.950912 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd4ab629-f73d-4153-bf67-b4464c93861a admin admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:23:15.961256 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:23:15.967701 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 1080/2153] 10.222.0.18 () {58 vars in 1153 bytes} [Tue Sep 13 03:23:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 03:23:15.972359 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ff2e1a8-8a2f-4a24-80be-4b9b6b04f856 None None] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:23:15.986144 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ff2e1a8-8a2f-4a24-80be-4b9b6b04f856 None None] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:23:15.993088 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf59a528-e7c3-454f-92dc-95137af16335 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76c5181fc7be45279ac00017bb4640b3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:23:15.993718 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf59a528-e7c3-454f-92dc-95137af16335 admin admin] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:23:15.993998 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf59a528-e7c3-454f-92dc-95137af16335 admin admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:23:15.994270 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf59a528-e7c3-454f-92dc-95137af16335 admin admin] PATH_INFO: `/v3/projects` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:23:15.995019 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf59a528-e7c3-454f-92dc-95137af16335 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:23:15.996954 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf59a528-e7c3-454f-92dc-95137af16335 admin admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:23:16.039623 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 1074/2154] 10.222.0.18 () {60 vars in 1153 bytes} [Tue Sep 13 03:23:15 2022] POST /identity/v3/projects => generated 385 bytes in 68 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 03:23:16.044115 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd4ab629-f73d-4153-bf67-b4464c93861a admin admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:23:16.056454 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd4ab629-f73d-4153-bf67-b4464c93861a admin admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:23:16.123754 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecd63d61-95ef-4908-9d58-889784001911 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76c5181fc7be45279ac00017bb4640b3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:23:16.124474 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd63d61-95ef-4908-9d58-889784001911 admin admin] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:23:16.124824 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd63d61-95ef-4908-9d58-889784001911 admin admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:23:16.125146 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd63d61-95ef-4908-9d58-889784001911 admin admin] PATH_INFO: `/v3/users` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:23:16.125882 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecd63d61-95ef-4908-9d58-889784001911 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:23:16.127948 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecd63d61-95ef-4908-9d58-889784001911 admin admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:23:16.156425 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 1081/2155] 10.222.0.18 () {60 vars in 1144 bytes} [Tue Sep 13 03:23:16 2022] POST /identity/v3/users => generated 293 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 13 03:23:16.157627 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf59a528-e7c3-454f-92dc-95137af16335 admin admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:23:16.168206 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf59a528-e7c3-454f-92dc-95137af16335 admin admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:23:16.174170 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33a6795b-f563-4293-916f-85f6b0b63c9d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76c5181fc7be45279ac00017bb4640b3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:23:16.175701 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33a6795b-f563-4293-916f-85f6b0b63c9d admin admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:23:16.175701 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33a6795b-f563-4293-916f-85f6b0b63c9d admin admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:23:16.175701 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33a6795b-f563-4293-916f-85f6b0b63c9d admin admin] PATH_INFO: `/v3/roles` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:23:16.176912 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33a6795b-f563-4293-916f-85f6b0b63c9d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:23:16.178818 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33a6795b-f563-4293-916f-85f6b0b63c9d admin admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:23:16.194375 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 1075/2156] 10.222.0.18 () {58 vars in 1122 bytes} [Tue Sep 13 03:23:16 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 03:23:16.195654 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecd63d61-95ef-4908-9d58-889784001911 admin admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:23:16.206563 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecd63d61-95ef-4908-9d58-889784001911 admin admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:23:16.211606 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d41d6271-89cf-4ff2-9875-da3209fb8b82 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76c5181fc7be45279ac00017bb4640b3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:23:16.212315 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d41d6271-89cf-4ff2-9875-da3209fb8b82 admin admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:23:16.212686 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d41d6271-89cf-4ff2-9875-da3209fb8b82 admin admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:23:16.213072 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d41d6271-89cf-4ff2-9875-da3209fb8b82 admin admin] PATH_INFO: `/v3/roles` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:23:16.213801 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d41d6271-89cf-4ff2-9875-da3209fb8b82 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:23:16.215558 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d41d6271-89cf-4ff2-9875-da3209fb8b82 admin admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:23:16.231680 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 1082/2157] 10.222.0.18 () {58 vars in 1122 bytes} [Tue Sep 13 03:23:16 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 13 03:23:16.233268 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33a6795b-f563-4293-916f-85f6b0b63c9d admin admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:23:16.244888 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33a6795b-f563-4293-916f-85f6b0b63c9d admin admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:23:16.249762 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6636c3b1-3989-4ed2-ba4e-18f9fc3cf000 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76c5181fc7be45279ac00017bb4640b3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:23:16.250520 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6636c3b1-3989-4ed2-ba4e-18f9fc3cf000 admin admin] REQUEST_METHOD: `PUT` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:23:16.251078 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6636c3b1-3989-4ed2-ba4e-18f9fc3cf000 admin admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:23:16.251358 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6636c3b1-3989-4ed2-ba4e-18f9fc3cf000 admin admin] PATH_INFO: `/v3/projects/ea39509ee9414e6cb73af56498a826d9/users/f00292e69fd645fd91014047d35bbe68/roles/1eb7a5b5203945c29e6d22d653f47847` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:23:16.270234 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6636c3b1-3989-4ed2-ba4e-18f9fc3cf000 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ea39509ee9414e6cb73af56498a826d9, user_id=f00292e69fd645fd91014047d35bbe68, role_id=1eb7a5b5203945c29e6d22d653f47847)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:23:16.273685 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6636c3b1-3989-4ed2-ba4e-18f9fc3cf000 admin admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:23:16.289375 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 1076/2158] 10.222.0.18 () {60 vars in 1483 bytes} [Tue Sep 13 03:23:16 2022] PUT /identity/v3/projects/ea39509ee9414e6cb73af56498a826d9/users/f00292e69fd645fd91014047d35bbe68/roles/1eb7a5b5203945c29e6d22d653f47847 => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 03:23:16.300982 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8caeaf25-4e85-4d03-a1c6-78d1b9ff5faa None None] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:23:16.301263 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8caeaf25-4e85-4d03-a1c6-78d1b9ff5faa None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:23:16.301536 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8caeaf25-4e85-4d03-a1c6-78d1b9ff5faa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:23:16.378402 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.auth.core [None req-8caeaf25-4e85-4d03-a1c6-78d1b9ff5faa None None] MFA Rules not processed for user `f00292e69fd645fd91014047d35bbe68`. Rule list: `[]` (Enabled: `True`). {{(pid=81399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 03:23:16.380165 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-8caeaf25-4e85-4d03-a1c6-78d1b9ff5faa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:23:16.535166 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6636c3b1-3989-4ed2-ba4e-18f9fc3cf000 admin admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:23:16.535911 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 1083/2159] 10.222.0.18 () {58 vars in 958 bytes} [Tue Sep 13 03:23:16 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 236 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 03:23:16.567622 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=88,keystone:INSERT=4,keystone:DELETE=8 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:23:16.638833 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6636c3b1-3989-4ed2-ba4e-18f9fc3cf000 admin admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:23:16.644913 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8be7ca31-0777-42b3-9320-b081623676fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76c5181fc7be45279ac00017bb4640b3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:23:16.645676 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8be7ca31-0777-42b3-9320-b081623676fb admin admin] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:23:16.646052 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8be7ca31-0777-42b3-9320-b081623676fb admin admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:23:16.646460 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8be7ca31-0777-42b3-9320-b081623676fb admin admin] PATH_INFO: `/v3/projects` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:23:16.647219 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8be7ca31-0777-42b3-9320-b081623676fb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:23:16.649371 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8be7ca31-0777-42b3-9320-b081623676fb admin admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:23:16.688907 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 1077/2160] 10.222.0.18 () {60 vars in 1153 bytes} [Tue Sep 13 03:23:16 2022] POST /identity/v3/projects => generated 383 bytes in 154 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 13 03:23:16.693909 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8caeaf25-4e85-4d03-a1c6-78d1b9ff5faa None None] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:23:16.709802 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8caeaf25-4e85-4d03-a1c6-78d1b9ff5faa None None] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:23:16.827328 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6a49cff-8100-4975-8df6-c1474459eb6d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76c5181fc7be45279ac00017bb4640b3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:23:16.828033 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a49cff-8100-4975-8df6-c1474459eb6d admin admin] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:23:16.828316 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a49cff-8100-4975-8df6-c1474459eb6d admin admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:23:16.828682 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a49cff-8100-4975-8df6-c1474459eb6d admin admin] PATH_INFO: `/v3/users` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:23:16.829894 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6a49cff-8100-4975-8df6-c1474459eb6d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:23:16.831952 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6a49cff-8100-4975-8df6-c1474459eb6d admin admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:23:16.871115 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:23:16.883296 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 1084/2161] 10.222.0.18 () {60 vars in 1144 bytes} [Tue Sep 13 03:23:16 2022] POST /identity/v3/users => generated 298 bytes in 190 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 03:23:16.887589 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8be7ca31-0777-42b3-9320-b081623676fb admin admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:23:16.900360 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8be7ca31-0777-42b3-9320-b081623676fb admin admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:23:16.905533 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84301047-e3ad-47df-b3dd-5a5cdbe85409 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76c5181fc7be45279ac00017bb4640b3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:23:16.906145 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84301047-e3ad-47df-b3dd-5a5cdbe85409 admin admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:23:16.906419 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84301047-e3ad-47df-b3dd-5a5cdbe85409 admin admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:23:16.906905 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84301047-e3ad-47df-b3dd-5a5cdbe85409 admin admin] PATH_INFO: `/v3/roles` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:23:16.907657 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84301047-e3ad-47df-b3dd-5a5cdbe85409 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:23:16.909731 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84301047-e3ad-47df-b3dd-5a5cdbe85409 admin admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:23:16.934823 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6a49cff-8100-4975-8df6-c1474459eb6d admin admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:23:16.939115 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 1078/2162] 10.222.0.18 () {58 vars in 1122 bytes} [Tue Sep 13 03:23:16 2022] GET /identity/v3/roles => generated 1420 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 03:23:16.953877 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6a49cff-8100-4975-8df6-c1474459eb6d admin admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:23:16.962663 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cc7ef40-5ecd-4f39-8359-79dc7849a64e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76c5181fc7be45279ac00017bb4640b3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:23:16.963826 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc7ef40-5ecd-4f39-8359-79dc7849a64e admin admin] REQUEST_METHOD: `PUT` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:23:16.964374 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc7ef40-5ecd-4f39-8359-79dc7849a64e admin admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:23:16.964672 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc7ef40-5ecd-4f39-8359-79dc7849a64e admin admin] PATH_INFO: `/v3/projects/17774109433a48b399a928808d16f7d6/users/f2c5d1b2e2ad4346bf58e8df76618b51/roles/425d4b481c7846e0bc07e7e805d5b15f` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:23:16.992917 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cc7ef40-5ecd-4f39-8359-79dc7849a64e admin admin] RBAC: Authorizing `identity:create_grant(project_id=17774109433a48b399a928808d16f7d6, user_id=f2c5d1b2e2ad4346bf58e8df76618b51, role_id=425d4b481c7846e0bc07e7e805d5b15f)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:23:16.995748 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cc7ef40-5ecd-4f39-8359-79dc7849a64e admin admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:23:17.016677 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 1085/2163] 10.222.0.18 () {60 vars in 1483 bytes} [Tue Sep 13 03:23:16 2022] PUT /identity/v3/projects/17774109433a48b399a928808d16f7d6/users/f2c5d1b2e2ad4346bf58e8df76618b51/roles/425d4b481c7846e0bc07e7e805d5b15f => generated 0 bytes in 82 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 03:23:17.021756 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84301047-e3ad-47df-b3dd-5a5cdbe85409 admin admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:23:17.039857 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84301047-e3ad-47df-b3dd-5a5cdbe85409 admin admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:23:17.134566 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7201e7eb-30b4-42db-aae5-f3e5f6b7cb1b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76c5181fc7be45279ac00017bb4640b3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:23:17.135216 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7201e7eb-30b4-42db-aae5-f3e5f6b7cb1b admin admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:23:17.135545 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7201e7eb-30b4-42db-aae5-f3e5f6b7cb1b admin admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:23:17.135818 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7201e7eb-30b4-42db-aae5-f3e5f6b7cb1b admin admin] PATH_INFO: `/v3/roles` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:23:17.136465 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7201e7eb-30b4-42db-aae5-f3e5f6b7cb1b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:23:17.138072 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7201e7eb-30b4-42db-aae5-f3e5f6b7cb1b admin admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:23:17.150569 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 1079/2164] 10.222.0.18 () {58 vars in 1122 bytes} [Tue Sep 13 03:23:17 2022] GET /identity/v3/roles => generated 1420 bytes in 130 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 03:23:17.154300 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cc7ef40-5ecd-4f39-8359-79dc7849a64e admin admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:23:17.166597 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cc7ef40-5ecd-4f39-8359-79dc7849a64e admin admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:23:17.177128 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b3d0c41-cee6-4978-bad1-8b94960e2717 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76c5181fc7be45279ac00017bb4640b3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:23:17.178479 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b3d0c41-cee6-4978-bad1-8b94960e2717 admin admin] REQUEST_METHOD: `PUT` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:23:17.178884 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b3d0c41-cee6-4978-bad1-8b94960e2717 admin admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:23:17.179248 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b3d0c41-cee6-4978-bad1-8b94960e2717 admin admin] PATH_INFO: `/v3/projects/17774109433a48b399a928808d16f7d6/users/f2c5d1b2e2ad4346bf58e8df76618b51/roles/425d4b481c7846e0bc07e7e805d5b15f` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:23:17.181642 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b3d0c41-cee6-4978-bad1-8b94960e2717 admin admin] RBAC: Authorizing `identity:create_grant(project_id=17774109433a48b399a928808d16f7d6, user_id=f2c5d1b2e2ad4346bf58e8df76618b51, role_id=425d4b481c7846e0bc07e7e805d5b15f)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:23:17.184462 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b3d0c41-cee6-4978-bad1-8b94960e2717 admin admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:23:17.196079 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 1086/2165] 10.222.0.18 () {60 vars in 1483 bytes} [Tue Sep 13 03:23:17 2022] PUT /identity/v3/projects/17774109433a48b399a928808d16f7d6/users/f2c5d1b2e2ad4346bf58e8df76618b51/roles/425d4b481c7846e0bc07e7e805d5b15f => generated 0 bytes in 42 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 03:23:17.206811 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1291779e-3be1-4039-94dc-9b0ee0dc33d3 None None] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:23:17.207095 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1291779e-3be1-4039-94dc-9b0ee0dc33d3 None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:23:17.207363 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1291779e-3be1-4039-94dc-9b0ee0dc33d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:23:17.241065 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.auth.core [None req-1291779e-3be1-4039-94dc-9b0ee0dc33d3 None None] MFA Rules not processed for user `f2c5d1b2e2ad4346bf58e8df76618b51`. Rule list: `[]` (Enabled: `True`). {{(pid=81400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 03:23:17.242167 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-1291779e-3be1-4039-94dc-9b0ee0dc33d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:23:17.333261 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 1080/2166] 10.222.0.18 () {58 vars in 958 bytes} [Tue Sep 13 03:23:17 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 03:23:17.340788 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b3d0c41-cee6-4978-bad1-8b94960e2717 admin admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:23:17.342916 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-8b3d0c41-cee6-4978-bad1-8b94960e2717 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:23:17.421672 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b3d0c41-cee6-4978-bad1-8b94960e2717 admin admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:23:17.425607 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-885ecb81-4080-4ca9-b35c-0933ea44c19c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '924ee3f608d34a18b0ddc4695f102a56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b9d94f6a68f7470d9494f112175620dc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:23:17.426352 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-885ecb81-4080-4ca9-b35c-0933ea44c19c service cinder] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:23:17.426352 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-885ecb81-4080-4ca9-b35c-0933ea44c19c service cinder] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:23:17.427047 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-885ecb81-4080-4ca9-b35c-0933ea44c19c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:23:17.508714 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-885ecb81-4080-4ca9-b35c-0933ea44c19c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:23:17.510478 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-885ecb81-4080-4ca9-b35c-0933ea44c19c service cinder] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:23:17.523894 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 1087/2167] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Sep 13 03:23:17 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 184 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 03:23:17.792882 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c380e5d-0eb7-4760-9150-a54e3ebcbb12 None None] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:23:17.792882 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c380e5d-0eb7-4760-9150-a54e3ebcbb12 None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:23:17.793777 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c380e5d-0eb7-4760-9150-a54e3ebcbb12 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:23:17.827663 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:23:17.838047 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.auth.core [None req-2c380e5d-0eb7-4760-9150-a54e3ebcbb12 None None] MFA Rules not processed for user `7ad4d86f4c8f450986870292477c1182`. Rule list: `[]` (Enabled: `True`). {{(pid=81400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 03:23:17.839726 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-2c380e5d-0eb7-4760-9150-a54e3ebcbb12 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:23:17.944706 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 1081/2168] 10.222.0.18 () {58 vars in 958 bytes} [Tue Sep 13 03:23:17 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 03:23:17.946205 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-885ecb81-4080-4ca9-b35c-0933ea44c19c service cinder] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:23:17.947808 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-885ecb81-4080-4ca9-b35c-0933ea44c19c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:23:18.034493 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-885ecb81-4080-4ca9-b35c-0933ea44c19c service cinder] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:23:18.039478 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db784832-baab-4cbc-a4e1-5da20e1fe71d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76c5181fc7be45279ac00017bb4640b3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:23:18.040081 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db784832-baab-4cbc-a4e1-5da20e1fe71d admin admin] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:23:18.040347 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db784832-baab-4cbc-a4e1-5da20e1fe71d admin admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:23:18.040616 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db784832-baab-4cbc-a4e1-5da20e1fe71d admin admin] PATH_INFO: `/v3/projects` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:23:18.041247 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db784832-baab-4cbc-a4e1-5da20e1fe71d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:23:18.043269 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db784832-baab-4cbc-a4e1-5da20e1fe71d admin admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:23:18.069142 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:23:18.076789 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 1088/2169] 10.222.0.18 () {60 vars in 1153 bytes} [Tue Sep 13 03:23:17 2022] POST /identity/v3/projects => generated 369 bytes in 131 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 03:23:18.080566 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c380e5d-0eb7-4760-9150-a54e3ebcbb12 None None] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:23:18.092424 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c380e5d-0eb7-4760-9150-a54e3ebcbb12 None None] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:23:18.165540 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2a47469-2490-4249-b8ab-9b87a25b5b7f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76c5181fc7be45279ac00017bb4640b3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:23:18.166166 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a47469-2490-4249-b8ab-9b87a25b5b7f admin admin] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:23:18.166443 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a47469-2490-4249-b8ab-9b87a25b5b7f admin admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:23:18.166838 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a47469-2490-4249-b8ab-9b87a25b5b7f admin admin] PATH_INFO: `/v3/users` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:23:18.167628 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2a47469-2490-4249-b8ab-9b87a25b5b7f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:23:18.169845 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2a47469-2490-4249-b8ab-9b87a25b5b7f admin admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:23:18.194794 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 1082/2170] 10.222.0.18 () {60 vars in 1144 bytes} [Tue Sep 13 03:23:18 2022] POST /identity/v3/users => generated 291 bytes in 115 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 13 03:23:18.198793 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db784832-baab-4cbc-a4e1-5da20e1fe71d admin admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:23:18.211265 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db784832-baab-4cbc-a4e1-5da20e1fe71d admin admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:23:18.218046 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f36f1475-eb3d-4dec-ae84-febf61b35978 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76c5181fc7be45279ac00017bb4640b3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:23:18.218803 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f36f1475-eb3d-4dec-ae84-febf61b35978 admin admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:23:18.219182 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f36f1475-eb3d-4dec-ae84-febf61b35978 admin admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:23:18.219479 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f36f1475-eb3d-4dec-ae84-febf61b35978 admin admin] PATH_INFO: `/v3/roles` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:23:18.220196 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f36f1475-eb3d-4dec-ae84-febf61b35978 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:23:18.222133 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f36f1475-eb3d-4dec-ae84-febf61b35978 admin admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:23:18.234823 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 1089/2171] 10.222.0.18 () {58 vars in 1122 bytes} [Tue Sep 13 03:23:18 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 03:23:18.238797 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2a47469-2490-4249-b8ab-9b87a25b5b7f admin admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:23:18.249776 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2a47469-2490-4249-b8ab-9b87a25b5b7f admin admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:23:18.254884 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0310227-5cd6-4e5e-82bd-50e735497344 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76c5181fc7be45279ac00017bb4640b3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:23:18.255546 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0310227-5cd6-4e5e-82bd-50e735497344 admin admin] REQUEST_METHOD: `PUT` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:23:18.256378 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0310227-5cd6-4e5e-82bd-50e735497344 admin admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:23:18.256651 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0310227-5cd6-4e5e-82bd-50e735497344 admin admin] PATH_INFO: `/v3/projects/c8bc9301cec443a2b98ec7e663ba6633/users/ab6d438d4909481191953b3185d3f0ce/roles/425d4b481c7846e0bc07e7e805d5b15f` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:23:18.274393 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0310227-5cd6-4e5e-82bd-50e735497344 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c8bc9301cec443a2b98ec7e663ba6633, user_id=ab6d438d4909481191953b3185d3f0ce, role_id=425d4b481c7846e0bc07e7e805d5b15f)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:23:18.277009 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0310227-5cd6-4e5e-82bd-50e735497344 admin admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:23:18.291146 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 1083/2172] 10.222.0.18 () {60 vars in 1483 bytes} [Tue Sep 13 03:23:18 2022] PUT /identity/v3/projects/c8bc9301cec443a2b98ec7e663ba6633/users/ab6d438d4909481191953b3185d3f0ce/roles/425d4b481c7846e0bc07e7e805d5b15f => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 03:23:18.295280 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f36f1475-eb3d-4dec-ae84-febf61b35978 admin admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:23:18.306690 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f36f1475-eb3d-4dec-ae84-febf61b35978 admin admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:23:18.369769 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c49779ca-93e4-4711-91f8-05bce8340c3c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76c5181fc7be45279ac00017bb4640b3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:23:18.370384 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c49779ca-93e4-4711-91f8-05bce8340c3c admin admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:23:18.370694 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c49779ca-93e4-4711-91f8-05bce8340c3c admin admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:23:18.370970 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c49779ca-93e4-4711-91f8-05bce8340c3c admin admin] PATH_INFO: `/v3/roles` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:23:18.371610 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c49779ca-93e4-4711-91f8-05bce8340c3c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:23:18.373386 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c49779ca-93e4-4711-91f8-05bce8340c3c admin admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:23:18.384801 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 1090/2173] 10.222.0.18 () {58 vars in 1122 bytes} [Tue Sep 13 03:23:18 2022] GET /identity/v3/roles => generated 1420 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 03:23:18.388714 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0310227-5cd6-4e5e-82bd-50e735497344 admin admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:23:18.399580 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0310227-5cd6-4e5e-82bd-50e735497344 admin admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:23:18.404396 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d1163b4-fcde-4f89-af52-f550c72e66a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76c5181fc7be45279ac00017bb4640b3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:23:18.405066 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d1163b4-fcde-4f89-af52-f550c72e66a8 admin admin] REQUEST_METHOD: `PUT` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:23:18.405335 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d1163b4-fcde-4f89-af52-f550c72e66a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:23:18.405600 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d1163b4-fcde-4f89-af52-f550c72e66a8 admin admin] PATH_INFO: `/v3/projects/c8bc9301cec443a2b98ec7e663ba6633/users/ab6d438d4909481191953b3185d3f0ce/roles/425d4b481c7846e0bc07e7e805d5b15f` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:23:18.407519 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d1163b4-fcde-4f89-af52-f550c72e66a8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c8bc9301cec443a2b98ec7e663ba6633, user_id=ab6d438d4909481191953b3185d3f0ce, role_id=425d4b481c7846e0bc07e7e805d5b15f)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:23:18.410169 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d1163b4-fcde-4f89-af52-f550c72e66a8 admin admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:23:18.421582 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 1084/2174] 10.222.0.18 () {60 vars in 1483 bytes} [Tue Sep 13 03:23:18 2022] PUT /identity/v3/projects/c8bc9301cec443a2b98ec7e663ba6633/users/ab6d438d4909481191953b3185d3f0ce/roles/425d4b481c7846e0bc07e7e805d5b15f => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 03:23:23.526269 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-065b85b9-4b66-4b35-be9f-06ccfdab3dc9 None None] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:23:23.526849 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-065b85b9-4b66-4b35-be9f-06ccfdab3dc9 None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:23:23.526849 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-065b85b9-4b66-4b35-be9f-06ccfdab3dc9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:23:23.556840 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.auth.core [None req-065b85b9-4b66-4b35-be9f-06ccfdab3dc9 None None] MFA Rules not processed for user `ab6d438d4909481191953b3185d3f0ce`. Rule list: `[]` (Enabled: `True`). {{(pid=81399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 03:23:23.558098 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-065b85b9-4b66-4b35-be9f-06ccfdab3dc9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:23:23.628972 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 1091/2175] 10.222.0.18 () {58 vars in 958 bytes} [Tue Sep 13 03:23:23 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 03:23:23.642718 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d1163b4-fcde-4f89-af52-f550c72e66a8 admin admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:23:23.644519 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-0d1163b4-fcde-4f89-af52-f550c72e66a8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:23:23.701414 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:23:23.725694 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d1163b4-fcde-4f89-af52-f550c72e66a8 admin admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:23:23.731297 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99cd59e8-740c-47cd-97d8-66fb5d37b51b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7377f5bee4a49959182e1bd34bc97a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b9d94f6a68f7470d9494f112175620dc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:23:23.731901 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99cd59e8-740c-47cd-97d8-66fb5d37b51b service nova] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:23:23.732172 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99cd59e8-740c-47cd-97d8-66fb5d37b51b service nova] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:23:23.732438 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99cd59e8-740c-47cd-97d8-66fb5d37b51b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:23:23.790030 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99cd59e8-740c-47cd-97d8-66fb5d37b51b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:23:23.791657 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99cd59e8-740c-47cd-97d8-66fb5d37b51b service nova] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:23:23.802801 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 1085/2176] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Sep 13 03:23:23 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 160 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 03:23:24.992071 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fcb0be1-ae6b-4c3c-b24b-73eeb7f2dba0 None None] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:23:24.992629 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fcb0be1-ae6b-4c3c-b24b-73eeb7f2dba0 None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:23:24.992629 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fcb0be1-ae6b-4c3c-b24b-73eeb7f2dba0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:23:25.014936 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.auth.core [None req-0fcb0be1-ae6b-4c3c-b24b-73eeb7f2dba0 None None] MFA Rules not processed for user `7ad4d86f4c8f450986870292477c1182`. Rule list: `[]` (Enabled: `True`). {{(pid=81399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 03:23:25.016445 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-0fcb0be1-ae6b-4c3c-b24b-73eeb7f2dba0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:23:25.027789 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 1092/2177] 10.222.0.18 () {58 vars in 958 bytes} [Tue Sep 13 03:23:24 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 03:23:33.798447 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=33 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:23:35.007296 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=71 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:23:36.915800 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99cd59e8-740c-47cd-97d8-66fb5d37b51b service nova] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:23:36.917350 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-99cd59e8-740c-47cd-97d8-66fb5d37b51b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:23:36.978672 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99cd59e8-740c-47cd-97d8-66fb5d37b51b service nova] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:23:36.982294 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bcaf353-3114-4356-867b-ad7aa564d0d1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '924ee3f608d34a18b0ddc4695f102a56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b9d94f6a68f7470d9494f112175620dc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:23:36.983042 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bcaf353-3114-4356-867b-ad7aa564d0d1 service cinder] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:23:36.983328 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bcaf353-3114-4356-867b-ad7aa564d0d1 service cinder] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:23:36.983599 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bcaf353-3114-4356-867b-ad7aa564d0d1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:23:37.043982 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bcaf353-3114-4356-867b-ad7aa564d0d1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:23:37.045416 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bcaf353-3114-4356-867b-ad7aa564d0d1 service cinder] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:23:37.055430 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 1086/2178] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Sep 13 03:23:36 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 140 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 03:23:43.037550 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fcb0be1-ae6b-4c3c-b24b-73eeb7f2dba0 None None] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:23:43.102188 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fcb0be1-ae6b-4c3c-b24b-73eeb7f2dba0 None None] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:23:43.106684 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b68d73f-584e-416c-b2e3-68ab2ea6ea14 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76c5181fc7be45279ac00017bb4640b3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:23:43.107305 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b68d73f-584e-416c-b2e3-68ab2ea6ea14 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:23:43.107581 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b68d73f-584e-416c-b2e3-68ab2ea6ea14 admin admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:23:43.107993 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b68d73f-584e-416c-b2e3-68ab2ea6ea14 admin admin] PATH_INFO: `/v3/users/f00292e69fd645fd91014047d35bbe68` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:23:43.109112 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b68d73f-584e-416c-b2e3-68ab2ea6ea14 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f00292e69fd645fd91014047d35bbe68)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:23:43.111049 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b68d73f-584e-416c-b2e3-68ab2ea6ea14 admin admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:23:43.155347 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-8b68d73f-584e-416c-b2e3-68ab2ea6ea14 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f00292e69fd645fd91014047d35bbe68'} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:23:43.162908 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-8b68d73f-584e-416c-b2e3-68ab2ea6ea14 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f00292e69fd645fd91014047d35bbe68'} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:23:43.181047 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-8b68d73f-584e-416c-b2e3-68ab2ea6ea14 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f00292e69fd645fd91014047d35bbe68'} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:23:43.181825 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-8b68d73f-584e-416c-b2e3-68ab2ea6ea14 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f00292e69fd645fd91014047d35bbe68'} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:23:43.188735 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-8b68d73f-584e-416c-b2e3-68ab2ea6ea14 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f00292e69fd645fd91014047d35bbe68'} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:23:43.194541 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 1093/2179] 10.222.0.18 () {58 vars in 1224 bytes} [Tue Sep 13 03:23:43 2022] DELETE /identity/v3/users/f00292e69fd645fd91014047d35bbe68 => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 03:23:43.199678 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a35331-89bb-492b-8136-0a363399664a None None] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:23:43.200150 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a35331-89bb-492b-8136-0a363399664a None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:23:43.200260 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a35331-89bb-492b-8136-0a363399664a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:23:43.219118 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.auth.core [None req-82a35331-89bb-492b-8136-0a363399664a None None] MFA Rules not processed for user `49f280ab086b4610855c4ffa378517d9`. Rule list: `[]` (Enabled: `True`). {{(pid=81400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 03:23:43.220328 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-82a35331-89bb-492b-8136-0a363399664a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:23:43.280360 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 1087/2180] 10.222.0.18 () {58 vars in 958 bytes} [Tue Sep 13 03:23:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 03:23:43.289972 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b68d73f-584e-416c-b2e3-68ab2ea6ea14 admin admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:23:43.291437 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-8b68d73f-584e-416c-b2e3-68ab2ea6ea14 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:23:43.355315 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b68d73f-584e-416c-b2e3-68ab2ea6ea14 admin admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:23:43.358921 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a6e1c4b-0426-4cbe-ab1d-d3059e4303ff service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdffbc851086441ebd01022d1e4c2609', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b9d94f6a68f7470d9494f112175620dc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:23:43.359563 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a6e1c4b-0426-4cbe-ab1d-d3059e4303ff service neutron] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:23:43.359838 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a6e1c4b-0426-4cbe-ab1d-d3059e4303ff service neutron] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:23:43.360110 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a6e1c4b-0426-4cbe-ab1d-d3059e4303ff service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:23:43.369256 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a6e1c4b-0426-4cbe-ab1d-d3059e4303ff service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:23:43.370714 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a6e1c4b-0426-4cbe-ab1d-d3059e4303ff service neutron] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:23:43.380549 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 1094/2181] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Sep 13 03:23:43 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 13 03:23:43.657554 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82a35331-89bb-492b-8136-0a363399664a None None] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:23:43.659314 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-82a35331-89bb-492b-8136-0a363399664a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:23:43.672564 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82a35331-89bb-492b-8136-0a363399664a None None] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:23:43.677535 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dbb2cb4-1d2b-4544-8873-b4adc155ae1a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76c5181fc7be45279ac00017bb4640b3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:23:43.678273 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dbb2cb4-1d2b-4544-8873-b4adc155ae1a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:23:43.678558 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dbb2cb4-1d2b-4544-8873-b4adc155ae1a admin admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:23:43.678867 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dbb2cb4-1d2b-4544-8873-b4adc155ae1a admin admin] PATH_INFO: `/v3/projects/ea39509ee9414e6cb73af56498a826d9` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:23:43.682643 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dbb2cb4-1d2b-4544-8873-b4adc155ae1a admin admin] RBAC: Authorizing `identity:delete_project(project_id=ea39509ee9414e6cb73af56498a826d9)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:23:43.685014 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dbb2cb4-1d2b-4544-8873-b4adc155ae1a admin admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:23:43.730996 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:23:43.734182 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-1dbb2cb4-1d2b-4544-8873-b4adc155ae1a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ea39509ee9414e6cb73af56498a826d9', 'request_id': 'req-1dbb2cb4-1d2b-4544-8873-b4adc155ae1a'} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:23:43.740214 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-1dbb2cb4-1d2b-4544-8873-b4adc155ae1a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ea39509ee9414e6cb73af56498a826d9', 'request_id': 'req-1dbb2cb4-1d2b-4544-8873-b4adc155ae1a'} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:23:43.746987 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-1dbb2cb4-1d2b-4544-8873-b4adc155ae1a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ea39509ee9414e6cb73af56498a826d9', 'request_id': 'req-1dbb2cb4-1d2b-4544-8873-b4adc155ae1a'} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:23:43.760317 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-1dbb2cb4-1d2b-4544-8873-b4adc155ae1a admin admin] The token cache is being invalidate because project ea39509ee9414e6cb73af56498a826d9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81400) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 03:23:43.760605 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-1dbb2cb4-1d2b-4544-8873-b4adc155ae1a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:23:43.761223 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-1dbb2cb4-1d2b-4544-8873-b4adc155ae1a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:23:43.762736 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 1088/2182] 10.222.0.18 () {58 vars in 1233 bytes} [Tue Sep 13 03:23:43 2022] DELETE /identity/v3/projects/ea39509ee9414e6cb73af56498a826d9 => generated 0 bytes in 106 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 03:23:43.766846 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a6e1c4b-0426-4cbe-ab1d-d3059e4303ff service neutron] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:23:43.776297 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a6e1c4b-0426-4cbe-ab1d-d3059e4303ff service neutron] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:23:43.795231 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:23:43.836710 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e662282-b3da-498e-9b08-68840609c782 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76c5181fc7be45279ac00017bb4640b3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:23:43.837344 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e662282-b3da-498e-9b08-68840609c782 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:23:43.837621 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e662282-b3da-498e-9b08-68840609c782 admin admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:23:43.837891 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e662282-b3da-498e-9b08-68840609c782 admin admin] PATH_INFO: `/v3/users/f2c5d1b2e2ad4346bf58e8df76618b51` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:23:43.839046 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e662282-b3da-498e-9b08-68840609c782 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f2c5d1b2e2ad4346bf58e8df76618b51)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:23:43.840970 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e662282-b3da-498e-9b08-68840609c782 admin admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:23:43.886797 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-0e662282-b3da-498e-9b08-68840609c782 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f2c5d1b2e2ad4346bf58e8df76618b51'} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:23:43.897268 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-0e662282-b3da-498e-9b08-68840609c782 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f2c5d1b2e2ad4346bf58e8df76618b51'} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:23:43.910531 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-0e662282-b3da-498e-9b08-68840609c782 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f2c5d1b2e2ad4346bf58e8df76618b51'} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:23:43.911176 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-0e662282-b3da-498e-9b08-68840609c782 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f2c5d1b2e2ad4346bf58e8df76618b51'} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:23:43.919039 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-0e662282-b3da-498e-9b08-68840609c782 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f2c5d1b2e2ad4346bf58e8df76618b51'} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:23:43.925026 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 1095/2183] 10.222.0.18 () {58 vars in 1224 bytes} [Tue Sep 13 03:23:43 2022] DELETE /identity/v3/users/f2c5d1b2e2ad4346bf58e8df76618b51 => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 03:23:44.291307 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dbb2cb4-1d2b-4544-8873-b4adc155ae1a admin admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:23:44.292810 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-1dbb2cb4-1d2b-4544-8873-b4adc155ae1a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:23:44.308199 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dbb2cb4-1d2b-4544-8873-b4adc155ae1a admin admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:23:44.368275 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2f16281-3d7b-45bc-8525-95270905ca65 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76c5181fc7be45279ac00017bb4640b3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:23:44.368909 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2f16281-3d7b-45bc-8525-95270905ca65 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:23:44.369181 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2f16281-3d7b-45bc-8525-95270905ca65 admin admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:23:44.369447 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2f16281-3d7b-45bc-8525-95270905ca65 admin admin] PATH_INFO: `/v3/projects/17774109433a48b399a928808d16f7d6` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:23:44.370544 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2f16281-3d7b-45bc-8525-95270905ca65 admin admin] RBAC: Authorizing `identity:delete_project(project_id=17774109433a48b399a928808d16f7d6)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:23:44.372481 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2f16281-3d7b-45bc-8525-95270905ca65 admin admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:23:44.419942 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-d2f16281-3d7b-45bc-8525-95270905ca65 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '17774109433a48b399a928808d16f7d6', 'request_id': 'req-d2f16281-3d7b-45bc-8525-95270905ca65'} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:23:44.426188 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-d2f16281-3d7b-45bc-8525-95270905ca65 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '17774109433a48b399a928808d16f7d6', 'request_id': 'req-d2f16281-3d7b-45bc-8525-95270905ca65'} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:23:44.432677 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-d2f16281-3d7b-45bc-8525-95270905ca65 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '17774109433a48b399a928808d16f7d6', 'request_id': 'req-d2f16281-3d7b-45bc-8525-95270905ca65'} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:23:44.445077 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-d2f16281-3d7b-45bc-8525-95270905ca65 admin admin] The token cache is being invalidate because project 17774109433a48b399a928808d16f7d6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81400) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 03:23:44.445366 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-d2f16281-3d7b-45bc-8525-95270905ca65 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:23:44.445993 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-d2f16281-3d7b-45bc-8525-95270905ca65 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:23:44.447603 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 1089/2184] 10.222.0.18 () {58 vars in 1233 bytes} [Tue Sep 13 03:23:44 2022] DELETE /identity/v3/projects/17774109433a48b399a928808d16f7d6 => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 03:23:44.453613 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb1d691-6ac0-41fb-8cdf-15232ec95ec4 None None] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:23:44.454091 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb1d691-6ac0-41fb-8cdf-15232ec95ec4 None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:23:44.454188 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb1d691-6ac0-41fb-8cdf-15232ec95ec4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:23:44.474535 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.auth.core [None req-9cb1d691-6ac0-41fb-8cdf-15232ec95ec4 None None] MFA Rules not processed for user `49f280ab086b4610855c4ffa378517d9`. Rule list: `[]` (Enabled: `True`). {{(pid=81399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 03:23:44.475754 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-9cb1d691-6ac0-41fb-8cdf-15232ec95ec4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:23:44.554851 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 1096/2185] 10.222.0.18 () {58 vars in 958 bytes} [Tue Sep 13 03:23:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 03:23:44.556044 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcd8c6f6-16ab-4419-b13d-e1aa6da64a58 None None] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:23:44.556293 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcd8c6f6-16ab-4419-b13d-e1aa6da64a58 None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:23:44.556565 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcd8c6f6-16ab-4419-b13d-e1aa6da64a58 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:23:44.587150 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.auth.core [None req-dcd8c6f6-16ab-4419-b13d-e1aa6da64a58 None None] MFA Rules not processed for user `49f280ab086b4610855c4ffa378517d9`. Rule list: `[]` (Enabled: `True`). {{(pid=81400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 03:23:44.588364 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-dcd8c6f6-16ab-4419-b13d-e1aa6da64a58 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:23:44.599253 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 1090/2186] 10.222.0.18 () {58 vars in 958 bytes} [Tue Sep 13 03:23:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 03:23:44.604197 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cb1d691-6ac0-41fb-8cdf-15232ec95ec4 None None] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:23:44.617893 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cb1d691-6ac0-41fb-8cdf-15232ec95ec4 None None] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:23:44.622703 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dda586b0-3959-4982-91b4-66c6b92038da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76c5181fc7be45279ac00017bb4640b3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:23:44.623308 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda586b0-3959-4982-91b4-66c6b92038da admin admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:23:44.623589 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda586b0-3959-4982-91b4-66c6b92038da admin admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:23:44.623857 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda586b0-3959-4982-91b4-66c6b92038da admin admin] PATH_INFO: `/v3/domains` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:23:44.624503 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dda586b0-3959-4982-91b4-66c6b92038da admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 03:23:44.624806 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dda586b0-3959-4982-91b4-66c6b92038da admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:23:44.626422 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dda586b0-3959-4982-91b4-66c6b92038da admin admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:23:44.644400 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 1097/2187] 10.222.0.18 () {58 vars in 1153 bytes} [Tue Sep 13 03:23:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 03:23:44.645810 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcd8c6f6-16ab-4419-b13d-e1aa6da64a58 None None] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:23:44.655935 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcd8c6f6-16ab-4419-b13d-e1aa6da64a58 None None] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:23:44.661553 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af8addfe-f5f9-4338-8c74-80fd0fb96be9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76c5181fc7be45279ac00017bb4640b3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:23:44.662268 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af8addfe-f5f9-4338-8c74-80fd0fb96be9 admin admin] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:23:44.662544 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af8addfe-f5f9-4338-8c74-80fd0fb96be9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:23:44.662920 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af8addfe-f5f9-4338-8c74-80fd0fb96be9 admin admin] PATH_INFO: `/v3/projects` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:23:44.663563 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af8addfe-f5f9-4338-8c74-80fd0fb96be9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:23:44.666178 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af8addfe-f5f9-4338-8c74-80fd0fb96be9 admin admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:23:44.692653 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 1091/2188] 10.222.0.18 () {60 vars in 1153 bytes} [Tue Sep 13 03:23:44 2022] POST /identity/v3/projects => generated 367 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 13 03:23:44.697383 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dda586b0-3959-4982-91b4-66c6b92038da admin admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:23:44.707563 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dda586b0-3959-4982-91b4-66c6b92038da admin admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:23:44.735521 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:23:44.767351 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b212e34e-5e3c-4ead-95b4-ad241d450ffb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76c5181fc7be45279ac00017bb4640b3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:23:44.768033 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b212e34e-5e3c-4ead-95b4-ad241d450ffb admin admin] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:23:44.768222 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b212e34e-5e3c-4ead-95b4-ad241d450ffb admin admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:23:44.768493 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b212e34e-5e3c-4ead-95b4-ad241d450ffb admin admin] PATH_INFO: `/v3/users` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:23:44.770264 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b212e34e-5e3c-4ead-95b4-ad241d450ffb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:23:44.771066 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b212e34e-5e3c-4ead-95b4-ad241d450ffb admin admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:23:44.794079 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 1098/2189] 10.222.0.18 () {60 vars in 1144 bytes} [Tue Sep 13 03:23:44 2022] POST /identity/v3/users => generated 284 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 03:23:44.798264 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af8addfe-f5f9-4338-8c74-80fd0fb96be9 admin admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:23:44.808581 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af8addfe-f5f9-4338-8c74-80fd0fb96be9 admin admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:23:44.813383 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73e699ad-3c5e-410f-bd92-0d707c34b969 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76c5181fc7be45279ac00017bb4640b3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:23:44.813974 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73e699ad-3c5e-410f-bd92-0d707c34b969 admin admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:23:44.814251 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73e699ad-3c5e-410f-bd92-0d707c34b969 admin admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:23:44.814520 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73e699ad-3c5e-410f-bd92-0d707c34b969 admin admin] PATH_INFO: `/v3/roles` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:23:44.815194 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73e699ad-3c5e-410f-bd92-0d707c34b969 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:23:44.816789 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73e699ad-3c5e-410f-bd92-0d707c34b969 admin admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:23:44.827311 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 1092/2190] 10.222.0.18 () {58 vars in 1122 bytes} [Tue Sep 13 03:23:44 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 03:23:44.831227 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b212e34e-5e3c-4ead-95b4-ad241d450ffb admin admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:23:44.841960 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b212e34e-5e3c-4ead-95b4-ad241d450ffb admin admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:23:44.846930 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8142304e-db19-4981-ba9b-676be53e11e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76c5181fc7be45279ac00017bb4640b3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:23:44.847534 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8142304e-db19-4981-ba9b-676be53e11e2 admin admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:23:44.847814 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8142304e-db19-4981-ba9b-676be53e11e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:23:44.848090 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8142304e-db19-4981-ba9b-676be53e11e2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:23:44.848739 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8142304e-db19-4981-ba9b-676be53e11e2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:23:44.850332 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8142304e-db19-4981-ba9b-676be53e11e2 admin admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:23:44.862000 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 1099/2191] 10.222.0.18 () {58 vars in 1122 bytes} [Tue Sep 13 03:23:44 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 03:23:44.866157 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73e699ad-3c5e-410f-bd92-0d707c34b969 admin admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:23:44.876169 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73e699ad-3c5e-410f-bd92-0d707c34b969 admin admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:23:44.880980 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e0f36b1-ad91-4ccf-9783-d12ca34ceeac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76c5181fc7be45279ac00017bb4640b3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:23:44.881625 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e0f36b1-ad91-4ccf-9783-d12ca34ceeac admin admin] REQUEST_METHOD: `PUT` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:23:44.881904 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e0f36b1-ad91-4ccf-9783-d12ca34ceeac admin admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:23:44.882176 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e0f36b1-ad91-4ccf-9783-d12ca34ceeac admin admin] PATH_INFO: `/v3/projects/e0f91d4de87444d58981b02025f369c1/users/c06c5393c7ab4034901ce081292f0eca/roles/1eb7a5b5203945c29e6d22d653f47847` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:23:44.898032 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e0f36b1-ad91-4ccf-9783-d12ca34ceeac admin admin] RBAC: Authorizing `identity:create_grant(project_id=e0f91d4de87444d58981b02025f369c1, user_id=c06c5393c7ab4034901ce081292f0eca, role_id=1eb7a5b5203945c29e6d22d653f47847)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:23:44.901341 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e0f36b1-ad91-4ccf-9783-d12ca34ceeac admin admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:23:44.909932 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:DELETE=11,keystone:SELECT=85,keystone:INSERT=4 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:23:44.915525 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 1093/2192] 10.222.0.18 () {60 vars in 1483 bytes} [Tue Sep 13 03:23:44 2022] PUT /identity/v3/projects/e0f91d4de87444d58981b02025f369c1/users/c06c5393c7ab4034901ce081292f0eca/roles/1eb7a5b5203945c29e6d22d653f47847 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 03:23:44.927117 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba49615-7a0a-4cd9-b5b9-4789d6fbe94e None None] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:23:44.927404 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba49615-7a0a-4cd9-b5b9-4789d6fbe94e None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:23:44.927674 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba49615-7a0a-4cd9-b5b9-4789d6fbe94e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:23:44.956848 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.auth.core [None req-aba49615-7a0a-4cd9-b5b9-4789d6fbe94e None None] MFA Rules not processed for user `c06c5393c7ab4034901ce081292f0eca`. Rule list: `[]` (Enabled: `True`). {{(pid=81399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 03:23:44.958017 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-aba49615-7a0a-4cd9-b5b9-4789d6fbe94e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:23:45.022627 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 1100/2193] 10.222.0.18 () {58 vars in 958 bytes} [Tue Sep 13 03:23:44 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 03:23:45.035357 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e0f36b1-ad91-4ccf-9783-d12ca34ceeac admin admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:23:45.037067 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-3e0f36b1-ad91-4ccf-9783-d12ca34ceeac admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:23:45.122728 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e0f36b1-ad91-4ccf-9783-d12ca34ceeac admin admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:23:45.126483 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2759be6b-8a2d-489b-914a-8497fadfcffa service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b5fdc2ff46a14638b409a13e73ba5500', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b9d94f6a68f7470d9494f112175620dc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:23:45.127119 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2759be6b-8a2d-489b-914a-8497fadfcffa service glance] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:23:45.127397 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2759be6b-8a2d-489b-914a-8497fadfcffa service glance] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:23:45.127663 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2759be6b-8a2d-489b-914a-8497fadfcffa service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:23:45.136604 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2759be6b-8a2d-489b-914a-8497fadfcffa service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:23:45.138030 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2759be6b-8a2d-489b-914a-8497fadfcffa service glance] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:23:45.147456 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 1094/2194] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Sep 13 03:23:45 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 03:23:50.693769 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf82f50-6f65-4440-adf6-18651449193d None None] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:23:50.694294 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf82f50-6f65-4440-adf6-18651449193d None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:23:50.694294 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf82f50-6f65-4440-adf6-18651449193d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:23:50.712865 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.auth.core [None req-fbf82f50-6f65-4440-adf6-18651449193d None None] MFA Rules not processed for user `b85258f01e1f436585eb6b9a4f3813b8`. Rule list: `[]` (Enabled: `True`). {{(pid=81399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 03:23:50.714046 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-fbf82f50-6f65-4440-adf6-18651449193d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:23:50.766882 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 1101/2195] 10.222.0.18 () {58 vars in 958 bytes} [Tue Sep 13 03:23:50 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 74 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 03:23:55.142821 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=35 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:23:59.957775 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2759be6b-8a2d-489b-914a-8497fadfcffa service glance] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:23:59.959442 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-2759be6b-8a2d-489b-914a-8497fadfcffa service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:24:00.035940 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2759be6b-8a2d-489b-914a-8497fadfcffa service glance] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:24:00.040893 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97285d0a-b998-49cb-b28a-4a031d79e43d service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b99f1b2374d4d2f8db78306316180d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b9d94f6a68f7470d9494f112175620dc', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:24:00.041521 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97285d0a-b998-49cb-b28a-4a031d79e43d service placement] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:24:00.041799 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97285d0a-b998-49cb-b28a-4a031d79e43d service placement] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:24:00.042072 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97285d0a-b998-49cb-b28a-4a031d79e43d service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:24:00.044161 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-97285d0a-b998-49cb-b28a-4a031d79e43d service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:24:00.056336 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97285d0a-b998-49cb-b28a-4a031d79e43d service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:24:00.062422 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97285d0a-b998-49cb-b28a-4a031d79e43d service placement] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:24:00.071036 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 1095/2196] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Sep 13 03:23:59 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 03:24:00.760551 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=78,keystone:INSERT=3 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:24:08.991452 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c55a5679-0369-4015-a7f4-63e815524f10 None None] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:24:08.992146 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c55a5679-0369-4015-a7f4-63e815524f10 None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:24:08.992146 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c55a5679-0369-4015-a7f4-63e815524f10 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:24:09.011802 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.auth.core [None req-c55a5679-0369-4015-a7f4-63e815524f10 None None] MFA Rules not processed for user `466f990dcac84f4faec701b28616273e`. Rule list: `[]` (Enabled: `True`). {{(pid=81399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 03:24:09.013050 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-c55a5679-0369-4015-a7f4-63e815524f10 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:24:09.074774 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 1102/2197] 10.222.0.18 () {58 vars in 958 bytes} [Tue Sep 13 03:24:08 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 03:24:10.071139 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=32 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:24:19.065316 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=25 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:24:47.372759 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97285d0a-b998-49cb-b28a-4a031d79e43d service placement] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:24:47.374207 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-97285d0a-b998-49cb-b28a-4a031d79e43d service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:24:47.429863 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97285d0a-b998-49cb-b28a-4a031d79e43d service placement] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:24:47.433599 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5d91c90-5d21-4f75-bcb8-5eccafe0f67e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdffbc851086441ebd01022d1e4c2609', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b9d94f6a68f7470d9494f112175620dc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:24:47.434186 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5d91c90-5d21-4f75-bcb8-5eccafe0f67e service neutron] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:24:47.434457 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5d91c90-5d21-4f75-bcb8-5eccafe0f67e service neutron] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:24:47.434808 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5d91c90-5d21-4f75-bcb8-5eccafe0f67e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:24:47.436551 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-f5d91c90-5d21-4f75-bcb8-5eccafe0f67e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:24:47.501032 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5d91c90-5d21-4f75-bcb8-5eccafe0f67e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:24:47.502573 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5d91c90-5d21-4f75-bcb8-5eccafe0f67e service neutron] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:24:47.513244 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 1096/2198] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Sep 13 03:24:47 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 141 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 03:24:50.530603 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c55a5679-0369-4015-a7f4-63e815524f10 None None] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:24:50.592333 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c55a5679-0369-4015-a7f4-63e815524f10 None None] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:24:50.596854 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5202fe7-b008-41b1-99a5-29b37bedfe43 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76c5181fc7be45279ac00017bb4640b3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:24:50.597481 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5202fe7-b008-41b1-99a5-29b37bedfe43 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:24:50.597753 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5202fe7-b008-41b1-99a5-29b37bedfe43 admin admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:24:50.598025 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5202fe7-b008-41b1-99a5-29b37bedfe43 admin admin] PATH_INFO: `/v3/users/c06c5393c7ab4034901ce081292f0eca` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:24:50.599268 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5202fe7-b008-41b1-99a5-29b37bedfe43 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c06c5393c7ab4034901ce081292f0eca)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:24:50.601767 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5202fe7-b008-41b1-99a5-29b37bedfe43 admin admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:24:50.645375 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-f5202fe7-b008-41b1-99a5-29b37bedfe43 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c06c5393c7ab4034901ce081292f0eca'} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:24:50.652790 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-f5202fe7-b008-41b1-99a5-29b37bedfe43 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c06c5393c7ab4034901ce081292f0eca'} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:24:50.664819 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-f5202fe7-b008-41b1-99a5-29b37bedfe43 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c06c5393c7ab4034901ce081292f0eca'} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:24:50.665438 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-f5202fe7-b008-41b1-99a5-29b37bedfe43 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c06c5393c7ab4034901ce081292f0eca'} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:24:50.672513 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-f5202fe7-b008-41b1-99a5-29b37bedfe43 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c06c5393c7ab4034901ce081292f0eca'} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:24:50.678420 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 1103/2199] 10.222.0.18 () {58 vars in 1224 bytes} [Tue Sep 13 03:24:50 2022] DELETE /identity/v3/users/c06c5393c7ab4034901ce081292f0eca => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 03:24:50.682950 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a2354a-a563-44f5-bf88-c718e72c9e32 None None] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:24:50.683524 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a2354a-a563-44f5-bf88-c718e72c9e32 None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:24:50.683524 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a2354a-a563-44f5-bf88-c718e72c9e32 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:24:50.702595 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.auth.core [None req-d5a2354a-a563-44f5-bf88-c718e72c9e32 None None] MFA Rules not processed for user `49f280ab086b4610855c4ffa378517d9`. Rule list: `[]` (Enabled: `True`). {{(pid=81400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 03:24:50.703828 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-d5a2354a-a563-44f5-bf88-c718e72c9e32 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:24:50.761959 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 1097/2200] 10.222.0.18 () {58 vars in 958 bytes} [Tue Sep 13 03:24:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 03:24:50.772331 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5202fe7-b008-41b1-99a5-29b37bedfe43 admin admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:24:50.773772 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-f5202fe7-b008-41b1-99a5-29b37bedfe43 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:24:50.830593 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5202fe7-b008-41b1-99a5-29b37bedfe43 admin admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:24:50.834949 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcd383dc-942e-4c83-ab87-d4c743260c63 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdffbc851086441ebd01022d1e4c2609', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b9d94f6a68f7470d9494f112175620dc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:24:50.835610 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd383dc-942e-4c83-ab87-d4c743260c63 service neutron] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:24:50.835834 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd383dc-942e-4c83-ab87-d4c743260c63 service neutron] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:24:50.836103 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd383dc-942e-4c83-ab87-d4c743260c63 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:24:50.844924 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcd383dc-942e-4c83-ab87-d4c743260c63 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:24:50.846354 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcd383dc-942e-4c83-ab87-d4c743260c63 service neutron] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:24:50.855746 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 1104/2201] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Sep 13 03:24:50 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 03:24:51.131994 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5a2354a-a563-44f5-bf88-c718e72c9e32 None None] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:24:51.134077 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-d5a2354a-a563-44f5-bf88-c718e72c9e32 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:24:51.146334 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5a2354a-a563-44f5-bf88-c718e72c9e32 None None] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:24:51.150894 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87d96ba7-c028-45f7-a765-d455017a6001 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76c5181fc7be45279ac00017bb4640b3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:24:51.151517 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d96ba7-c028-45f7-a765-d455017a6001 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:24:51.151797 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d96ba7-c028-45f7-a765-d455017a6001 admin admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:24:51.152048 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d96ba7-c028-45f7-a765-d455017a6001 admin admin] PATH_INFO: `/v3/projects/e0f91d4de87444d58981b02025f369c1` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:24:51.153142 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87d96ba7-c028-45f7-a765-d455017a6001 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e0f91d4de87444d58981b02025f369c1)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:24:51.155909 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87d96ba7-c028-45f7-a765-d455017a6001 admin admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:24:51.200489 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:24:51.204247 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-87d96ba7-c028-45f7-a765-d455017a6001 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e0f91d4de87444d58981b02025f369c1', 'request_id': 'req-87d96ba7-c028-45f7-a765-d455017a6001'} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:24:51.210471 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-87d96ba7-c028-45f7-a765-d455017a6001 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e0f91d4de87444d58981b02025f369c1', 'request_id': 'req-87d96ba7-c028-45f7-a765-d455017a6001'} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:24:51.216844 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-87d96ba7-c028-45f7-a765-d455017a6001 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e0f91d4de87444d58981b02025f369c1', 'request_id': 'req-87d96ba7-c028-45f7-a765-d455017a6001'} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:24:51.230149 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-87d96ba7-c028-45f7-a765-d455017a6001 admin admin] The token cache is being invalidate because project e0f91d4de87444d58981b02025f369c1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81400) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 03:24:51.230435 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-87d96ba7-c028-45f7-a765-d455017a6001 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:24:51.231083 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-87d96ba7-c028-45f7-a765-d455017a6001 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:24:51.232610 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 1098/2202] 10.222.0.18 () {58 vars in 1233 bytes} [Tue Sep 13 03:24:51 2022] DELETE /identity/v3/projects/e0f91d4de87444d58981b02025f369c1 => generated 0 bytes in 101 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 03:24:51.237859 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be906101-9722-4077-b109-e2997276a35d None None] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:24:51.238150 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be906101-9722-4077-b109-e2997276a35d None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:24:51.238427 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be906101-9722-4077-b109-e2997276a35d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:24:51.259174 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.auth.core [None req-be906101-9722-4077-b109-e2997276a35d None None] MFA Rules not processed for user `49f280ab086b4610855c4ffa378517d9`. Rule list: `[]` (Enabled: `True`). {{(pid=81399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 03:24:51.260044 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-be906101-9722-4077-b109-e2997276a35d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:24:51.276893 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:24:51.322204 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 1105/2203] 10.222.0.18 () {58 vars in 958 bytes} [Tue Sep 13 03:24:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 03:24:51.330185 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d11ef57-fdb9-4694-a8c9-a2128a2399d4 None None] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:24:51.330467 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d11ef57-fdb9-4694-a8c9-a2128a2399d4 None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:24:51.330799 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d11ef57-fdb9-4694-a8c9-a2128a2399d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:24:51.350406 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.auth.core [None req-7d11ef57-fdb9-4694-a8c9-a2128a2399d4 None None] MFA Rules not processed for user `49f280ab086b4610855c4ffa378517d9`. Rule list: `[]` (Enabled: `True`). {{(pid=81400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 03:24:51.351696 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-7d11ef57-fdb9-4694-a8c9-a2128a2399d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:24:51.359438 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 1099/2204] 10.222.0.18 () {58 vars in 958 bytes} [Tue Sep 13 03:24:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 03:24:51.363633 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be906101-9722-4077-b109-e2997276a35d None None] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:24:51.373728 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be906101-9722-4077-b109-e2997276a35d None None] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:24:51.378297 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b10ab10-c305-4bed-a216-aec0f7357873 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76c5181fc7be45279ac00017bb4640b3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:24:51.378921 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b10ab10-c305-4bed-a216-aec0f7357873 admin admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:24:51.379208 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b10ab10-c305-4bed-a216-aec0f7357873 admin admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:24:51.379477 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b10ab10-c305-4bed-a216-aec0f7357873 admin admin] PATH_INFO: `/v3/domains` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:24:51.380117 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b10ab10-c305-4bed-a216-aec0f7357873 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 03:24:51.380421 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b10ab10-c305-4bed-a216-aec0f7357873 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:24:51.382718 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b10ab10-c305-4bed-a216-aec0f7357873 admin admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:24:51.394256 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 1106/2205] 10.222.0.18 () {58 vars in 1153 bytes} [Tue Sep 13 03:24:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 03:24:51.398310 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d11ef57-fdb9-4694-a8c9-a2128a2399d4 None None] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:24:51.407384 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d11ef57-fdb9-4694-a8c9-a2128a2399d4 None None] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:24:51.412020 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4696e81-e2fc-4db6-b9c5-85b1b196e7c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76c5181fc7be45279ac00017bb4640b3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:24:51.413094 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4696e81-e2fc-4db6-b9c5-85b1b196e7c8 admin admin] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:24:51.413368 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4696e81-e2fc-4db6-b9c5-85b1b196e7c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:24:51.413635 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4696e81-e2fc-4db6-b9c5-85b1b196e7c8 admin admin] PATH_INFO: `/v3/projects` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:24:51.414254 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4696e81-e2fc-4db6-b9c5-85b1b196e7c8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:24:51.416193 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4696e81-e2fc-4db6-b9c5-85b1b196e7c8 admin admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:24:51.444624 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 1100/2206] 10.222.0.18 () {60 vars in 1153 bytes} [Tue Sep 13 03:24:51 2022] POST /identity/v3/projects => generated 385 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 03:24:51.448714 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b10ab10-c305-4bed-a216-aec0f7357873 admin admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:24:51.458425 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b10ab10-c305-4bed-a216-aec0f7357873 admin admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:24:51.514760 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef67d944-2613-4b33-8d38-4ea7d4c6c7b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76c5181fc7be45279ac00017bb4640b3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:24:51.515362 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef67d944-2613-4b33-8d38-4ea7d4c6c7b0 admin admin] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:24:51.515640 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef67d944-2613-4b33-8d38-4ea7d4c6c7b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:24:51.515913 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef67d944-2613-4b33-8d38-4ea7d4c6c7b0 admin admin] PATH_INFO: `/v3/users` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:24:51.516540 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef67d944-2613-4b33-8d38-4ea7d4c6c7b0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:24:51.518444 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef67d944-2613-4b33-8d38-4ea7d4c6c7b0 admin admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:24:51.539672 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 1107/2207] 10.222.0.18 () {60 vars in 1144 bytes} [Tue Sep 13 03:24:51 2022] POST /identity/v3/users => generated 293 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 13 03:24:51.544101 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4696e81-e2fc-4db6-b9c5-85b1b196e7c8 admin admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:24:51.554766 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4696e81-e2fc-4db6-b9c5-85b1b196e7c8 admin admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:24:51.559200 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24fc018f-1c18-4b12-995c-c7eccadd07bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76c5181fc7be45279ac00017bb4640b3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:24:51.559782 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24fc018f-1c18-4b12-995c-c7eccadd07bd admin admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:24:51.560059 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24fc018f-1c18-4b12-995c-c7eccadd07bd admin admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:24:51.560325 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24fc018f-1c18-4b12-995c-c7eccadd07bd admin admin] PATH_INFO: `/v3/roles` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:24:51.560952 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24fc018f-1c18-4b12-995c-c7eccadd07bd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:24:51.562527 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24fc018f-1c18-4b12-995c-c7eccadd07bd admin admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:24:51.575102 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 1101/2208] 10.222.0.18 () {58 vars in 1122 bytes} [Tue Sep 13 03:24:51 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 03:24:51.576729 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef67d944-2613-4b33-8d38-4ea7d4c6c7b0 admin admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:24:51.586706 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef67d944-2613-4b33-8d38-4ea7d4c6c7b0 admin admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:24:51.591533 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab479687-db7b-45e8-a9a9-6b2f09bc211f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76c5181fc7be45279ac00017bb4640b3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:24:51.592125 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab479687-db7b-45e8-a9a9-6b2f09bc211f admin admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:24:51.592468 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab479687-db7b-45e8-a9a9-6b2f09bc211f admin admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:24:51.592656 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab479687-db7b-45e8-a9a9-6b2f09bc211f admin admin] PATH_INFO: `/v3/roles` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:24:51.593278 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab479687-db7b-45e8-a9a9-6b2f09bc211f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:24:51.594882 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab479687-db7b-45e8-a9a9-6b2f09bc211f admin admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:24:51.606024 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 1108/2209] 10.222.0.18 () {58 vars in 1122 bytes} [Tue Sep 13 03:24:51 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 03:24:51.610328 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24fc018f-1c18-4b12-995c-c7eccadd07bd admin admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:24:51.621402 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24fc018f-1c18-4b12-995c-c7eccadd07bd admin admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:24:51.625990 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-703e796c-c94c-48cd-8bc9-8dd81858e4d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76c5181fc7be45279ac00017bb4640b3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:24:51.626667 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-703e796c-c94c-48cd-8bc9-8dd81858e4d9 admin admin] REQUEST_METHOD: `PUT` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:24:51.626953 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-703e796c-c94c-48cd-8bc9-8dd81858e4d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:24:51.627222 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-703e796c-c94c-48cd-8bc9-8dd81858e4d9 admin admin] PATH_INFO: `/v3/projects/fcee150952d14de897e0881d44fae246/users/7a102ffaedf74ecb97f173a39e34c50f/roles/1eb7a5b5203945c29e6d22d653f47847` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:24:51.641388 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-703e796c-c94c-48cd-8bc9-8dd81858e4d9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fcee150952d14de897e0881d44fae246, user_id=7a102ffaedf74ecb97f173a39e34c50f, role_id=1eb7a5b5203945c29e6d22d653f47847)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:24:51.644124 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-703e796c-c94c-48cd-8bc9-8dd81858e4d9 admin admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:24:51.657138 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 1102/2210] 10.222.0.18 () {60 vars in 1483 bytes} [Tue Sep 13 03:24:51 2022] PUT /identity/v3/projects/fcee150952d14de897e0881d44fae246/users/7a102ffaedf74ecb97f173a39e34c50f/roles/1eb7a5b5203945c29e6d22d653f47847 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 03:24:51.668968 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-424c81ea-7fbe-4d3e-a41a-1034f525816e None None] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:24:51.669254 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-424c81ea-7fbe-4d3e-a41a-1034f525816e None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:24:51.669527 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-424c81ea-7fbe-4d3e-a41a-1034f525816e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:24:51.697083 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.auth.core [None req-424c81ea-7fbe-4d3e-a41a-1034f525816e None None] MFA Rules not processed for user `7a102ffaedf74ecb97f173a39e34c50f`. Rule list: `[]` (Enabled: `True`). {{(pid=81399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 03:24:51.698281 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-424c81ea-7fbe-4d3e-a41a-1034f525816e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:24:51.766768 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 1109/2211] 10.222.0.18 () {58 vars in 958 bytes} [Tue Sep 13 03:24:51 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 03:24:51.779126 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-703e796c-c94c-48cd-8bc9-8dd81858e4d9 admin admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:24:51.780889 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-703e796c-c94c-48cd-8bc9-8dd81858e4d9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:24:51.843048 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-703e796c-c94c-48cd-8bc9-8dd81858e4d9 admin admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:24:51.846891 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba39eae8-ce9d-45be-bc1d-ef4088547df3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '924ee3f608d34a18b0ddc4695f102a56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b9d94f6a68f7470d9494f112175620dc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:24:51.847486 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba39eae8-ce9d-45be-bc1d-ef4088547df3 service cinder] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:24:51.847760 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba39eae8-ce9d-45be-bc1d-ef4088547df3 service cinder] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:24:51.848025 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba39eae8-ce9d-45be-bc1d-ef4088547df3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:24:51.856806 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba39eae8-ce9d-45be-bc1d-ef4088547df3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:24:51.858223 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba39eae8-ce9d-45be-bc1d-ef4088547df3 service cinder] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:24:51.867952 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 1103/2212] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Sep 13 03:24:51 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 03:25:00.723338 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-424c81ea-7fbe-4d3e-a41a-1034f525816e None None] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:25:00.742863 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:25:00.786465 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-424c81ea-7fbe-4d3e-a41a-1034f525816e None None] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:25:00.791663 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bb86198-f295-4ed5-97a4-d6bb00d14c69 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76c5181fc7be45279ac00017bb4640b3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:25:00.792554 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bb86198-f295-4ed5-97a4-d6bb00d14c69 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:25:00.792826 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bb86198-f295-4ed5-97a4-d6bb00d14c69 admin admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:25:00.793098 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bb86198-f295-4ed5-97a4-d6bb00d14c69 admin admin] PATH_INFO: `/v3/users/7a102ffaedf74ecb97f173a39e34c50f` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:25:00.794207 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bb86198-f295-4ed5-97a4-d6bb00d14c69 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7a102ffaedf74ecb97f173a39e34c50f)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:25:00.796151 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bb86198-f295-4ed5-97a4-d6bb00d14c69 admin admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:25:00.841942 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-5bb86198-f295-4ed5-97a4-d6bb00d14c69 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7a102ffaedf74ecb97f173a39e34c50f'} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:25:00.848575 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-5bb86198-f295-4ed5-97a4-d6bb00d14c69 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7a102ffaedf74ecb97f173a39e34c50f'} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:25:00.861137 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-5bb86198-f295-4ed5-97a4-d6bb00d14c69 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7a102ffaedf74ecb97f173a39e34c50f'} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:25:00.861771 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-5bb86198-f295-4ed5-97a4-d6bb00d14c69 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7a102ffaedf74ecb97f173a39e34c50f'} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:25:00.868676 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-5bb86198-f295-4ed5-97a4-d6bb00d14c69 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7a102ffaedf74ecb97f173a39e34c50f'} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:25:00.874480 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 1110/2213] 10.222.0.18 () {58 vars in 1224 bytes} [Tue Sep 13 03:25:00 2022] DELETE /identity/v3/users/7a102ffaedf74ecb97f173a39e34c50f => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 03:25:00.878875 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d1fcb6-cc25-498c-85e9-934492790f6e None None] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:25:00.879353 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d1fcb6-cc25-498c-85e9-934492790f6e None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:25:00.879457 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d1fcb6-cc25-498c-85e9-934492790f6e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:25:00.898789 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.auth.core [None req-58d1fcb6-cc25-498c-85e9-934492790f6e None None] MFA Rules not processed for user `49f280ab086b4610855c4ffa378517d9`. Rule list: `[]` (Enabled: `True`). {{(pid=81400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 03:25:00.899995 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-58d1fcb6-cc25-498c-85e9-934492790f6e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:25:00.957843 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 1104/2214] 10.222.0.18 () {58 vars in 958 bytes} [Tue Sep 13 03:25:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 03:25:00.968017 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bb86198-f295-4ed5-97a4-d6bb00d14c69 admin admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:25:00.969464 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-5bb86198-f295-4ed5-97a4-d6bb00d14c69 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:25:01.031442 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bb86198-f295-4ed5-97a4-d6bb00d14c69 admin admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:25:01.035093 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f376f8b-c232-43d7-9ae6-ef29ac1b3934 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdffbc851086441ebd01022d1e4c2609', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b9d94f6a68f7470d9494f112175620dc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:25:01.035694 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f376f8b-c232-43d7-9ae6-ef29ac1b3934 service neutron] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:25:01.035976 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f376f8b-c232-43d7-9ae6-ef29ac1b3934 service neutron] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:25:01.036249 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f376f8b-c232-43d7-9ae6-ef29ac1b3934 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:25:01.046018 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f376f8b-c232-43d7-9ae6-ef29ac1b3934 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:25:01.047495 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f376f8b-c232-43d7-9ae6-ef29ac1b3934 service neutron] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:25:01.056864 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 1111/2215] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Sep 13 03:25:00 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 03:25:01.338028 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58d1fcb6-cc25-498c-85e9-934492790f6e None None] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:25:01.339582 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-58d1fcb6-cc25-498c-85e9-934492790f6e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:25:01.355098 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58d1fcb6-cc25-498c-85e9-934492790f6e None None] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:25:01.360042 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57979004-d885-4ccf-8f08-1ae2f0521088 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76c5181fc7be45279ac00017bb4640b3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:25:01.360667 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57979004-d885-4ccf-8f08-1ae2f0521088 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:25:01.360938 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57979004-d885-4ccf-8f08-1ae2f0521088 admin admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:25:01.361206 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57979004-d885-4ccf-8f08-1ae2f0521088 admin admin] PATH_INFO: `/v3/projects/fcee150952d14de897e0881d44fae246` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:25:01.363065 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57979004-d885-4ccf-8f08-1ae2f0521088 admin admin] RBAC: Authorizing `identity:delete_project(project_id=fcee150952d14de897e0881d44fae246)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:25:01.365198 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57979004-d885-4ccf-8f08-1ae2f0521088 admin admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:25:01.373760 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:DELETE=4,keystone:SELECT=93,keystone:INSERT=3 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:25:01.415989 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-57979004-d885-4ccf-8f08-1ae2f0521088 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fcee150952d14de897e0881d44fae246', 'request_id': 'req-57979004-d885-4ccf-8f08-1ae2f0521088'} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:25:01.421991 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-57979004-d885-4ccf-8f08-1ae2f0521088 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fcee150952d14de897e0881d44fae246', 'request_id': 'req-57979004-d885-4ccf-8f08-1ae2f0521088'} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:25:01.428376 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-57979004-d885-4ccf-8f08-1ae2f0521088 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fcee150952d14de897e0881d44fae246', 'request_id': 'req-57979004-d885-4ccf-8f08-1ae2f0521088'} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:25:01.440068 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-57979004-d885-4ccf-8f08-1ae2f0521088 admin admin] The token cache is being invalidate because project fcee150952d14de897e0881d44fae246 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81400) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 03:25:01.440351 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-57979004-d885-4ccf-8f08-1ae2f0521088 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:25:01.440963 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-57979004-d885-4ccf-8f08-1ae2f0521088 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:25:01.442444 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 1105/2216] 10.222.0.18 () {58 vars in 1233 bytes} [Tue Sep 13 03:25:01 2022] DELETE /identity/v3/projects/fcee150952d14de897e0881d44fae246 => generated 0 bytes in 105 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 13 03:25:01.447836 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0770321e-84d1-41f7-9cda-ffdf9bbd5c91 None None] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:25:01.448119 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0770321e-84d1-41f7-9cda-ffdf9bbd5c91 None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:25:01.448415 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0770321e-84d1-41f7-9cda-ffdf9bbd5c91 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:25:01.469814 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.auth.core [None req-0770321e-84d1-41f7-9cda-ffdf9bbd5c91 None None] MFA Rules not processed for user `49f280ab086b4610855c4ffa378517d9`. Rule list: `[]` (Enabled: `True`). {{(pid=81399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 03:25:01.471170 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-0770321e-84d1-41f7-9cda-ffdf9bbd5c91 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:25:01.505502 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:25:01.537868 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 1112/2217] 10.222.0.18 () {58 vars in 958 bytes} [Tue Sep 13 03:25:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 13 03:25:01.545874 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-726084cf-7bf0-4931-ba20-9e569a25a18a None None] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:25:01.546160 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-726084cf-7bf0-4931-ba20-9e569a25a18a None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:25:01.546438 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-726084cf-7bf0-4931-ba20-9e569a25a18a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:25:01.566635 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.auth.core [None req-726084cf-7bf0-4931-ba20-9e569a25a18a None None] MFA Rules not processed for user `49f280ab086b4610855c4ffa378517d9`. Rule list: `[]` (Enabled: `True`). {{(pid=81400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 03:25:01.568296 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-726084cf-7bf0-4931-ba20-9e569a25a18a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:25:01.576676 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 1106/2218] 10.222.0.18 () {58 vars in 958 bytes} [Tue Sep 13 03:25:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 03:25:01.581126 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0770321e-84d1-41f7-9cda-ffdf9bbd5c91 None None] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:25:01.593402 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0770321e-84d1-41f7-9cda-ffdf9bbd5c91 None None] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:25:01.597961 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6be0ce6-cb84-4786-b539-d9fcd4eeee6a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76c5181fc7be45279ac00017bb4640b3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:25:01.598558 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6be0ce6-cb84-4786-b539-d9fcd4eeee6a admin admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:25:01.598910 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6be0ce6-cb84-4786-b539-d9fcd4eeee6a admin admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:25:01.599181 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6be0ce6-cb84-4786-b539-d9fcd4eeee6a admin admin] PATH_INFO: `/v3/domains` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:25:01.599868 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6be0ce6-cb84-4786-b539-d9fcd4eeee6a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 03:25:01.600167 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6be0ce6-cb84-4786-b539-d9fcd4eeee6a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:25:01.601750 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6be0ce6-cb84-4786-b539-d9fcd4eeee6a admin admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:25:01.613986 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 1113/2219] 10.222.0.18 () {58 vars in 1153 bytes} [Tue Sep 13 03:25:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 03:25:01.618948 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-726084cf-7bf0-4931-ba20-9e569a25a18a None None] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:25:01.628093 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-726084cf-7bf0-4931-ba20-9e569a25a18a None None] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:25:01.633375 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f653795-aa7f-4c9d-877a-e54fa6cfbfd0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76c5181fc7be45279ac00017bb4640b3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:25:01.633959 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f653795-aa7f-4c9d-877a-e54fa6cfbfd0 admin admin] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:25:01.634231 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f653795-aa7f-4c9d-877a-e54fa6cfbfd0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:25:01.634498 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f653795-aa7f-4c9d-877a-e54fa6cfbfd0 admin admin] PATH_INFO: `/v3/projects` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:25:01.635153 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f653795-aa7f-4c9d-877a-e54fa6cfbfd0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:25:01.637036 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f653795-aa7f-4c9d-877a-e54fa6cfbfd0 admin admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:25:01.664004 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 1107/2220] 10.222.0.18 () {60 vars in 1153 bytes} [Tue Sep 13 03:25:01 2022] POST /identity/v3/projects => generated 365 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 03:25:01.668287 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6be0ce6-cb84-4786-b539-d9fcd4eeee6a admin admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:25:01.678428 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6be0ce6-cb84-4786-b539-d9fcd4eeee6a admin admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:25:01.732616 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7a16eb5-e5a9-4c8d-8164-c02b82b9e975 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76c5181fc7be45279ac00017bb4640b3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:25:01.733326 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a16eb5-e5a9-4c8d-8164-c02b82b9e975 admin admin] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:25:01.733658 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a16eb5-e5a9-4c8d-8164-c02b82b9e975 admin admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:25:01.733874 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a16eb5-e5a9-4c8d-8164-c02b82b9e975 admin admin] PATH_INFO: `/v3/users` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:25:01.734513 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7a16eb5-e5a9-4c8d-8164-c02b82b9e975 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:25:01.736460 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7a16eb5-e5a9-4c8d-8164-c02b82b9e975 admin admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:25:01.759762 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 1114/2221] 10.222.0.18 () {60 vars in 1144 bytes} [Tue Sep 13 03:25:01 2022] POST /identity/v3/users => generated 283 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 03:25:01.763849 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f653795-aa7f-4c9d-877a-e54fa6cfbfd0 admin admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:25:01.774108 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f653795-aa7f-4c9d-877a-e54fa6cfbfd0 admin admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:25:01.779002 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-069137c3-64ff-49aa-b067-61b2e96b808b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76c5181fc7be45279ac00017bb4640b3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:25:01.779605 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-069137c3-64ff-49aa-b067-61b2e96b808b admin admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:25:01.779889 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-069137c3-64ff-49aa-b067-61b2e96b808b admin admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:25:01.780162 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-069137c3-64ff-49aa-b067-61b2e96b808b admin admin] PATH_INFO: `/v3/roles` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:25:01.780803 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-069137c3-64ff-49aa-b067-61b2e96b808b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:25:01.782399 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-069137c3-64ff-49aa-b067-61b2e96b808b admin admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:25:01.792516 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 1108/2222] 10.222.0.18 () {58 vars in 1122 bytes} [Tue Sep 13 03:25:01 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 03:25:01.796414 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7a16eb5-e5a9-4c8d-8164-c02b82b9e975 admin admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:25:01.806569 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7a16eb5-e5a9-4c8d-8164-c02b82b9e975 admin admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:25:01.811111 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d249074-84d8-4767-8815-dd5e716f6af1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76c5181fc7be45279ac00017bb4640b3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:25:01.811689 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d249074-84d8-4767-8815-dd5e716f6af1 admin admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:25:01.811963 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d249074-84d8-4767-8815-dd5e716f6af1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:25:01.812227 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d249074-84d8-4767-8815-dd5e716f6af1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:25:01.812850 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d249074-84d8-4767-8815-dd5e716f6af1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:25:01.814418 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d249074-84d8-4767-8815-dd5e716f6af1 admin admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:25:01.825585 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 1115/2223] 10.222.0.18 () {58 vars in 1122 bytes} [Tue Sep 13 03:25:01 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 13 03:25:01.829635 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-069137c3-64ff-49aa-b067-61b2e96b808b admin admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:25:01.839540 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-069137c3-64ff-49aa-b067-61b2e96b808b admin admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:25:01.844199 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b583df1-dc3b-471d-a9e3-9d100b23cae1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76c5181fc7be45279ac00017bb4640b3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:25:01.844860 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b583df1-dc3b-471d-a9e3-9d100b23cae1 admin admin] REQUEST_METHOD: `PUT` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:25:01.845130 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b583df1-dc3b-471d-a9e3-9d100b23cae1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:25:01.845420 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b583df1-dc3b-471d-a9e3-9d100b23cae1 admin admin] PATH_INFO: `/v3/projects/5bdfaf801af1413293a8a8e390b2e617/users/1ee484b8e3f8481aaf33bd2caf095919/roles/1eb7a5b5203945c29e6d22d653f47847` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:25:01.859730 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b583df1-dc3b-471d-a9e3-9d100b23cae1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5bdfaf801af1413293a8a8e390b2e617, user_id=1ee484b8e3f8481aaf33bd2caf095919, role_id=1eb7a5b5203945c29e6d22d653f47847)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:25:01.862370 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b583df1-dc3b-471d-a9e3-9d100b23cae1 admin admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:25:01.875469 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 1109/2224] 10.222.0.18 () {60 vars in 1483 bytes} [Tue Sep 13 03:25:01 2022] PUT /identity/v3/projects/5bdfaf801af1413293a8a8e390b2e617/users/1ee484b8e3f8481aaf33bd2caf095919/roles/1eb7a5b5203945c29e6d22d653f47847 => generated 0 bytes in 46 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 03:25:01.886055 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feb765f2-7564-41bf-8599-e466b65ff058 None None] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:25:01.886322 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feb765f2-7564-41bf-8599-e466b65ff058 None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:25:01.886588 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feb765f2-7564-41bf-8599-e466b65ff058 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:25:01.913927 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.auth.core [None req-feb765f2-7564-41bf-8599-e466b65ff058 None None] MFA Rules not processed for user `1ee484b8e3f8481aaf33bd2caf095919`. Rule list: `[]` (Enabled: `True`). {{(pid=81399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 03:25:01.915158 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-feb765f2-7564-41bf-8599-e466b65ff058 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:25:01.978296 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 1116/2225] 10.222.0.18 () {58 vars in 958 bytes} [Tue Sep 13 03:25:01 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 03:25:01.989598 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b583df1-dc3b-471d-a9e3-9d100b23cae1 admin admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:25:01.991814 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-7b583df1-dc3b-471d-a9e3-9d100b23cae1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:25:02.046987 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b583df1-dc3b-471d-a9e3-9d100b23cae1 admin admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:25:02.050575 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4af1960-4488-4140-804f-40dce1b4f191 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '924ee3f608d34a18b0ddc4695f102a56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b9d94f6a68f7470d9494f112175620dc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:25:02.051190 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4af1960-4488-4140-804f-40dce1b4f191 service cinder] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:25:02.051465 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4af1960-4488-4140-804f-40dce1b4f191 service cinder] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:25:02.051729 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4af1960-4488-4140-804f-40dce1b4f191 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:25:02.060592 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4af1960-4488-4140-804f-40dce1b4f191 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:25:02.062021 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4af1960-4488-4140-804f-40dce1b4f191 service cinder] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:25:02.071205 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 1110/2226] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Sep 13 03:25:01 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 13 03:25:11.974943 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=81,keystone:INSERT=3 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:25:12.066984 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=74,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:25:46.439607 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feb765f2-7564-41bf-8599-e466b65ff058 None None] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:25:46.440985 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-feb765f2-7564-41bf-8599-e466b65ff058 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:25:46.496432 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feb765f2-7564-41bf-8599-e466b65ff058 None None] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:25:46.500007 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff57e111-b67b-4e61-bc47-c548c9aa6157 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdffbc851086441ebd01022d1e4c2609', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b9d94f6a68f7470d9494f112175620dc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:25:46.500603 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff57e111-b67b-4e61-bc47-c548c9aa6157 service neutron] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:25:46.500881 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff57e111-b67b-4e61-bc47-c548c9aa6157 service neutron] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:25:46.501151 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff57e111-b67b-4e61-bc47-c548c9aa6157 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:25:46.502844 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-ff57e111-b67b-4e61-bc47-c548c9aa6157 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:25:46.571515 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff57e111-b67b-4e61-bc47-c548c9aa6157 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:25:46.572888 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff57e111-b67b-4e61-bc47-c548c9aa6157 service neutron] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:25:46.583651 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 1117/2227] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Sep 13 03:25:46 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 145 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 03:25:50.103561 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4af1960-4488-4140-804f-40dce1b4f191 service cinder] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:25:50.106017 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-c4af1960-4488-4140-804f-40dce1b4f191 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:25:50.122339 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4af1960-4488-4140-804f-40dce1b4f191 service cinder] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:25:50.127248 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae2c2131-ee07-4c13-9a5f-e2d501f05dae service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdffbc851086441ebd01022d1e4c2609', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b9d94f6a68f7470d9494f112175620dc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:25:50.127816 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae2c2131-ee07-4c13-9a5f-e2d501f05dae service neutron] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:25:50.128505 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae2c2131-ee07-4c13-9a5f-e2d501f05dae service neutron] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:25:50.128845 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae2c2131-ee07-4c13-9a5f-e2d501f05dae service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:25:50.130756 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-ae2c2131-ee07-4c13-9a5f-e2d501f05dae service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:25:50.145288 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae2c2131-ee07-4c13-9a5f-e2d501f05dae service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:25:50.147054 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae2c2131-ee07-4c13-9a5f-e2d501f05dae service neutron] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:25:50.160421 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 1111/2228] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Sep 13 03:25:50 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 58 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 03:25:50.229576 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff57e111-b67b-4e61-bc47-c548c9aa6157 service neutron] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:25:50.231152 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-ff57e111-b67b-4e61-bc47-c548c9aa6157 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:25:50.302690 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff57e111-b67b-4e61-bc47-c548c9aa6157 service neutron] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:25:50.307531 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8713fb6a-3182-4b4f-b3e8-a2930eca2fec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76c5181fc7be45279ac00017bb4640b3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:25:50.308100 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8713fb6a-3182-4b4f-b3e8-a2930eca2fec admin admin] REQUEST_METHOD: `DELETE` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:25:50.308396 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8713fb6a-3182-4b4f-b3e8-a2930eca2fec admin admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:25:50.308649 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8713fb6a-3182-4b4f-b3e8-a2930eca2fec admin admin] PATH_INFO: `/v3/users/466f990dcac84f4faec701b28616273e` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:25:50.309761 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8713fb6a-3182-4b4f-b3e8-a2930eca2fec admin admin] RBAC: Authorizing `identity:delete_user(user_id=466f990dcac84f4faec701b28616273e)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:25:50.311722 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8713fb6a-3182-4b4f-b3e8-a2930eca2fec admin admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:25:50.356046 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=94,keystone:DELETE=6 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:25:50.359156 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-8713fb6a-3182-4b4f-b3e8-a2930eca2fec admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '466f990dcac84f4faec701b28616273e'} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:25:50.366422 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-8713fb6a-3182-4b4f-b3e8-a2930eca2fec admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '466f990dcac84f4faec701b28616273e'} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:25:50.378456 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-8713fb6a-3182-4b4f-b3e8-a2930eca2fec admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '466f990dcac84f4faec701b28616273e'} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:25:50.379128 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-8713fb6a-3182-4b4f-b3e8-a2930eca2fec admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '466f990dcac84f4faec701b28616273e'} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:25:50.385986 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-8713fb6a-3182-4b4f-b3e8-a2930eca2fec admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '466f990dcac84f4faec701b28616273e'} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:25:50.391971 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 1118/2229] 10.222.0.18 () {58 vars in 1224 bytes} [Tue Sep 13 03:25:50 2022] DELETE /identity/v3/users/466f990dcac84f4faec701b28616273e => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 03:25:50.567223 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae2c2131-ee07-4c13-9a5f-e2d501f05dae service neutron] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:25:50.578150 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae2c2131-ee07-4c13-9a5f-e2d501f05dae service neutron] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:25:50.637900 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a464a84-2114-473e-bdf4-869242d94a36 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76c5181fc7be45279ac00017bb4640b3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:25:50.638526 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a464a84-2114-473e-bdf4-869242d94a36 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:25:50.638832 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a464a84-2114-473e-bdf4-869242d94a36 admin admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:25:50.639103 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a464a84-2114-473e-bdf4-869242d94a36 admin admin] PATH_INFO: `/v3/projects/b3ef3a6638114e1f9c838b00285f43f6` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:25:50.640199 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a464a84-2114-473e-bdf4-869242d94a36 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b3ef3a6638114e1f9c838b00285f43f6)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:25:50.642103 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a464a84-2114-473e-bdf4-869242d94a36 admin admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:25:50.686959 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-1a464a84-2114-473e-bdf4-869242d94a36 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b3ef3a6638114e1f9c838b00285f43f6', 'request_id': 'req-1a464a84-2114-473e-bdf4-869242d94a36'} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:25:50.692423 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-1a464a84-2114-473e-bdf4-869242d94a36 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b3ef3a6638114e1f9c838b00285f43f6', 'request_id': 'req-1a464a84-2114-473e-bdf4-869242d94a36'} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:25:50.698966 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-1a464a84-2114-473e-bdf4-869242d94a36 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b3ef3a6638114e1f9c838b00285f43f6', 'request_id': 'req-1a464a84-2114-473e-bdf4-869242d94a36'} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:25:50.712521 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-1a464a84-2114-473e-bdf4-869242d94a36 admin admin] The token cache is being invalidate because project b3ef3a6638114e1f9c838b00285f43f6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81400) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 03:25:50.712802 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-1a464a84-2114-473e-bdf4-869242d94a36 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:25:50.714044 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-1a464a84-2114-473e-bdf4-869242d94a36 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:25:50.715540 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 1112/2230] 10.222.0.18 () {58 vars in 1233 bytes} [Tue Sep 13 03:25:50 2022] DELETE /identity/v3/projects/b3ef3a6638114e1f9c838b00285f43f6 => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 03:25:50.721686 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cc18d0e-1370-4481-b8ec-2cdfe1b9f271 None None] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:25:50.722002 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cc18d0e-1370-4481-b8ec-2cdfe1b9f271 None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:25:50.722299 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cc18d0e-1370-4481-b8ec-2cdfe1b9f271 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:25:50.741494 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.auth.core [None req-7cc18d0e-1370-4481-b8ec-2cdfe1b9f271 None None] MFA Rules not processed for user `49f280ab086b4610855c4ffa378517d9`. Rule list: `[]` (Enabled: `True`). {{(pid=81399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 03:25:50.742720 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-7cc18d0e-1370-4481-b8ec-2cdfe1b9f271 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:25:50.800817 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 1119/2231] 10.222.0.18 () {58 vars in 958 bytes} [Tue Sep 13 03:25:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 03:25:50.809226 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e81561f-c89b-4060-91f4-d72c0ed0573e None None] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:25:50.809507 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e81561f-c89b-4060-91f4-d72c0ed0573e None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:25:50.809794 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e81561f-c89b-4060-91f4-d72c0ed0573e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:25:50.829702 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.auth.core [None req-8e81561f-c89b-4060-91f4-d72c0ed0573e None None] MFA Rules not processed for user `49f280ab086b4610855c4ffa378517d9`. Rule list: `[]` (Enabled: `True`). {{(pid=81400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 03:25:50.830846 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-8e81561f-c89b-4060-91f4-d72c0ed0573e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:25:50.839236 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 1113/2232] 10.222.0.18 () {58 vars in 958 bytes} [Tue Sep 13 03:25:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 03:25:50.843384 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cc18d0e-1370-4481-b8ec-2cdfe1b9f271 None None] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:25:50.852814 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cc18d0e-1370-4481-b8ec-2cdfe1b9f271 None None] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:25:50.857584 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4597613-69b4-4da1-930d-e8dd9bac20e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76c5181fc7be45279ac00017bb4640b3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:25:50.858218 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4597613-69b4-4da1-930d-e8dd9bac20e1 admin admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:25:50.858494 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4597613-69b4-4da1-930d-e8dd9bac20e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:25:50.858805 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4597613-69b4-4da1-930d-e8dd9bac20e1 admin admin] PATH_INFO: `/v3/domains` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:25:50.859483 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4597613-69b4-4da1-930d-e8dd9bac20e1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 03:25:50.859774 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4597613-69b4-4da1-930d-e8dd9bac20e1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:25:50.861361 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4597613-69b4-4da1-930d-e8dd9bac20e1 admin admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:25:50.872996 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 1120/2233] 10.222.0.18 () {58 vars in 1153 bytes} [Tue Sep 13 03:25:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 03:25:50.877171 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e81561f-c89b-4060-91f4-d72c0ed0573e None None] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:25:50.886983 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e81561f-c89b-4060-91f4-d72c0ed0573e None None] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:25:50.891608 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b56ad91d-4b4d-4cc6-b877-3f42f14e3a9f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76c5181fc7be45279ac00017bb4640b3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:25:50.892193 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b56ad91d-4b4d-4cc6-b877-3f42f14e3a9f admin admin] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:25:50.892467 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b56ad91d-4b4d-4cc6-b877-3f42f14e3a9f admin admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:25:50.892738 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b56ad91d-4b4d-4cc6-b877-3f42f14e3a9f admin admin] PATH_INFO: `/v3/projects` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:25:50.893367 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b56ad91d-4b4d-4cc6-b877-3f42f14e3a9f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:25:50.895306 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b56ad91d-4b4d-4cc6-b877-3f42f14e3a9f admin admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:25:50.922605 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 1114/2234] 10.222.0.18 () {60 vars in 1153 bytes} [Tue Sep 13 03:25:50 2022] POST /identity/v3/projects => generated 379 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 03:25:50.926936 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4597613-69b4-4da1-930d-e8dd9bac20e1 admin admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:25:50.937165 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4597613-69b4-4da1-930d-e8dd9bac20e1 admin admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:25:50.992967 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5ac9264-4887-4d6d-b86e-b0a672c142ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76c5181fc7be45279ac00017bb4640b3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:25:50.993569 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5ac9264-4887-4d6d-b86e-b0a672c142ca admin admin] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:25:50.993847 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5ac9264-4887-4d6d-b86e-b0a672c142ca admin admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:25:50.994141 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5ac9264-4887-4d6d-b86e-b0a672c142ca admin admin] PATH_INFO: `/v3/users` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:25:50.994833 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5ac9264-4887-4d6d-b86e-b0a672c142ca admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:25:50.996753 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5ac9264-4887-4d6d-b86e-b0a672c142ca admin admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:25:51.016695 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 1121/2235] 10.222.0.18 () {60 vars in 1144 bytes} [Tue Sep 13 03:25:50 2022] POST /identity/v3/users => generated 290 bytes in 90 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 03:25:51.020780 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b56ad91d-4b4d-4cc6-b877-3f42f14e3a9f admin admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:25:51.031412 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b56ad91d-4b4d-4cc6-b877-3f42f14e3a9f admin admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:25:51.035976 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b923740-9546-4bbd-835d-888c3dee5387 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76c5181fc7be45279ac00017bb4640b3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:25:51.036562 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b923740-9546-4bbd-835d-888c3dee5387 admin admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:25:51.036837 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b923740-9546-4bbd-835d-888c3dee5387 admin admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:25:51.037103 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b923740-9546-4bbd-835d-888c3dee5387 admin admin] PATH_INFO: `/v3/roles` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:25:51.037736 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b923740-9546-4bbd-835d-888c3dee5387 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:25:51.039390 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b923740-9546-4bbd-835d-888c3dee5387 admin admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:25:51.049918 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 1115/2236] 10.222.0.18 () {58 vars in 1122 bytes} [Tue Sep 13 03:25:51 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 03:25:51.053466 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5ac9264-4887-4d6d-b86e-b0a672c142ca admin admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:25:51.063528 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5ac9264-4887-4d6d-b86e-b0a672c142ca admin admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:25:51.068318 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed0b9b62-45c7-4bc1-8f76-34077eb1b97a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76c5181fc7be45279ac00017bb4640b3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:25:51.068943 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed0b9b62-45c7-4bc1-8f76-34077eb1b97a admin admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:25:51.069222 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed0b9b62-45c7-4bc1-8f76-34077eb1b97a admin admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:25:51.069487 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed0b9b62-45c7-4bc1-8f76-34077eb1b97a admin admin] PATH_INFO: `/v3/roles` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:25:51.070112 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed0b9b62-45c7-4bc1-8f76-34077eb1b97a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:25:51.071750 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed0b9b62-45c7-4bc1-8f76-34077eb1b97a admin admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:25:51.082379 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 1122/2237] 10.222.0.18 () {58 vars in 1122 bytes} [Tue Sep 13 03:25:51 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 13 03:25:51.086347 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b923740-9546-4bbd-835d-888c3dee5387 admin admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:25:51.096851 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b923740-9546-4bbd-835d-888c3dee5387 admin admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:25:51.101491 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38cbcdbe-7e1e-4df6-8198-0a3ac9f67569 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76c5181fc7be45279ac00017bb4640b3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:25:51.102126 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38cbcdbe-7e1e-4df6-8198-0a3ac9f67569 admin admin] REQUEST_METHOD: `PUT` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:25:51.102401 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38cbcdbe-7e1e-4df6-8198-0a3ac9f67569 admin admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:25:51.102706 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38cbcdbe-7e1e-4df6-8198-0a3ac9f67569 admin admin] PATH_INFO: `/v3/projects/ab6c92ba81254caaad368fd0ef382b0e/users/47fa1f7ce54145ceb3564b9ba7b4964f/roles/1eb7a5b5203945c29e6d22d653f47847` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:25:51.117572 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38cbcdbe-7e1e-4df6-8198-0a3ac9f67569 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ab6c92ba81254caaad368fd0ef382b0e, user_id=47fa1f7ce54145ceb3564b9ba7b4964f, role_id=1eb7a5b5203945c29e6d22d653f47847)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:25:51.120284 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38cbcdbe-7e1e-4df6-8198-0a3ac9f67569 admin admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:25:51.132383 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 1116/2238] 10.222.0.18 () {60 vars in 1483 bytes} [Tue Sep 13 03:25:51 2022] PUT /identity/v3/projects/ab6c92ba81254caaad368fd0ef382b0e/users/47fa1f7ce54145ceb3564b9ba7b4964f/roles/1eb7a5b5203945c29e6d22d653f47847 => generated 0 bytes in 46 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 03:25:51.137472 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ab347b-c3e8-4155-93df-1f1000704f3d None None] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:25:51.137760 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ab347b-c3e8-4155-93df-1f1000704f3d None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:25:51.138118 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ab347b-c3e8-4155-93df-1f1000704f3d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:25:51.156826 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.auth.core [None req-f7ab347b-c3e8-4155-93df-1f1000704f3d None None] MFA Rules not processed for user `49f280ab086b4610855c4ffa378517d9`. Rule list: `[]` (Enabled: `True`). {{(pid=81399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 03:25:51.158006 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-f7ab347b-c3e8-4155-93df-1f1000704f3d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:25:51.185138 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:DELETE=4,keystone:SELECT=92,keystone:INSERT=4 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:25:51.224812 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 1123/2239] 10.222.0.18 () {58 vars in 958 bytes} [Tue Sep 13 03:25:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 03:25:51.235123 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38cbcdbe-7e1e-4df6-8198-0a3ac9f67569 admin admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:25:51.236812 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-38cbcdbe-7e1e-4df6-8198-0a3ac9f67569 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:25:51.251025 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:25:51.295099 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38cbcdbe-7e1e-4df6-8198-0a3ac9f67569 admin admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:25:51.299093 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6c79465-de2a-488b-b97e-1c3950e53c10 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdffbc851086441ebd01022d1e4c2609', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b9d94f6a68f7470d9494f112175620dc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:25:51.299701 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c79465-de2a-488b-b97e-1c3950e53c10 service neutron] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:25:51.299974 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c79465-de2a-488b-b97e-1c3950e53c10 service neutron] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:25:51.300243 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c79465-de2a-488b-b97e-1c3950e53c10 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:25:51.309809 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6c79465-de2a-488b-b97e-1c3950e53c10 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:25:51.311304 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6c79465-de2a-488b-b97e-1c3950e53c10 service neutron] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:25:51.322109 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 1117/2240] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Sep 13 03:25:51 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 03:25:55.440084 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd9e8ea-f01f-41e4-9995-c3d5f22dfac1 None None] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:25:55.440639 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd9e8ea-f01f-41e4-9995-c3d5f22dfac1 None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:25:55.440639 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd9e8ea-f01f-41e4-9995-c3d5f22dfac1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:25:55.469853 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.auth.core [None req-1fd9e8ea-f01f-41e4-9995-c3d5f22dfac1 None None] MFA Rules not processed for user `47fa1f7ce54145ceb3564b9ba7b4964f`. Rule list: `[]` (Enabled: `True`). {{(pid=81399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 03:25:55.470974 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-1fd9e8ea-f01f-41e4-9995-c3d5f22dfac1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:25:55.565797 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 1124/2241] 10.222.0.18 () {58 vars in 958 bytes} [Tue Sep 13 03:25:55 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 03:25:55.570288 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6c79465-de2a-488b-b97e-1c3950e53c10 service neutron] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:25:55.580817 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6c79465-de2a-488b-b97e-1c3950e53c10 service neutron] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:25:55.585628 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78fab3dd-3373-470f-86cd-b79b593967d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76c5181fc7be45279ac00017bb4640b3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:25:55.586233 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78fab3dd-3373-470f-86cd-b79b593967d7 admin admin] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:25:55.586509 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78fab3dd-3373-470f-86cd-b79b593967d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:25:55.586846 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78fab3dd-3373-470f-86cd-b79b593967d7 admin admin] PATH_INFO: `/v3/projects` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:25:55.587491 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78fab3dd-3373-470f-86cd-b79b593967d7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:25:55.589965 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78fab3dd-3373-470f-86cd-b79b593967d7 admin admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:25:55.623462 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 1118/2242] 10.222.0.18 () {60 vars in 1153 bytes} [Tue Sep 13 03:25:55 2022] POST /identity/v3/projects => generated 379 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 13 03:25:55.627489 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fd9e8ea-f01f-41e4-9995-c3d5f22dfac1 None None] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:25:55.638386 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fd9e8ea-f01f-41e4-9995-c3d5f22dfac1 None None] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:25:55.699656 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bde50529-9bb9-4590-9a49-c2ca75fd742f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76c5181fc7be45279ac00017bb4640b3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:25:55.700275 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde50529-9bb9-4590-9a49-c2ca75fd742f admin admin] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:25:55.700551 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde50529-9bb9-4590-9a49-c2ca75fd742f admin admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:25:55.700997 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde50529-9bb9-4590-9a49-c2ca75fd742f admin admin] PATH_INFO: `/v3/users` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:25:55.701648 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bde50529-9bb9-4590-9a49-c2ca75fd742f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:25:55.703655 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bde50529-9bb9-4590-9a49-c2ca75fd742f admin admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:25:55.728285 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 1125/2243] 10.222.0.18 () {60 vars in 1144 bytes} [Tue Sep 13 03:25:55 2022] POST /identity/v3/users => generated 296 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 03:25:55.730055 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78fab3dd-3373-470f-86cd-b79b593967d7 admin admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:25:55.740729 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78fab3dd-3373-470f-86cd-b79b593967d7 admin admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:25:55.745691 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6ca9d2d-4d73-41fd-9ad7-54d435abf8d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76c5181fc7be45279ac00017bb4640b3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:25:55.746214 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ca9d2d-4d73-41fd-9ad7-54d435abf8d6 admin admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:25:55.746492 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ca9d2d-4d73-41fd-9ad7-54d435abf8d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:25:55.746807 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ca9d2d-4d73-41fd-9ad7-54d435abf8d6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:25:55.747448 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6ca9d2d-4d73-41fd-9ad7-54d435abf8d6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:25:55.749046 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6ca9d2d-4d73-41fd-9ad7-54d435abf8d6 admin admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:25:55.760234 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 1119/2244] 10.222.0.18 () {58 vars in 1122 bytes} [Tue Sep 13 03:25:55 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 03:25:55.764526 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bde50529-9bb9-4590-9a49-c2ca75fd742f admin admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:25:55.774781 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bde50529-9bb9-4590-9a49-c2ca75fd742f admin admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:25:55.780012 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e0a98ea-07bd-43d4-ac8d-d2739cf59717 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76c5181fc7be45279ac00017bb4640b3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:25:55.780669 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e0a98ea-07bd-43d4-ac8d-d2739cf59717 admin admin] REQUEST_METHOD: `PUT` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:25:55.780994 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e0a98ea-07bd-43d4-ac8d-d2739cf59717 admin admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:25:55.781268 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e0a98ea-07bd-43d4-ac8d-d2739cf59717 admin admin] PATH_INFO: `/v3/projects/279761f67a6c4a89b7cb9358ef5bfd06/users/fae11259c5894e94b8b1db60c00089c4/roles/425d4b481c7846e0bc07e7e805d5b15f` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:25:55.797879 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e0a98ea-07bd-43d4-ac8d-d2739cf59717 admin admin] RBAC: Authorizing `identity:create_grant(project_id=279761f67a6c4a89b7cb9358ef5bfd06, user_id=fae11259c5894e94b8b1db60c00089c4, role_id=425d4b481c7846e0bc07e7e805d5b15f)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:25:55.800597 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e0a98ea-07bd-43d4-ac8d-d2739cf59717 admin admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:25:55.815158 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 1126/2245] 10.222.0.18 () {60 vars in 1483 bytes} [Tue Sep 13 03:25:55 2022] PUT /identity/v3/projects/279761f67a6c4a89b7cb9358ef5bfd06/users/fae11259c5894e94b8b1db60c00089c4/roles/425d4b481c7846e0bc07e7e805d5b15f => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 13 03:25:55.819008 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6ca9d2d-4d73-41fd-9ad7-54d435abf8d6 admin admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:25:55.830072 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6ca9d2d-4d73-41fd-9ad7-54d435abf8d6 admin admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:25:55.889534 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b09c100e-862e-46e8-845e-135406fa7676 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76c5181fc7be45279ac00017bb4640b3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:25:55.890143 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b09c100e-862e-46e8-845e-135406fa7676 admin admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:25:55.890419 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b09c100e-862e-46e8-845e-135406fa7676 admin admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:25:55.890725 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b09c100e-862e-46e8-845e-135406fa7676 admin admin] PATH_INFO: `/v3/roles` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:25:55.891465 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b09c100e-862e-46e8-845e-135406fa7676 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:25:55.892960 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b09c100e-862e-46e8-845e-135406fa7676 admin admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:25:55.910715 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 1120/2246] 10.222.0.18 () {58 vars in 1122 bytes} [Tue Sep 13 03:25:55 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 13 03:25:55.911140 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e0a98ea-07bd-43d4-ac8d-d2739cf59717 admin admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:25:55.928057 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e0a98ea-07bd-43d4-ac8d-d2739cf59717 admin admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:25:55.934308 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c07b16e6-615d-4352-b93f-9f3d28a1958e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76c5181fc7be45279ac00017bb4640b3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:25:55.935229 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c07b16e6-615d-4352-b93f-9f3d28a1958e admin admin] REQUEST_METHOD: `PUT` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:25:55.937862 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c07b16e6-615d-4352-b93f-9f3d28a1958e admin admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:25:55.938140 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c07b16e6-615d-4352-b93f-9f3d28a1958e admin admin] PATH_INFO: `/v3/projects/279761f67a6c4a89b7cb9358ef5bfd06/users/fae11259c5894e94b8b1db60c00089c4/roles/425d4b481c7846e0bc07e7e805d5b15f` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:25:55.940109 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c07b16e6-615d-4352-b93f-9f3d28a1958e admin admin] RBAC: Authorizing `identity:create_grant(project_id=279761f67a6c4a89b7cb9358ef5bfd06, user_id=fae11259c5894e94b8b1db60c00089c4, role_id=425d4b481c7846e0bc07e7e805d5b15f)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:25:55.942791 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c07b16e6-615d-4352-b93f-9f3d28a1958e admin admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:25:55.953858 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 1127/2247] 10.222.0.18 () {60 vars in 1483 bytes} [Tue Sep 13 03:25:55 2022] PUT /identity/v3/projects/279761f67a6c4a89b7cb9358ef5bfd06/users/fae11259c5894e94b8b1db60c00089c4/roles/425d4b481c7846e0bc07e7e805d5b15f => generated 0 bytes in 44 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 13 03:25:58.574339 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b09c100e-862e-46e8-845e-135406fa7676 admin admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:25:58.578878 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-b09c100e-862e-46e8-845e-135406fa7676 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:25:58.698378 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b09c100e-862e-46e8-845e-135406fa7676 admin admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:25:58.713711 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:25:58.781226 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bd54046-7c55-48f3-a50b-b2c6d346ef3a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7377f5bee4a49959182e1bd34bc97a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b9d94f6a68f7470d9494f112175620dc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:25:58.781842 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bd54046-7c55-48f3-a50b-b2c6d346ef3a service nova] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:25:58.782120 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bd54046-7c55-48f3-a50b-b2c6d346ef3a service nova] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:25:58.782390 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bd54046-7c55-48f3-a50b-b2c6d346ef3a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:25:58.784547 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-4bd54046-7c55-48f3-a50b-b2c6d346ef3a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:25:58.890521 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bd54046-7c55-48f3-a50b-b2c6d346ef3a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:25:58.892381 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bd54046-7c55-48f3-a50b-b2c6d346ef3a service nova] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:25:58.913702 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 1121/2248] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Sep 13 03:25:58 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 340 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 03:26:00.590010 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c07b16e6-615d-4352-b93f-9f3d28a1958e admin admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:26:00.615110 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c07b16e6-615d-4352-b93f-9f3d28a1958e admin admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:26:00.629761 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b696d22-c4d1-472b-afd0-7fdb8203e602 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7377f5bee4a49959182e1bd34bc97a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b9d94f6a68f7470d9494f112175620dc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:26:00.630435 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b696d22-c4d1-472b-afd0-7fdb8203e602 service nova] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:26:00.631385 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b696d22-c4d1-472b-afd0-7fdb8203e602 service nova] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:26:00.631666 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b696d22-c4d1-472b-afd0-7fdb8203e602 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:26:00.636616 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-9b696d22-c4d1-472b-afd0-7fdb8203e602 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:26:00.654528 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:26:00.757838 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b696d22-c4d1-472b-afd0-7fdb8203e602 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:26:00.759647 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b696d22-c4d1-472b-afd0-7fdb8203e602 service nova] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:26:00.771300 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 1128/2249] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Sep 13 03:26:00 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 182 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 03:26:01.473526 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5aeb223-801e-4fe8-b758-1c4d7be6d97a None None] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:26:01.474130 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5aeb223-801e-4fe8-b758-1c4d7be6d97a None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:26:01.474369 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5aeb223-801e-4fe8-b758-1c4d7be6d97a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:26:01.502869 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.auth.core [None req-d5aeb223-801e-4fe8-b758-1c4d7be6d97a None None] MFA Rules not processed for user `fae11259c5894e94b8b1db60c00089c4`. Rule list: `[]` (Enabled: `True`). {{(pid=81400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 03:26:01.504002 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-d5aeb223-801e-4fe8-b758-1c4d7be6d97a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:26:01.582260 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 1122/2250] 10.222.0.18 () {58 vars in 958 bytes} [Tue Sep 13 03:26:01 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 03:26:01.595879 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b696d22-c4d1-472b-afd0-7fdb8203e602 service nova] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:26:01.597382 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-9b696d22-c4d1-472b-afd0-7fdb8203e602 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:26:01.613850 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b696d22-c4d1-472b-afd0-7fdb8203e602 service nova] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:26:01.619692 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae2f3e8d-77f4-4054-b37a-e6969ab99ec1 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7377f5bee4a49959182e1bd34bc97a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b9d94f6a68f7470d9494f112175620dc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:26:01.620642 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae2f3e8d-77f4-4054-b37a-e6969ab99ec1 service nova] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:26:01.620929 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae2f3e8d-77f4-4054-b37a-e6969ab99ec1 service nova] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:26:01.621203 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae2f3e8d-77f4-4054-b37a-e6969ab99ec1 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:26:01.681550 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae2f3e8d-77f4-4054-b37a-e6969ab99ec1 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:26:01.683241 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae2f3e8d-77f4-4054-b37a-e6969ab99ec1 service nova] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:26:01.692294 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 1129/2251] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Sep 13 03:26:01 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 03:26:06.260969 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5aeb223-801e-4fe8-b758-1c4d7be6d97a None None] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:26:06.262600 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-d5aeb223-801e-4fe8-b758-1c4d7be6d97a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:26:06.309653 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:26:06.340336 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5aeb223-801e-4fe8-b758-1c4d7be6d97a None None] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:26:06.345495 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a911023e-29d6-4619-9ffd-fad943679d46 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76c5181fc7be45279ac00017bb4640b3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:26:06.346142 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a911023e-29d6-4619-9ffd-fad943679d46 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:26:06.346414 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a911023e-29d6-4619-9ffd-fad943679d46 admin admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:26:06.346745 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a911023e-29d6-4619-9ffd-fad943679d46 admin admin] PATH_INFO: `/v3/users/1ee484b8e3f8481aaf33bd2caf095919` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:26:06.347885 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a911023e-29d6-4619-9ffd-fad943679d46 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1ee484b8e3f8481aaf33bd2caf095919)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:26:06.349805 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a911023e-29d6-4619-9ffd-fad943679d46 admin admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:26:06.396113 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-a911023e-29d6-4619-9ffd-fad943679d46 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1ee484b8e3f8481aaf33bd2caf095919'} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:26:06.403416 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-a911023e-29d6-4619-9ffd-fad943679d46 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1ee484b8e3f8481aaf33bd2caf095919'} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:26:06.404035 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-a911023e-29d6-4619-9ffd-fad943679d46 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1ee484b8e3f8481aaf33bd2caf095919'} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:26:06.417184 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-a911023e-29d6-4619-9ffd-fad943679d46 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1ee484b8e3f8481aaf33bd2caf095919'} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:26:06.426115 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-a911023e-29d6-4619-9ffd-fad943679d46 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1ee484b8e3f8481aaf33bd2caf095919'} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:26:06.432987 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 1123/2252] 10.222.0.18 () {58 vars in 1224 bytes} [Tue Sep 13 03:26:06 2022] DELETE /identity/v3/users/1ee484b8e3f8481aaf33bd2caf095919 => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 03:26:06.438765 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df9852d-a534-427a-bc2c-63c3ea0987c0 None None] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:26:06.438765 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df9852d-a534-427a-bc2c-63c3ea0987c0 None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:26:06.438765 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df9852d-a534-427a-bc2c-63c3ea0987c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:26:06.457119 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.auth.core [None req-6df9852d-a534-427a-bc2c-63c3ea0987c0 None None] MFA Rules not processed for user `49f280ab086b4610855c4ffa378517d9`. Rule list: `[]` (Enabled: `True`). {{(pid=81399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 03:26:06.458325 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-6df9852d-a534-427a-bc2c-63c3ea0987c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:26:06.523229 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 1130/2253] 10.222.0.18 () {58 vars in 958 bytes} [Tue Sep 13 03:26:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 03:26:06.533271 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a911023e-29d6-4619-9ffd-fad943679d46 admin admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:26:06.534730 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-a911023e-29d6-4619-9ffd-fad943679d46 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:26:06.594885 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a911023e-29d6-4619-9ffd-fad943679d46 admin admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:26:06.598420 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f65495d-acf3-49fd-bba7-2d304398d788 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdffbc851086441ebd01022d1e4c2609', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b9d94f6a68f7470d9494f112175620dc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:26:06.599057 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f65495d-acf3-49fd-bba7-2d304398d788 service neutron] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:26:06.599457 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f65495d-acf3-49fd-bba7-2d304398d788 service neutron] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:26:06.599735 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f65495d-acf3-49fd-bba7-2d304398d788 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:26:06.609710 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f65495d-acf3-49fd-bba7-2d304398d788 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:26:06.611114 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f65495d-acf3-49fd-bba7-2d304398d788 service neutron] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:26:06.621132 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 1124/2254] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Sep 13 03:26:06 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 03:26:06.899997 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6df9852d-a534-427a-bc2c-63c3ea0987c0 None None] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:26:06.901984 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-6df9852d-a534-427a-bc2c-63c3ea0987c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:26:06.914975 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6df9852d-a534-427a-bc2c-63c3ea0987c0 None None] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:26:06.920942 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e0e5368-a807-46d4-8389-66cab151f424 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76c5181fc7be45279ac00017bb4640b3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:26:06.921700 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e0e5368-a807-46d4-8389-66cab151f424 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:26:06.922074 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e0e5368-a807-46d4-8389-66cab151f424 admin admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:26:06.922976 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e0e5368-a807-46d4-8389-66cab151f424 admin admin] PATH_INFO: `/v3/projects/5bdfaf801af1413293a8a8e390b2e617` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:26:06.924484 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e0e5368-a807-46d4-8389-66cab151f424 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5bdfaf801af1413293a8a8e390b2e617)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:26:06.926888 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e0e5368-a807-46d4-8389-66cab151f424 admin admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:26:06.982389 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:26:06.988842 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-4e0e5368-a807-46d4-8389-66cab151f424 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5bdfaf801af1413293a8a8e390b2e617', 'request_id': 'req-4e0e5368-a807-46d4-8389-66cab151f424'} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:26:06.994296 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-4e0e5368-a807-46d4-8389-66cab151f424 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5bdfaf801af1413293a8a8e390b2e617', 'request_id': 'req-4e0e5368-a807-46d4-8389-66cab151f424'} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:26:07.000490 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-4e0e5368-a807-46d4-8389-66cab151f424 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5bdfaf801af1413293a8a8e390b2e617', 'request_id': 'req-4e0e5368-a807-46d4-8389-66cab151f424'} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:26:07.012705 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-4e0e5368-a807-46d4-8389-66cab151f424 admin admin] The token cache is being invalidate because project 5bdfaf801af1413293a8a8e390b2e617 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81399) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 03:26:07.012995 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-4e0e5368-a807-46d4-8389-66cab151f424 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:26:07.013608 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-4e0e5368-a807-46d4-8389-66cab151f424 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:26:07.015250 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 1131/2255] 10.222.0.18 () {58 vars in 1233 bytes} [Tue Sep 13 03:26:06 2022] DELETE /identity/v3/projects/5bdfaf801af1413293a8a8e390b2e617 => generated 0 bytes in 116 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 03:26:07.020259 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a5d6875-ce0b-4afc-bc1a-26de9fee0074 None None] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:26:07.020541 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a5d6875-ce0b-4afc-bc1a-26de9fee0074 None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:26:07.020813 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a5d6875-ce0b-4afc-bc1a-26de9fee0074 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:26:07.041454 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.auth.core [None req-8a5d6875-ce0b-4afc-bc1a-26de9fee0074 None None] MFA Rules not processed for user `49f280ab086b4610855c4ffa378517d9`. Rule list: `[]` (Enabled: `True`). {{(pid=81400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 03:26:07.042921 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-8a5d6875-ce0b-4afc-bc1a-26de9fee0074 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:26:07.116869 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 1125/2256] 10.222.0.18 () {58 vars in 958 bytes} [Tue Sep 13 03:26:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 03:26:07.125076 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8815cfc2-0456-4644-bd64-603deba6fac4 None None] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:26:07.125342 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8815cfc2-0456-4644-bd64-603deba6fac4 None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:26:07.125630 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8815cfc2-0456-4644-bd64-603deba6fac4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:26:07.148827 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.auth.core [None req-8815cfc2-0456-4644-bd64-603deba6fac4 None None] MFA Rules not processed for user `49f280ab086b4610855c4ffa378517d9`. Rule list: `[]` (Enabled: `True`). {{(pid=81399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 03:26:07.150053 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-8815cfc2-0456-4644-bd64-603deba6fac4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:26:07.159054 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 1132/2257] 10.222.0.18 () {58 vars in 958 bytes} [Tue Sep 13 03:26:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 03:26:07.163757 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a5d6875-ce0b-4afc-bc1a-26de9fee0074 None None] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:26:07.168947 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:26:07.175231 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a5d6875-ce0b-4afc-bc1a-26de9fee0074 None None] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:26:07.181598 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77f412ad-5c5a-48e4-a841-e33798d5adc6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76c5181fc7be45279ac00017bb4640b3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:26:07.182274 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f412ad-5c5a-48e4-a841-e33798d5adc6 admin admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:26:07.182949 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f412ad-5c5a-48e4-a841-e33798d5adc6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:26:07.182949 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f412ad-5c5a-48e4-a841-e33798d5adc6 admin admin] PATH_INFO: `/v3/domains` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:26:07.183483 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77f412ad-5c5a-48e4-a841-e33798d5adc6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 03:26:07.183780 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77f412ad-5c5a-48e4-a841-e33798d5adc6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:26:07.185379 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77f412ad-5c5a-48e4-a841-e33798d5adc6 admin admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:26:07.198332 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 1126/2258] 10.222.0.18 () {58 vars in 1153 bytes} [Tue Sep 13 03:26:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 03:26:07.202712 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8815cfc2-0456-4644-bd64-603deba6fac4 None None] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:26:07.212355 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8815cfc2-0456-4644-bd64-603deba6fac4 None None] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:26:07.222541 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9075d98-3738-4a13-bb72-48dd403592fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76c5181fc7be45279ac00017bb4640b3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:26:07.223373 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9075d98-3738-4a13-bb72-48dd403592fb admin admin] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:26:07.224601 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9075d98-3738-4a13-bb72-48dd403592fb admin admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:26:07.225054 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9075d98-3738-4a13-bb72-48dd403592fb admin admin] PATH_INFO: `/v3/projects` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:26:07.226497 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9075d98-3738-4a13-bb72-48dd403592fb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:26:07.229505 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9075d98-3738-4a13-bb72-48dd403592fb admin admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:26:07.258776 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 1133/2259] 10.222.0.18 () {60 vars in 1153 bytes} [Tue Sep 13 03:26:07 2022] POST /identity/v3/projects => generated 397 bytes in 57 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 03:26:07.263401 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77f412ad-5c5a-48e4-a841-e33798d5adc6 admin admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:26:07.274604 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77f412ad-5c5a-48e4-a841-e33798d5adc6 admin admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:26:07.349133 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c68a5891-b11f-428b-83e6-52fade97fc45 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76c5181fc7be45279ac00017bb4640b3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:26:07.349775 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c68a5891-b11f-428b-83e6-52fade97fc45 admin admin] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:26:07.350060 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c68a5891-b11f-428b-83e6-52fade97fc45 admin admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:26:07.350334 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c68a5891-b11f-428b-83e6-52fade97fc45 admin admin] PATH_INFO: `/v3/users` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:26:07.351002 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c68a5891-b11f-428b-83e6-52fade97fc45 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:26:07.353000 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c68a5891-b11f-428b-83e6-52fade97fc45 admin admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:26:07.379829 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 1127/2260] 10.222.0.18 () {60 vars in 1144 bytes} [Tue Sep 13 03:26:07 2022] POST /identity/v3/users => generated 299 bytes in 117 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 13 03:26:07.384006 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9075d98-3738-4a13-bb72-48dd403592fb admin admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:26:07.394136 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9075d98-3738-4a13-bb72-48dd403592fb admin admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:26:07.398986 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d740e26-89a6-4372-b3a7-8b9a92ca86a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76c5181fc7be45279ac00017bb4640b3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:26:07.399522 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d740e26-89a6-4372-b3a7-8b9a92ca86a1 admin admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:26:07.399796 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d740e26-89a6-4372-b3a7-8b9a92ca86a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:26:07.400565 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d740e26-89a6-4372-b3a7-8b9a92ca86a1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:26:07.401195 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d740e26-89a6-4372-b3a7-8b9a92ca86a1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:26:07.402840 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d740e26-89a6-4372-b3a7-8b9a92ca86a1 admin admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:26:07.413184 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 1134/2261] 10.222.0.18 () {58 vars in 1122 bytes} [Tue Sep 13 03:26:07 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 13 03:26:07.418384 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c68a5891-b11f-428b-83e6-52fade97fc45 admin admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:26:07.428392 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c68a5891-b11f-428b-83e6-52fade97fc45 admin admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:26:07.432990 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-496ac7ec-39c4-46fc-94f5-8dc8cb8de81c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76c5181fc7be45279ac00017bb4640b3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:26:07.433572 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-496ac7ec-39c4-46fc-94f5-8dc8cb8de81c admin admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:26:07.433844 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-496ac7ec-39c4-46fc-94f5-8dc8cb8de81c admin admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:26:07.434138 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-496ac7ec-39c4-46fc-94f5-8dc8cb8de81c admin admin] PATH_INFO: `/v3/roles` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:26:07.434800 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-496ac7ec-39c4-46fc-94f5-8dc8cb8de81c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:26:07.436434 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-496ac7ec-39c4-46fc-94f5-8dc8cb8de81c admin admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:26:07.447037 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 1128/2262] 10.222.0.18 () {58 vars in 1122 bytes} [Tue Sep 13 03:26:07 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 13 03:26:07.451269 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d740e26-89a6-4372-b3a7-8b9a92ca86a1 admin admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:26:07.461720 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d740e26-89a6-4372-b3a7-8b9a92ca86a1 admin admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:26:07.467158 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb957add-db84-4270-8730-be0173ba72a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76c5181fc7be45279ac00017bb4640b3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:26:07.467821 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb957add-db84-4270-8730-be0173ba72a3 admin admin] REQUEST_METHOD: `PUT` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:26:07.468106 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb957add-db84-4270-8730-be0173ba72a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:26:07.468451 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb957add-db84-4270-8730-be0173ba72a3 admin admin] PATH_INFO: `/v3/projects/990797b82f2941aba643e3fcc86eaa22/users/4985f488e88b44ae8abaa9a52ea045e8/roles/1eb7a5b5203945c29e6d22d653f47847` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:26:07.487112 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb957add-db84-4270-8730-be0173ba72a3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=990797b82f2941aba643e3fcc86eaa22, user_id=4985f488e88b44ae8abaa9a52ea045e8, role_id=1eb7a5b5203945c29e6d22d653f47847)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:26:07.489785 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb957add-db84-4270-8730-be0173ba72a3 admin admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:26:07.502267 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 1135/2263] 10.222.0.18 () {60 vars in 1483 bytes} [Tue Sep 13 03:26:07 2022] PUT /identity/v3/projects/990797b82f2941aba643e3fcc86eaa22/users/4985f488e88b44ae8abaa9a52ea045e8/roles/1eb7a5b5203945c29e6d22d653f47847 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 13 03:26:07.513902 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fc6adfc-a232-4622-8c3f-ea54eeeffa0c None None] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:26:07.514253 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fc6adfc-a232-4622-8c3f-ea54eeeffa0c None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:26:07.514585 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fc6adfc-a232-4622-8c3f-ea54eeeffa0c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:26:07.544145 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.auth.core [None req-1fc6adfc-a232-4622-8c3f-ea54eeeffa0c None None] MFA Rules not processed for user `4985f488e88b44ae8abaa9a52ea045e8`. Rule list: `[]` (Enabled: `True`). {{(pid=81400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 03:26:07.545339 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-1fc6adfc-a232-4622-8c3f-ea54eeeffa0c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:26:07.612422 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 1129/2264] 10.222.0.18 () {58 vars in 958 bytes} [Tue Sep 13 03:26:07 2022] POST /identity/v3/auth/tokens => generated 3274 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 13 03:26:07.623772 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb957add-db84-4270-8730-be0173ba72a3 admin admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:26:07.625464 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-fb957add-db84-4270-8730-be0173ba72a3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:26:07.688845 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb957add-db84-4270-8730-be0173ba72a3 admin admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:26:07.692523 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e614f83-df57-4781-bbfd-88eca8e955d6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '924ee3f608d34a18b0ddc4695f102a56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b9d94f6a68f7470d9494f112175620dc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:26:07.693129 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e614f83-df57-4781-bbfd-88eca8e955d6 service cinder] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:26:07.693410 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e614f83-df57-4781-bbfd-88eca8e955d6 service cinder] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:26:07.693681 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e614f83-df57-4781-bbfd-88eca8e955d6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:26:07.702682 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e614f83-df57-4781-bbfd-88eca8e955d6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:26:07.704152 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e614f83-df57-4781-bbfd-88eca8e955d6 service cinder] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:26:07.714002 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 1136/2265] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Sep 13 03:26:07 2022] GET /identity/v3/auth/tokens => generated 3274 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 03:26:11.058821 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fc6adfc-a232-4622-8c3f-ea54eeeffa0c None None] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:26:11.060392 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-1fc6adfc-a232-4622-8c3f-ea54eeeffa0c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:26:11.072962 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fc6adfc-a232-4622-8c3f-ea54eeeffa0c None None] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:26:11.076617 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8070f89-1153-4381-b073-3d623ddda523 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '924ee3f608d34a18b0ddc4695f102a56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b9d94f6a68f7470d9494f112175620dc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:26:11.077213 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8070f89-1153-4381-b073-3d623ddda523 service cinder] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:26:11.077488 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8070f89-1153-4381-b073-3d623ddda523 service cinder] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:26:11.077758 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8070f89-1153-4381-b073-3d623ddda523 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:26:11.080030 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-a8070f89-1153-4381-b073-3d623ddda523 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:26:11.134508 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:26:11.137454 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8070f89-1153-4381-b073-3d623ddda523 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:26:11.139492 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8070f89-1153-4381-b073-3d623ddda523 service cinder] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:26:11.151811 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 1130/2266] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Sep 13 03:26:11 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 03:26:12.578975 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e5833f-d03a-46da-be7a-d377cd218717 None None] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:26:12.579473 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e5833f-d03a-46da-be7a-d377cd218717 None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:26:12.579573 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e5833f-d03a-46da-be7a-d377cd218717 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:26:12.604784 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.auth.core [None req-e9e5833f-d03a-46da-be7a-d377cd218717 None None] MFA Rules not processed for user `4f76a7c6fe9347b0bea5f701bccb6150`. Rule list: `[]` (Enabled: `True`). {{(pid=81399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 03:26:12.606059 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-e9e5833f-d03a-46da-be7a-d377cd218717 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:26:12.613082 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 1137/2267] 10.222.0.18 () {58 vars in 958 bytes} [Tue Sep 13 03:26:12 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 03:26:12.918931 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8070f89-1153-4381-b073-3d623ddda523 service cinder] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:26:12.920325 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-a8070f89-1153-4381-b073-3d623ddda523 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:26:12.974168 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8070f89-1153-4381-b073-3d623ddda523 service cinder] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:26:12.977660 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6ffd270-b828-47e8-ac48-f06ac9c2f1cd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdffbc851086441ebd01022d1e4c2609', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b9d94f6a68f7470d9494f112175620dc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:26:12.978240 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ffd270-b828-47e8-ac48-f06ac9c2f1cd service neutron] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:26:12.978510 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ffd270-b828-47e8-ac48-f06ac9c2f1cd service neutron] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:26:12.978816 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ffd270-b828-47e8-ac48-f06ac9c2f1cd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:26:12.981138 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-d6ffd270-b828-47e8-ac48-f06ac9c2f1cd service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:26:12.990474 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6ffd270-b828-47e8-ac48-f06ac9c2f1cd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:26:12.991930 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6ffd270-b828-47e8-ac48-f06ac9c2f1cd service neutron] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:26:12.999671 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 1131/2268] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Sep 13 03:26:12 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 81 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 03:26:13.327675 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9e5833f-d03a-46da-be7a-d377cd218717 None None] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:26:13.329098 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-e9e5833f-d03a-46da-be7a-d377cd218717 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:26:13.395330 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=93,keystone:DELETE=4,keystone:INSERT=3 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:26:13.399643 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9e5833f-d03a-46da-be7a-d377cd218717 None None] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:26:13.406282 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceeba7a8-69f2-48ae-ba76-b9910ec671ee service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b99f1b2374d4d2f8db78306316180d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b9d94f6a68f7470d9494f112175620dc', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:26:13.407142 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceeba7a8-69f2-48ae-ba76-b9910ec671ee service placement] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:26:13.407458 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceeba7a8-69f2-48ae-ba76-b9910ec671ee service placement] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:26:13.407727 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceeba7a8-69f2-48ae-ba76-b9910ec671ee service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:26:13.409508 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-ceeba7a8-69f2-48ae-ba76-b9910ec671ee service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:26:13.421209 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceeba7a8-69f2-48ae-ba76-b9910ec671ee service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:26:13.422808 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceeba7a8-69f2-48ae-ba76-b9910ec671ee service placement] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:26:13.432283 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 1138/2269] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Sep 13 03:26:13 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 03:26:21.079570 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6ffd270-b828-47e8-ac48-f06ac9c2f1cd service neutron] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:26:21.152910 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6ffd270-b828-47e8-ac48-f06ac9c2f1cd service neutron] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:26:21.158758 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2988ed89-6b26-4f5f-9aba-702fa312cc69 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76c5181fc7be45279ac00017bb4640b3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:26:21.159396 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2988ed89-6b26-4f5f-9aba-702fa312cc69 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:26:21.159675 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2988ed89-6b26-4f5f-9aba-702fa312cc69 admin admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:26:21.159946 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2988ed89-6b26-4f5f-9aba-702fa312cc69 admin admin] PATH_INFO: `/v3/users/4985f488e88b44ae8abaa9a52ea045e8` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:26:21.161169 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2988ed89-6b26-4f5f-9aba-702fa312cc69 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4985f488e88b44ae8abaa9a52ea045e8)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:26:21.163277 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2988ed89-6b26-4f5f-9aba-702fa312cc69 admin admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:26:21.213796 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-2988ed89-6b26-4f5f-9aba-702fa312cc69 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4985f488e88b44ae8abaa9a52ea045e8'} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:26:21.222554 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-2988ed89-6b26-4f5f-9aba-702fa312cc69 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4985f488e88b44ae8abaa9a52ea045e8'} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:26:21.223424 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-2988ed89-6b26-4f5f-9aba-702fa312cc69 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4985f488e88b44ae8abaa9a52ea045e8'} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:26:21.237143 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-2988ed89-6b26-4f5f-9aba-702fa312cc69 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4985f488e88b44ae8abaa9a52ea045e8'} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:26:21.244800 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-2988ed89-6b26-4f5f-9aba-702fa312cc69 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4985f488e88b44ae8abaa9a52ea045e8'} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:26:21.251473 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 1132/2270] 10.222.0.18 () {58 vars in 1224 bytes} [Tue Sep 13 03:26:21 2022] DELETE /identity/v3/users/4985f488e88b44ae8abaa9a52ea045e8 => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 03:26:21.255889 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ca94e38-86cf-46fd-8323-b98e6900079a None None] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:26:21.256717 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ca94e38-86cf-46fd-8323-b98e6900079a None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:26:21.256996 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ca94e38-86cf-46fd-8323-b98e6900079a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:26:21.275950 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.auth.core [None req-0ca94e38-86cf-46fd-8323-b98e6900079a None None] MFA Rules not processed for user `49f280ab086b4610855c4ffa378517d9`. Rule list: `[]` (Enabled: `True`). {{(pid=81399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 03:26:21.277179 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-0ca94e38-86cf-46fd-8323-b98e6900079a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:26:21.337597 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 1139/2271] 10.222.0.18 () {58 vars in 958 bytes} [Tue Sep 13 03:26:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 03:26:21.351546 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2988ed89-6b26-4f5f-9aba-702fa312cc69 admin admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:26:21.351546 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-2988ed89-6b26-4f5f-9aba-702fa312cc69 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:26:21.380283 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:26:21.422327 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2988ed89-6b26-4f5f-9aba-702fa312cc69 admin admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:26:21.426317 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb0132b8-29a5-46ba-af65-b22c09d22c66 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdffbc851086441ebd01022d1e4c2609', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b9d94f6a68f7470d9494f112175620dc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:26:21.426952 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb0132b8-29a5-46ba-af65-b22c09d22c66 service neutron] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:26:21.427251 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb0132b8-29a5-46ba-af65-b22c09d22c66 service neutron] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:26:21.427490 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb0132b8-29a5-46ba-af65-b22c09d22c66 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:26:21.437595 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb0132b8-29a5-46ba-af65-b22c09d22c66 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:26:21.439110 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb0132b8-29a5-46ba-af65-b22c09d22c66 service neutron] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:26:21.449487 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 1133/2272] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Sep 13 03:26:21 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 03:26:21.756910 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ca94e38-86cf-46fd-8323-b98e6900079a None None] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:26:21.758451 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-0ca94e38-86cf-46fd-8323-b98e6900079a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:26:21.783988 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ca94e38-86cf-46fd-8323-b98e6900079a None None] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:26:21.790719 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-213171be-3ea2-4d4d-845d-6e53a9e6c1ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76c5181fc7be45279ac00017bb4640b3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:26:21.791360 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-213171be-3ea2-4d4d-845d-6e53a9e6c1ec admin admin] REQUEST_METHOD: `DELETE` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:26:21.791886 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-213171be-3ea2-4d4d-845d-6e53a9e6c1ec admin admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:26:21.792151 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-213171be-3ea2-4d4d-845d-6e53a9e6c1ec admin admin] PATH_INFO: `/v3/projects/990797b82f2941aba643e3fcc86eaa22` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:26:21.793291 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-213171be-3ea2-4d4d-845d-6e53a9e6c1ec admin admin] RBAC: Authorizing `identity:delete_project(project_id=990797b82f2941aba643e3fcc86eaa22)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:26:21.795263 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-213171be-3ea2-4d4d-845d-6e53a9e6c1ec admin admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:26:21.849441 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-213171be-3ea2-4d4d-845d-6e53a9e6c1ec admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '990797b82f2941aba643e3fcc86eaa22', 'request_id': 'req-213171be-3ea2-4d4d-845d-6e53a9e6c1ec'} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:26:21.855332 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-213171be-3ea2-4d4d-845d-6e53a9e6c1ec admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '990797b82f2941aba643e3fcc86eaa22', 'request_id': 'req-213171be-3ea2-4d4d-845d-6e53a9e6c1ec'} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:26:21.861998 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-213171be-3ea2-4d4d-845d-6e53a9e6c1ec admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '990797b82f2941aba643e3fcc86eaa22', 'request_id': 'req-213171be-3ea2-4d4d-845d-6e53a9e6c1ec'} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:26:21.875034 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-213171be-3ea2-4d4d-845d-6e53a9e6c1ec admin admin] The token cache is being invalidate because project 990797b82f2941aba643e3fcc86eaa22 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81399) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 03:26:21.875325 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-213171be-3ea2-4d4d-845d-6e53a9e6c1ec admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:26:21.875957 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-213171be-3ea2-4d4d-845d-6e53a9e6c1ec admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:26:21.877448 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 1140/2273] 10.222.0.18 () {58 vars in 1233 bytes} [Tue Sep 13 03:26:21 2022] DELETE /identity/v3/projects/990797b82f2941aba643e3fcc86eaa22 => generated 0 bytes in 121 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 03:26:31.444617 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=18 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:26:31.873368 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=58,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:26:33.393514 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a1738b1-9f0a-4d8e-ade7-fb24ccf029d4 None None] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:26:33.394013 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a1738b1-9f0a-4d8e-ade7-fb24ccf029d4 None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:26:33.394115 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a1738b1-9f0a-4d8e-ade7-fb24ccf029d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:26:33.413137 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.auth.core [None req-3a1738b1-9f0a-4d8e-ade7-fb24ccf029d4 None None] MFA Rules not processed for user `47fa1f7ce54145ceb3564b9ba7b4964f`. Rule list: `[]` (Enabled: `True`). {{(pid=81400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 03:26:33.414313 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-3a1738b1-9f0a-4d8e-ade7-fb24ccf029d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:26:33.468052 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 1134/2274] 10.222.0.18 () {58 vars in 958 bytes} [Tue Sep 13 03:26:33 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 76 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 03:26:35.660728 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-213171be-3ea2-4d4d-845d-6e53a9e6c1ec admin admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:26:35.662303 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-213171be-3ea2-4d4d-845d-6e53a9e6c1ec admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:26:35.757636 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-213171be-3ea2-4d4d-845d-6e53a9e6c1ec admin admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:26:35.764160 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28e11737-09d1-45ae-8340-6cf9580b76b5 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7377f5bee4a49959182e1bd34bc97a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b9d94f6a68f7470d9494f112175620dc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:26:35.764881 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e11737-09d1-45ae-8340-6cf9580b76b5 service nova] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:26:35.765183 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e11737-09d1-45ae-8340-6cf9580b76b5 service nova] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:26:35.765430 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e11737-09d1-45ae-8340-6cf9580b76b5 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:26:35.767309 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-28e11737-09d1-45ae-8340-6cf9580b76b5 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:26:35.780839 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28e11737-09d1-45ae-8340-6cf9580b76b5 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:26:35.782796 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28e11737-09d1-45ae-8340-6cf9580b76b5 service nova] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:26:35.802805 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 1141/2275] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Sep 13 03:26:35 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 143 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 03:26:35.944550 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a1738b1-9f0a-4d8e-ade7-fb24ccf029d4 None None] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:26:35.947307 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-3a1738b1-9f0a-4d8e-ade7-fb24ccf029d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:26:36.024948 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a1738b1-9f0a-4d8e-ade7-fb24ccf029d4 None None] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:26:36.031797 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6306b05c-e9ea-4588-b46b-02d36b0d5d91 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdffbc851086441ebd01022d1e4c2609', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b9d94f6a68f7470d9494f112175620dc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:26:36.032197 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6306b05c-e9ea-4588-b46b-02d36b0d5d91 service neutron] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:26:36.032571 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6306b05c-e9ea-4588-b46b-02d36b0d5d91 service neutron] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:26:36.032940 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6306b05c-e9ea-4588-b46b-02d36b0d5d91 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:26:36.035979 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-6306b05c-e9ea-4588-b46b-02d36b0d5d91 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:26:36.048039 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6306b05c-e9ea-4588-b46b-02d36b0d5d91 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:26:36.049686 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6306b05c-e9ea-4588-b46b-02d36b0d5d91 service neutron] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:26:36.070762 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 1135/2276] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Sep 13 03:26:35 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 126 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 03:26:37.854458 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28e11737-09d1-45ae-8340-6cf9580b76b5 service nova] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:26:37.857937 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-28e11737-09d1-45ae-8340-6cf9580b76b5 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:26:37.876489 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28e11737-09d1-45ae-8340-6cf9580b76b5 service nova] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:26:37.886818 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d888ef4f-6e8f-4974-ab5b-aef023706651 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7377f5bee4a49959182e1bd34bc97a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b9d94f6a68f7470d9494f112175620dc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:26:37.888314 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d888ef4f-6e8f-4974-ab5b-aef023706651 service nova] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:26:37.888773 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d888ef4f-6e8f-4974-ab5b-aef023706651 service nova] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:26:37.889203 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d888ef4f-6e8f-4974-ab5b-aef023706651 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:26:37.892557 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-d888ef4f-6e8f-4974-ab5b-aef023706651 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:26:37.906298 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d888ef4f-6e8f-4974-ab5b-aef023706651 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:26:37.908100 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d888ef4f-6e8f-4974-ab5b-aef023706651 service nova] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:26:37.927884 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 1142/2277] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Sep 13 03:26:37 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 74 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 03:26:38.905283 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88c2eccb-d652-48b8-b041-5c252542072e None None] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:26:38.905870 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88c2eccb-d652-48b8-b041-5c252542072e None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:26:38.905870 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88c2eccb-d652-48b8-b041-5c252542072e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:26:38.925832 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.auth.core [None req-88c2eccb-d652-48b8-b041-5c252542072e None None] MFA Rules not processed for user `4c2d20fb54a54ecc99b47f237be4917c`. Rule list: `[]` (Enabled: `True`). {{(pid=81400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 03:26:38.927135 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-88c2eccb-d652-48b8-b041-5c252542072e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:26:38.984270 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 1136/2278] 10.222.0.18 () {58 vars in 1009 bytes} [Tue Sep 13 03:26:38 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 03:26:39.025767 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d888ef4f-6e8f-4974-ab5b-aef023706651 service nova] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:26:39.027228 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-d888ef4f-6e8f-4974-ab5b-aef023706651 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:26:39.098590 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d888ef4f-6e8f-4974-ab5b-aef023706651 service nova] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:26:39.103887 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fd9ee35-4a16-4f91-96b4-e96e0cee771b service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '893b0c0d85964b7dbc9705c0a091682c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b9d94f6a68f7470d9494f112175620dc', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:26:39.104646 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd9ee35-4a16-4f91-96b4-e96e0cee771b service swift] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:26:39.104990 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd9ee35-4a16-4f91-96b4-e96e0cee771b service swift] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:26:39.105326 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd9ee35-4a16-4f91-96b4-e96e0cee771b service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:26:39.118596 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fd9ee35-4a16-4f91-96b4-e96e0cee771b service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:26:39.120219 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fd9ee35-4a16-4f91-96b4-e96e0cee771b service swift] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:26:39.129236 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 1143/2279] 10.222.0.18 () {60 vars in 1384 bytes} [Tue Sep 13 03:26:39 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 13 03:26:48.977496 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=76 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:26:49.125706 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=72 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:26:51.250379 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88c2eccb-d652-48b8-b041-5c252542072e None None] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:26:51.268121 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88c2eccb-d652-48b8-b041-5c252542072e None None] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:26:51.274179 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6cace54-180c-46c4-8406-5384aa6b5b17 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7377f5bee4a49959182e1bd34bc97a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b9d94f6a68f7470d9494f112175620dc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:26:51.274826 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6cace54-180c-46c4-8406-5384aa6b5b17 service nova] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:26:51.275152 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6cace54-180c-46c4-8406-5384aa6b5b17 service nova] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:26:51.275434 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6cace54-180c-46c4-8406-5384aa6b5b17 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:26:51.277323 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-b6cace54-180c-46c4-8406-5384aa6b5b17 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:26:51.291958 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6cace54-180c-46c4-8406-5384aa6b5b17 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:26:51.294109 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6cace54-180c-46c4-8406-5384aa6b5b17 service nova] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:26:51.317049 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 1137/2280] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Sep 13 03:26:51 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 67 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 03:26:54.267345 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a10e55f-0717-4a76-9290-5254b6673bbc None None] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:26:54.267922 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a10e55f-0717-4a76-9290-5254b6673bbc None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:26:54.267922 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a10e55f-0717-4a76-9290-5254b6673bbc None None] PATH_INFO: `/` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:26:54.269483 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 1144/2281] 10.222.0.18 () {54 vars in 921 bytes} [Tue Sep 13 03:26:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 03:26:54.275343 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe8a3c8-f0f6-45ce-9533-04c395a6078d None None] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:26:54.275797 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe8a3c8-f0f6-45ce-9533-04c395a6078d None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:26:54.275904 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe8a3c8-f0f6-45ce-9533-04c395a6078d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:26:54.298282 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.auth.core [None req-4fe8a3c8-f0f6-45ce-9533-04c395a6078d None None] MFA Rules not processed for user `c7377f5bee4a49959182e1bd34bc97a4`. Rule list: `[]` (Enabled: `True`). {{(pid=81400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 03:26:54.299433 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-4fe8a3c8-f0f6-45ce-9533-04c395a6078d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:26:54.311498 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 1138/2282] 10.222.0.18 () {58 vars in 1018 bytes} [Tue Sep 13 03:26:54 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 03:26:54.323867 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a10e55f-0717-4a76-9290-5254b6673bbc None None] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:26:54.326253 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-3a10e55f-0717-4a76-9290-5254b6673bbc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:26:54.385881 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a10e55f-0717-4a76-9290-5254b6673bbc None None] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:26:54.389611 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f0376ef-c139-44fb-a6a2-53fae8706b18 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b5fdc2ff46a14638b409a13e73ba5500', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b9d94f6a68f7470d9494f112175620dc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:26:54.390218 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f0376ef-c139-44fb-a6a2-53fae8706b18 service glance] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:26:54.390495 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f0376ef-c139-44fb-a6a2-53fae8706b18 service glance] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:26:54.390797 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f0376ef-c139-44fb-a6a2-53fae8706b18 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:26:54.401186 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f0376ef-c139-44fb-a6a2-53fae8706b18 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:26:54.402672 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f0376ef-c139-44fb-a6a2-53fae8706b18 service glance] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:26:54.413926 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 1145/2283] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Sep 13 03:26:54 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 03:26:54.482688 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b0d6f6-ba0d-4e69-a203-fbe606d79b80 None None] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:26:54.482973 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b0d6f6-ba0d-4e69-a203-fbe606d79b80 None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:26:54.483243 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b0d6f6-ba0d-4e69-a203-fbe606d79b80 None None] PATH_INFO: `/` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:26:54.484609 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 1139/2284] 10.222.0.18 () {54 vars in 921 bytes} [Tue Sep 13 03:26:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 03:26:54.489499 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c5ae465-14f7-4583-bf4c-6ad0d6588e2e None None] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:26:54.489763 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c5ae465-14f7-4583-bf4c-6ad0d6588e2e None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:26:54.490170 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c5ae465-14f7-4583-bf4c-6ad0d6588e2e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:26:54.509875 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.auth.core [None req-4c5ae465-14f7-4583-bf4c-6ad0d6588e2e None None] MFA Rules not processed for user `cdffbc851086441ebd01022d1e4c2609`. Rule list: `[]` (Enabled: `True`). {{(pid=81399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 03:26:54.511118 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-4c5ae465-14f7-4583-bf4c-6ad0d6588e2e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:26:54.517556 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 1146/2285] 10.222.0.18 () {58 vars in 1018 bytes} [Tue Sep 13 03:26:54 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 03:26:54.529188 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1b0d6f6-ba0d-4e69-a203-fbe606d79b80 None None] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:26:54.530911 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-d1b0d6f6-ba0d-4e69-a203-fbe606d79b80 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:26:54.544868 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1b0d6f6-ba0d-4e69-a203-fbe606d79b80 None None] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:26:54.548241 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00c55190-d0b5-4f64-b125-59368fec0cff service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdffbc851086441ebd01022d1e4c2609', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b9d94f6a68f7470d9494f112175620dc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:26:54.549142 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c55190-d0b5-4f64-b125-59368fec0cff service neutron] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:26:54.549142 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c55190-d0b5-4f64-b125-59368fec0cff service neutron] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:26:54.549348 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c55190-d0b5-4f64-b125-59368fec0cff service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:26:54.556971 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00c55190-d0b5-4f64-b125-59368fec0cff service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:26:54.558413 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00c55190-d0b5-4f64-b125-59368fec0cff service neutron] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:26:54.567459 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 1140/2286] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Sep 13 03:26:54 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 03:26:54.657151 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b49c87-5f36-49e2-aecb-0a490d469586 None None] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:26:54.657438 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b49c87-5f36-49e2-aecb-0a490d469586 None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:26:54.657706 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b49c87-5f36-49e2-aecb-0a490d469586 None None] PATH_INFO: `/` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:26:54.659109 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 1147/2287] 10.222.0.18 () {54 vars in 921 bytes} [Tue Sep 13 03:26:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 03:26:54.664006 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c72c15d8-3ad7-4de4-973d-3d091989a84b None None] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:26:54.664292 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c72c15d8-3ad7-4de4-973d-3d091989a84b None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:26:54.664566 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c72c15d8-3ad7-4de4-973d-3d091989a84b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:26:54.685988 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.auth.core [None req-c72c15d8-3ad7-4de4-973d-3d091989a84b None None] MFA Rules not processed for user `c7377f5bee4a49959182e1bd34bc97a4`. Rule list: `[]` (Enabled: `True`). {{(pid=81400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 03:26:54.687196 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-c72c15d8-3ad7-4de4-973d-3d091989a84b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:26:54.696423 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 1141/2288] 10.222.0.18 () {58 vars in 1018 bytes} [Tue Sep 13 03:26:54 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 03:26:54.706924 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91b49c87-5f36-49e2-aecb-0a490d469586 None None] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:26:54.718060 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91b49c87-5f36-49e2-aecb-0a490d469586 None None] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:26:54.721555 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6299fd5f-a6c9-4b28-a694-b8b732942800 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdffbc851086441ebd01022d1e4c2609', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b9d94f6a68f7470d9494f112175620dc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:26:54.722139 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6299fd5f-a6c9-4b28-a694-b8b732942800 service neutron] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:26:54.722410 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6299fd5f-a6c9-4b28-a694-b8b732942800 service neutron] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:26:54.722704 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6299fd5f-a6c9-4b28-a694-b8b732942800 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:26:54.732995 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6299fd5f-a6c9-4b28-a694-b8b732942800 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:26:54.734414 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6299fd5f-a6c9-4b28-a694-b8b732942800 service neutron] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:26:54.744769 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 1148/2289] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Sep 13 03:26:54 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 13 03:26:55.719560 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c72c15d8-3ad7-4de4-973d-3d091989a84b None None] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:26:55.728685 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c72c15d8-3ad7-4de4-973d-3d091989a84b None None] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:26:55.732703 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3cdf2e2-7748-4287-b135-a5b7d1c60816 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdffbc851086441ebd01022d1e4c2609', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b9d94f6a68f7470d9494f112175620dc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:26:55.733267 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3cdf2e2-7748-4287-b135-a5b7d1c60816 service neutron] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:26:55.733541 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3cdf2e2-7748-4287-b135-a5b7d1c60816 service neutron] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:26:55.733808 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3cdf2e2-7748-4287-b135-a5b7d1c60816 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:26:55.735512 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-b3cdf2e2-7748-4287-b135-a5b7d1c60816 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:26:55.798743 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3cdf2e2-7748-4287-b135-a5b7d1c60816 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:26:55.800189 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3cdf2e2-7748-4287-b135-a5b7d1c60816 service neutron] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:26:55.810188 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 1142/2290] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Sep 13 03:26:55 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 03:26:57.946020 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d4b1e62-dce5-4f52-91af-7e87cb267578 None None] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:26:57.946535 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d4b1e62-dce5-4f52-91af-7e87cb267578 None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:26:57.946670 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d4b1e62-dce5-4f52-91af-7e87cb267578 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:26:57.967518 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.auth.core [None req-9d4b1e62-dce5-4f52-91af-7e87cb267578 None None] MFA Rules not processed for user `4c2d20fb54a54ecc99b47f237be4917c`. Rule list: `[]` (Enabled: `True`). {{(pid=81399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 03:26:57.968913 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-9d4b1e62-dce5-4f52-91af-7e87cb267578 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:26:57.979514 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 1149/2291] 10.222.0.18 () {58 vars in 1009 bytes} [Tue Sep 13 03:26:57 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 03:26:57.996002 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3cdf2e2-7748-4287-b135-a5b7d1c60816 service neutron] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:26:58.007465 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3cdf2e2-7748-4287-b135-a5b7d1c60816 service neutron] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:26:58.018163 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a43d10ea-601f-436a-a529-89c21ce665a3 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '893b0c0d85964b7dbc9705c0a091682c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b9d94f6a68f7470d9494f112175620dc', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:26:58.020500 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a43d10ea-601f-436a-a529-89c21ce665a3 service swift] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:26:58.021040 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a43d10ea-601f-436a-a529-89c21ce665a3 service swift] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:26:58.021469 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a43d10ea-601f-436a-a529-89c21ce665a3 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:26:58.046166 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a43d10ea-601f-436a-a529-89c21ce665a3 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:26:58.051400 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a43d10ea-601f-436a-a529-89c21ce665a3 service swift] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:26:58.062484 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 1143/2292] 10.222.0.18 () {60 vars in 1384 bytes} [Tue Sep 13 03:26:57 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 67 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 13 03:27:07.494505 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d4b1e62-dce5-4f52-91af-7e87cb267578 None None] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:27:07.508912 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d4b1e62-dce5-4f52-91af-7e87cb267578 None None] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:27:07.515900 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5adceebd-8bbe-4659-a6a8-dcb28a33f5a6 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7377f5bee4a49959182e1bd34bc97a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b9d94f6a68f7470d9494f112175620dc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:27:07.522942 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5adceebd-8bbe-4659-a6a8-dcb28a33f5a6 service nova] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:27:07.524167 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5adceebd-8bbe-4659-a6a8-dcb28a33f5a6 service nova] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:27:07.524469 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5adceebd-8bbe-4659-a6a8-dcb28a33f5a6 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:27:07.528726 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-5adceebd-8bbe-4659-a6a8-dcb28a33f5a6 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:27:07.604834 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5adceebd-8bbe-4659-a6a8-dcb28a33f5a6 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:27:07.606995 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5adceebd-8bbe-4659-a6a8-dcb28a33f5a6 service nova] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:27:07.616026 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 1150/2293] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Sep 13 03:27:07 2022] GET /identity/v3/auth/tokens => generated 3270 bytes in 122 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 13 03:27:08.057662 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=54 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:27:17.611918 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=66 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:27:29.121310 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c96b4c-49cb-4980-a1a4-3b2a0fc9d146 None None] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:27:29.121758 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c96b4c-49cb-4980-a1a4-3b2a0fc9d146 None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:27:29.121854 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c96b4c-49cb-4980-a1a4-3b2a0fc9d146 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:27:29.141618 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.auth.core [None req-48c96b4c-49cb-4980-a1a4-3b2a0fc9d146 None None] MFA Rules not processed for user `4c2d20fb54a54ecc99b47f237be4917c`. Rule list: `[]` (Enabled: `True`). {{(pid=81400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 03:27:29.142861 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-48c96b4c-49cb-4980-a1a4-3b2a0fc9d146 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:27:29.149935 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 1144/2294] 10.222.0.18 () {58 vars in 967 bytes} [Tue Sep 13 03:27:29 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 03:27:29.162984 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5adceebd-8bbe-4659-a6a8-dcb28a33f5a6 service nova] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:27:29.174345 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5adceebd-8bbe-4659-a6a8-dcb28a33f5a6 service nova] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:27:29.179318 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa67b85e-92c4-46ba-b8de-c3765a2d745d service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '893b0c0d85964b7dbc9705c0a091682c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b9d94f6a68f7470d9494f112175620dc', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:27:29.179909 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa67b85e-92c4-46ba-b8de-c3765a2d745d service swift] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:27:29.180182 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa67b85e-92c4-46ba-b8de-c3765a2d745d service swift] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:27:29.180451 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa67b85e-92c4-46ba-b8de-c3765a2d745d service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:27:29.188137 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa67b85e-92c4-46ba-b8de-c3765a2d745d service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:27:29.189753 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa67b85e-92c4-46ba-b8de-c3765a2d745d service swift] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:27:29.197710 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 1151/2295] 10.222.0.18 () {60 vars in 1384 bytes} [Tue Sep 13 03:27:29 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Sep 13 03:27:39.145059 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=3 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:27:39.194348 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=6 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:27:50.214558 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a3c219-66aa-449d-801b-a01f0c0b7fd2 None None] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:27:50.215107 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a3c219-66aa-449d-801b-a01f0c0b7fd2 None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:27:50.215107 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a3c219-66aa-449d-801b-a01f0c0b7fd2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:27:50.233789 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.auth.core [None req-23a3c219-66aa-449d-801b-a01f0c0b7fd2 None None] MFA Rules not processed for user `b85258f01e1f436585eb6b9a4f3813b8`. Rule list: `[]` (Enabled: `True`). {{(pid=81400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 03:27:50.234991 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-23a3c219-66aa-449d-801b-a01f0c0b7fd2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:27:50.242653 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 1145/2296] 10.222.0.18 () {58 vars in 958 bytes} [Tue Sep 13 03:27:50 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 03:28:00.228582 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=3 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:28:23.864926 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa67b85e-92c4-46ba-b8de-c3765a2d745d service swift] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:28:23.878676 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa67b85e-92c4-46ba-b8de-c3765a2d745d service swift] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:28:23.884346 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe01e4bf-c54f-44aa-8b1e-1d03cc579277 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7377f5bee4a49959182e1bd34bc97a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b9d94f6a68f7470d9494f112175620dc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:28:23.885023 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe01e4bf-c54f-44aa-8b1e-1d03cc579277 service nova] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:28:23.885370 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe01e4bf-c54f-44aa-8b1e-1d03cc579277 service nova] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:28:23.885709 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe01e4bf-c54f-44aa-8b1e-1d03cc579277 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:28:23.887507 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-fe01e4bf-c54f-44aa-8b1e-1d03cc579277 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:28:23.949307 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe01e4bf-c54f-44aa-8b1e-1d03cc579277 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:28:23.951150 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe01e4bf-c54f-44aa-8b1e-1d03cc579277 service nova] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:28:23.960356 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 1152/2297] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Sep 13 03:28:23 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 03:28:30.184282 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e9058d-523c-4323-9ddc-49341e12dd22 None None] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:28:30.184778 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e9058d-523c-4323-9ddc-49341e12dd22 None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:28:30.184880 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e9058d-523c-4323-9ddc-49341e12dd22 None None] PATH_INFO: `/` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:28:30.186188 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 1146/2298] 10.222.0.18 () {54 vars in 921 bytes} [Tue Sep 13 03:28:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 13 03:28:30.193377 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee2ca55a-1077-441a-9377-4984502366d5 None None] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:28:30.193820 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee2ca55a-1077-441a-9377-4984502366d5 None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:28:30.194027 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee2ca55a-1077-441a-9377-4984502366d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:28:30.230309 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.auth.core [None req-ee2ca55a-1077-441a-9377-4984502366d5 None None] MFA Rules not processed for user `cdffbc851086441ebd01022d1e4c2609`. Rule list: `[]` (Enabled: `True`). {{(pid=81399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 03:28:30.231690 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-ee2ca55a-1077-441a-9377-4984502366d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:28:30.239734 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 1153/2299] 10.222.0.18 () {58 vars in 1018 bytes} [Tue Sep 13 03:28:30 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 48 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 03:28:30.252991 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81e9058d-523c-4323-9ddc-49341e12dd22 None None] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:28:30.271604 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81e9058d-523c-4323-9ddc-49341e12dd22 None None] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:28:30.277083 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f2511f8-4a97-45a6-8e28-135935a0ea43 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdffbc851086441ebd01022d1e4c2609', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b9d94f6a68f7470d9494f112175620dc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:28:30.279192 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f2511f8-4a97-45a6-8e28-135935a0ea43 service neutron] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:28:30.279647 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f2511f8-4a97-45a6-8e28-135935a0ea43 service neutron] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:28:30.281647 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f2511f8-4a97-45a6-8e28-135935a0ea43 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:28:30.304785 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f2511f8-4a97-45a6-8e28-135935a0ea43 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:28:30.306447 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f2511f8-4a97-45a6-8e28-135935a0ea43 service neutron] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:28:30.315643 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 1147/2300] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Sep 13 03:28:30 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 63 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 03:28:30.407186 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04e299a9-6160-444a-818f-a6b6bb7ba8a1 None None] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:28:30.407475 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04e299a9-6160-444a-818f-a6b6bb7ba8a1 None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:28:30.407749 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04e299a9-6160-444a-818f-a6b6bb7ba8a1 None None] PATH_INFO: `/` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:28:30.409119 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 1154/2301] 10.222.0.18 () {54 vars in 921 bytes} [Tue Sep 13 03:28:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 03:28:30.414301 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c38c85-9745-489c-b90e-2dfd5534d6cd None None] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:28:30.414584 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c38c85-9745-489c-b90e-2dfd5534d6cd None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:28:30.414888 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c38c85-9745-489c-b90e-2dfd5534d6cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:28:30.435092 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.auth.core [None req-93c38c85-9745-489c-b90e-2dfd5534d6cd None None] MFA Rules not processed for user `c7377f5bee4a49959182e1bd34bc97a4`. Rule list: `[]` (Enabled: `True`). {{(pid=81400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 03:28:30.436296 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-93c38c85-9745-489c-b90e-2dfd5534d6cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:28:30.446079 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 1148/2302] 10.222.0.18 () {58 vars in 1018 bytes} [Tue Sep 13 03:28:30 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 03:28:30.456779 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04e299a9-6160-444a-818f-a6b6bb7ba8a1 None None] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:28:30.468283 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04e299a9-6160-444a-818f-a6b6bb7ba8a1 None None] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:28:30.472471 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b6f37f2-d949-4caa-8066-bf7159898437 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdffbc851086441ebd01022d1e4c2609', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b9d94f6a68f7470d9494f112175620dc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:28:30.473220 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b6f37f2-d949-4caa-8066-bf7159898437 service neutron] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:28:30.473560 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b6f37f2-d949-4caa-8066-bf7159898437 service neutron] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:28:30.473895 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b6f37f2-d949-4caa-8066-bf7159898437 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:28:30.483969 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b6f37f2-d949-4caa-8066-bf7159898437 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:28:30.485476 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b6f37f2-d949-4caa-8066-bf7159898437 service neutron] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:28:30.496189 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 1155/2303] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Sep 13 03:28:30 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 03:28:33.262755 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2eb0215-8075-429e-966b-15ba72dd6032 None None] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:28:33.263268 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2eb0215-8075-429e-966b-15ba72dd6032 None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:28:33.263365 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2eb0215-8075-429e-966b-15ba72dd6032 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:28:33.281864 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.auth.core [None req-d2eb0215-8075-429e-966b-15ba72dd6032 None None] MFA Rules not processed for user `4f76a7c6fe9347b0bea5f701bccb6150`. Rule list: `[]` (Enabled: `True`). {{(pid=81400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 03:28:33.283092 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-d2eb0215-8075-429e-966b-15ba72dd6032 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:28:33.337183 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 1149/2304] 10.222.0.18 () {58 vars in 958 bytes} [Tue Sep 13 03:28:33 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 76 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 03:28:33.637193 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b6f37f2-d949-4caa-8066-bf7159898437 service neutron] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:28:33.645707 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b6f37f2-d949-4caa-8066-bf7159898437 service neutron] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:28:33.649355 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bd8a258-1f1b-4190-a93b-534a7d2e21fd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdffbc851086441ebd01022d1e4c2609', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b9d94f6a68f7470d9494f112175620dc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:28:33.649941 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd8a258-1f1b-4190-a93b-534a7d2e21fd service neutron] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:28:33.650217 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd8a258-1f1b-4190-a93b-534a7d2e21fd service neutron] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:28:33.650483 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd8a258-1f1b-4190-a93b-534a7d2e21fd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:28:33.652212 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-1bd8a258-1f1b-4190-a93b-534a7d2e21fd service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:28:33.662066 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bd8a258-1f1b-4190-a93b-534a7d2e21fd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:28:33.663577 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bd8a258-1f1b-4190-a93b-534a7d2e21fd service neutron] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:28:33.671691 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 1156/2305] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Sep 13 03:28:33 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 03:28:43.330579 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=34 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:28:43.668286 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=43 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:28:50.932120 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b100fa00-e38e-41e0-b4b5-edd81f0516f9 None None] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:28:50.932608 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b100fa00-e38e-41e0-b4b5-edd81f0516f9 None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:28:50.932722 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b100fa00-e38e-41e0-b4b5-edd81f0516f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:28:50.961201 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.auth.core [None req-b100fa00-e38e-41e0-b4b5-edd81f0516f9 None None] MFA Rules not processed for user `83bd9ab163ba4229bcfd810345430564`. Rule list: `[]` (Enabled: `True`). {{(pid=81400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 03:28:50.962368 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-b100fa00-e38e-41e0-b4b5-edd81f0516f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:28:51.035051 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 1150/2306] 10.222.0.18 () {58 vars in 958 bytes} [Tue Sep 13 03:28:50 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 03:29:01.033145 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=37 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:29:04.895830 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bd8a258-1f1b-4190-a93b-534a7d2e21fd service neutron] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:29:04.900740 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-1bd8a258-1f1b-4190-a93b-534a7d2e21fd service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:29:05.066788 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bd8a258-1f1b-4190-a93b-534a7d2e21fd service neutron] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:29:05.080929 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb58ab57-9141-4556-849d-85c1ea6436da service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b99f1b2374d4d2f8db78306316180d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b9d94f6a68f7470d9494f112175620dc', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:29:05.081564 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb58ab57-9141-4556-849d-85c1ea6436da service placement] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:29:05.082091 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb58ab57-9141-4556-849d-85c1ea6436da service placement] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:29:05.082374 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb58ab57-9141-4556-849d-85c1ea6436da service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:29:05.084280 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-fb58ab57-9141-4556-849d-85c1ea6436da service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:29:05.099431 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb58ab57-9141-4556-849d-85c1ea6436da service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:29:05.101109 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb58ab57-9141-4556-849d-85c1ea6436da service placement] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:29:05.111839 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 1157/2307] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Sep 13 03:29:04 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 217 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 03:29:15.108082 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=57 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:29:17.532778 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d934fa43-8acb-47e6-946e-bd64f83123bf None None] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:29:17.533293 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d934fa43-8acb-47e6-946e-bd64f83123bf None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:29:17.533395 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d934fa43-8acb-47e6-946e-bd64f83123bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:29:17.553132 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.auth.core [None req-d934fa43-8acb-47e6-946e-bd64f83123bf None None] MFA Rules not processed for user `80f6f9a952d84ccab0de028d58aa6914`. Rule list: `[]` (Enabled: `True`). {{(pid=81400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 03:29:17.554323 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-d934fa43-8acb-47e6-946e-bd64f83123bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:29:17.608529 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 1151/2308] 10.222.0.18 () {58 vars in 958 bytes} [Tue Sep 13 03:29:17 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 03:29:27.601717 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=25 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:29:28.406337 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb58ab57-9141-4556-849d-85c1ea6436da service placement] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:29:28.428269 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb58ab57-9141-4556-849d-85c1ea6436da service placement] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:29:28.433558 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a61218c1-b097-4383-af00-071acac9d1d5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdffbc851086441ebd01022d1e4c2609', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b9d94f6a68f7470d9494f112175620dc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:29:28.434432 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a61218c1-b097-4383-af00-071acac9d1d5 service neutron] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:29:28.434766 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a61218c1-b097-4383-af00-071acac9d1d5 service neutron] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:29:28.435314 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a61218c1-b097-4383-af00-071acac9d1d5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:29:28.437332 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-a61218c1-b097-4383-af00-071acac9d1d5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:29:28.530976 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a61218c1-b097-4383-af00-071acac9d1d5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:29:28.532551 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a61218c1-b097-4383-af00-071acac9d1d5 service neutron] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:29:28.552820 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 1158/2309] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Sep 13 03:29:28 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 147 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 13 03:29:32.241887 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d934fa43-8acb-47e6-946e-bd64f83123bf None None] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:29:32.253335 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d934fa43-8acb-47e6-946e-bd64f83123bf None None] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:29:32.258190 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-223a3c85-fcb9-4cb6-b003-3751e94ddb6d service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b99f1b2374d4d2f8db78306316180d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b9d94f6a68f7470d9494f112175620dc', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:29:32.258808 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-223a3c85-fcb9-4cb6-b003-3751e94ddb6d service placement] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:29:32.259086 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-223a3c85-fcb9-4cb6-b003-3751e94ddb6d service placement] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:29:32.259358 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-223a3c85-fcb9-4cb6-b003-3751e94ddb6d service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:29:32.261089 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-223a3c85-fcb9-4cb6-b003-3751e94ddb6d service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:29:32.273217 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-223a3c85-fcb9-4cb6-b003-3751e94ddb6d service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:29:32.274823 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-223a3c85-fcb9-4cb6-b003-3751e94ddb6d service placement] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:29:32.284844 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 1152/2310] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Sep 13 03:29:32 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 03:29:32.365339 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a61218c1-b097-4383-af00-071acac9d1d5 service neutron] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:29:32.366862 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-a61218c1-b097-4383-af00-071acac9d1d5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:29:32.378484 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a61218c1-b097-4383-af00-071acac9d1d5 service neutron] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:29:32.383071 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dc064ab-3bae-4a32-99ed-1cb1bb173647 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76c5181fc7be45279ac00017bb4640b3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:29:32.383698 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc064ab-3bae-4a32-99ed-1cb1bb173647 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:29:32.383999 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc064ab-3bae-4a32-99ed-1cb1bb173647 admin admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:29:32.384241 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc064ab-3bae-4a32-99ed-1cb1bb173647 admin admin] PATH_INFO: `/v3/users/b85258f01e1f436585eb6b9a4f3813b8` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:29:32.385839 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dc064ab-3bae-4a32-99ed-1cb1bb173647 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b85258f01e1f436585eb6b9a4f3813b8)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:29:32.387300 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dc064ab-3bae-4a32-99ed-1cb1bb173647 admin admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:29:32.434594 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-2dc064ab-3bae-4a32-99ed-1cb1bb173647 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b85258f01e1f436585eb6b9a4f3813b8'} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:29:32.442137 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-2dc064ab-3bae-4a32-99ed-1cb1bb173647 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b85258f01e1f436585eb6b9a4f3813b8'} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:29:32.454298 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-2dc064ab-3bae-4a32-99ed-1cb1bb173647 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b85258f01e1f436585eb6b9a4f3813b8'} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:29:32.455132 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-2dc064ab-3bae-4a32-99ed-1cb1bb173647 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b85258f01e1f436585eb6b9a4f3813b8'} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:29:32.462618 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-2dc064ab-3bae-4a32-99ed-1cb1bb173647 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b85258f01e1f436585eb6b9a4f3813b8'} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:29:32.468530 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 1159/2311] 10.222.0.18 () {58 vars in 1224 bytes} [Tue Sep 13 03:29:32 2022] DELETE /identity/v3/users/b85258f01e1f436585eb6b9a4f3813b8 => generated 0 bytes in 104 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 03:29:32.640132 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-223a3c85-fcb9-4cb6-b003-3751e94ddb6d service placement] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:29:32.651609 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-223a3c85-fcb9-4cb6-b003-3751e94ddb6d service placement] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:29:32.713919 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dc93a8b-a303-4424-939b-20b171035d18 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76c5181fc7be45279ac00017bb4640b3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:29:32.714554 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc93a8b-a303-4424-939b-20b171035d18 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:29:32.714849 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc93a8b-a303-4424-939b-20b171035d18 admin admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:29:32.715288 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc93a8b-a303-4424-939b-20b171035d18 admin admin] PATH_INFO: `/v3/projects/609f8dcac1034d09a04192f4238b2857` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:29:32.716588 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dc93a8b-a303-4424-939b-20b171035d18 admin admin] RBAC: Authorizing `identity:delete_project(project_id=609f8dcac1034d09a04192f4238b2857)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:29:32.718506 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dc93a8b-a303-4424-939b-20b171035d18 admin admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:29:32.767029 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-5dc93a8b-a303-4424-939b-20b171035d18 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '609f8dcac1034d09a04192f4238b2857', 'request_id': 'req-5dc93a8b-a303-4424-939b-20b171035d18'} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:29:32.773109 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-5dc93a8b-a303-4424-939b-20b171035d18 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '609f8dcac1034d09a04192f4238b2857', 'request_id': 'req-5dc93a8b-a303-4424-939b-20b171035d18'} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:29:32.779293 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-5dc93a8b-a303-4424-939b-20b171035d18 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '609f8dcac1034d09a04192f4238b2857', 'request_id': 'req-5dc93a8b-a303-4424-939b-20b171035d18'} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:29:32.791922 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-5dc93a8b-a303-4424-939b-20b171035d18 admin admin] The token cache is being invalidate because project 609f8dcac1034d09a04192f4238b2857 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81400) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 03:29:32.792210 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-5dc93a8b-a303-4424-939b-20b171035d18 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:29:32.792921 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-5dc93a8b-a303-4424-939b-20b171035d18 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:29:32.794517 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 1153/2312] 10.222.0.18 () {58 vars in 1233 bytes} [Tue Sep 13 03:29:32 2022] DELETE /identity/v3/projects/609f8dcac1034d09a04192f4238b2857 => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 03:29:32.804984 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77fe9c0a-c84f-4fdb-907c-0fbd023d2c2a None None] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:29:32.805270 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77fe9c0a-c84f-4fdb-907c-0fbd023d2c2a None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:29:32.805546 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77fe9c0a-c84f-4fdb-907c-0fbd023d2c2a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:29:32.849976 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:29:32.888391 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.auth.core [None req-77fe9c0a-c84f-4fdb-907c-0fbd023d2c2a None None] MFA Rules not processed for user `49f280ab086b4610855c4ffa378517d9`. Rule list: `[]` (Enabled: `True`). {{(pid=81399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 03:29:32.889712 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-77fe9c0a-c84f-4fdb-907c-0fbd023d2c2a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:29:32.959500 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 1160/2313] 10.222.0.18 () {58 vars in 958 bytes} [Tue Sep 13 03:29:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 03:29:32.967806 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-808dddb7-6fa1-4f04-8a1e-e5e6d21deaf1 None None] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:29:32.968089 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-808dddb7-6fa1-4f04-8a1e-e5e6d21deaf1 None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:29:32.968357 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-808dddb7-6fa1-4f04-8a1e-e5e6d21deaf1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:29:32.990593 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.auth.core [None req-808dddb7-6fa1-4f04-8a1e-e5e6d21deaf1 None None] MFA Rules not processed for user `49f280ab086b4610855c4ffa378517d9`. Rule list: `[]` (Enabled: `True`). {{(pid=81400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 03:29:32.991847 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-808dddb7-6fa1-4f04-8a1e-e5e6d21deaf1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:29:32.999957 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 1154/2314] 10.222.0.18 () {58 vars in 958 bytes} [Tue Sep 13 03:29:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 03:29:33.004270 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77fe9c0a-c84f-4fdb-907c-0fbd023d2c2a None None] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:29:33.014893 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77fe9c0a-c84f-4fdb-907c-0fbd023d2c2a None None] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:29:33.019854 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fdcc050-fb03-4e11-b082-73297616a329 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76c5181fc7be45279ac00017bb4640b3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:29:33.021580 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fdcc050-fb03-4e11-b082-73297616a329 admin admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:29:33.021864 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fdcc050-fb03-4e11-b082-73297616a329 admin admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:29:33.022134 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fdcc050-fb03-4e11-b082-73297616a329 admin admin] PATH_INFO: `/v3/domains` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:29:33.022820 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fdcc050-fb03-4e11-b082-73297616a329 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 03:29:33.023725 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fdcc050-fb03-4e11-b082-73297616a329 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:29:33.025374 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fdcc050-fb03-4e11-b082-73297616a329 admin admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:29:33.037527 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 1161/2315] 10.222.0.18 () {58 vars in 1153 bytes} [Tue Sep 13 03:29:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 03:29:33.042204 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-808dddb7-6fa1-4f04-8a1e-e5e6d21deaf1 None None] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:29:33.051632 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-808dddb7-6fa1-4f04-8a1e-e5e6d21deaf1 None None] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:29:33.057643 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-133602bc-efeb-4c6a-b6cd-82f0af661997 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76c5181fc7be45279ac00017bb4640b3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:29:33.058256 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-133602bc-efeb-4c6a-b6cd-82f0af661997 admin admin] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:29:33.058531 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-133602bc-efeb-4c6a-b6cd-82f0af661997 admin admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:29:33.058833 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-133602bc-efeb-4c6a-b6cd-82f0af661997 admin admin] PATH_INFO: `/v3/projects` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:29:33.059572 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-133602bc-efeb-4c6a-b6cd-82f0af661997 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:29:33.061498 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-133602bc-efeb-4c6a-b6cd-82f0af661997 admin admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:29:33.096633 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 1155/2316] 10.222.0.18 () {60 vars in 1153 bytes} [Tue Sep 13 03:29:33 2022] POST /identity/v3/projects => generated 403 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 03:29:33.098421 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fdcc050-fb03-4e11-b082-73297616a329 admin admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:29:33.108473 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fdcc050-fb03-4e11-b082-73297616a329 admin admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:29:33.181228 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c2e30e2-c36e-48b4-bb7b-7bdece566a88 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76c5181fc7be45279ac00017bb4640b3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:29:33.181946 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c2e30e2-c36e-48b4-bb7b-7bdece566a88 admin admin] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:29:33.182412 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c2e30e2-c36e-48b4-bb7b-7bdece566a88 admin admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:29:33.182596 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c2e30e2-c36e-48b4-bb7b-7bdece566a88 admin admin] PATH_INFO: `/v3/users` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:29:33.183330 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c2e30e2-c36e-48b4-bb7b-7bdece566a88 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:29:33.185540 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c2e30e2-c36e-48b4-bb7b-7bdece566a88 admin admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:29:33.217763 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 1162/2317] 10.222.0.18 () {60 vars in 1144 bytes} [Tue Sep 13 03:29:33 2022] POST /identity/v3/users => generated 302 bytes in 117 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 13 03:29:33.219019 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-133602bc-efeb-4c6a-b6cd-82f0af661997 admin admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:29:33.231346 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-133602bc-efeb-4c6a-b6cd-82f0af661997 admin admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:29:33.244505 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1615adaf-9a23-431f-b9c5-02eb5ba51eac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76c5181fc7be45279ac00017bb4640b3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:29:33.247759 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1615adaf-9a23-431f-b9c5-02eb5ba51eac admin admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:29:33.247759 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1615adaf-9a23-431f-b9c5-02eb5ba51eac admin admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:29:33.250459 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1615adaf-9a23-431f-b9c5-02eb5ba51eac admin admin] PATH_INFO: `/v3/roles` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:29:33.251284 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1615adaf-9a23-431f-b9c5-02eb5ba51eac admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:29:33.255762 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1615adaf-9a23-431f-b9c5-02eb5ba51eac admin admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:29:33.270650 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 1156/2318] 10.222.0.18 () {58 vars in 1122 bytes} [Tue Sep 13 03:29:33 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 03:29:33.272972 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c2e30e2-c36e-48b4-bb7b-7bdece566a88 admin admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:29:33.283496 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c2e30e2-c36e-48b4-bb7b-7bdece566a88 admin admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:29:33.288817 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-710b71a7-19b9-4244-ba52-9c6c7bfa8618 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76c5181fc7be45279ac00017bb4640b3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:29:33.289809 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-710b71a7-19b9-4244-ba52-9c6c7bfa8618 admin admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:29:33.290182 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-710b71a7-19b9-4244-ba52-9c6c7bfa8618 admin admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:29:33.290522 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-710b71a7-19b9-4244-ba52-9c6c7bfa8618 admin admin] PATH_INFO: `/v3/roles` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:29:33.291250 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-710b71a7-19b9-4244-ba52-9c6c7bfa8618 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:29:33.292903 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-710b71a7-19b9-4244-ba52-9c6c7bfa8618 admin admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:29:33.306929 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 1163/2319] 10.222.0.18 () {58 vars in 1122 bytes} [Tue Sep 13 03:29:33 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 03:29:33.311860 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1615adaf-9a23-431f-b9c5-02eb5ba51eac admin admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:29:33.326117 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1615adaf-9a23-431f-b9c5-02eb5ba51eac admin admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:29:33.331596 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3f884a2-b1d9-4f2d-8972-0e4618595288 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76c5181fc7be45279ac00017bb4640b3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:29:33.332254 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3f884a2-b1d9-4f2d-8972-0e4618595288 admin admin] REQUEST_METHOD: `PUT` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:29:33.332529 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3f884a2-b1d9-4f2d-8972-0e4618595288 admin admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:29:33.332797 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3f884a2-b1d9-4f2d-8972-0e4618595288 admin admin] PATH_INFO: `/v3/projects/85482abe8bd640008cfc4765a2eaf35b/users/77da95d05d8d41e98a040781cea7c813/roles/1eb7a5b5203945c29e6d22d653f47847` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:29:33.350801 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3f884a2-b1d9-4f2d-8972-0e4618595288 admin admin] RBAC: Authorizing `identity:create_grant(project_id=85482abe8bd640008cfc4765a2eaf35b, user_id=77da95d05d8d41e98a040781cea7c813, role_id=1eb7a5b5203945c29e6d22d653f47847)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:29:33.353381 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3f884a2-b1d9-4f2d-8972-0e4618595288 admin admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:29:33.370761 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 1157/2320] 10.222.0.18 () {60 vars in 1483 bytes} [Tue Sep 13 03:29:33 2022] PUT /identity/v3/projects/85482abe8bd640008cfc4765a2eaf35b/users/77da95d05d8d41e98a040781cea7c813/roles/1eb7a5b5203945c29e6d22d653f47847 => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 03:29:33.381253 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-334318dc-ad3e-473d-a327-4427cae3ab2c None None] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:29:33.381702 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-334318dc-ad3e-473d-a327-4427cae3ab2c None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:29:33.381896 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-334318dc-ad3e-473d-a327-4427cae3ab2c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:29:33.411095 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.auth.core [None req-334318dc-ad3e-473d-a327-4427cae3ab2c None None] MFA Rules not processed for user `77da95d05d8d41e98a040781cea7c813`. Rule list: `[]` (Enabled: `True`). {{(pid=81399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 03:29:33.412914 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-334318dc-ad3e-473d-a327-4427cae3ab2c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:29:33.475666 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:29:33.487446 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 1164/2321] 10.222.0.18 () {58 vars in 958 bytes} [Tue Sep 13 03:29:33 2022] POST /identity/v3/auth/tokens => generated 3280 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 03:29:33.492678 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3f884a2-b1d9-4f2d-8972-0e4618595288 admin admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:29:33.506357 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:29:33.562151 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3f884a2-b1d9-4f2d-8972-0e4618595288 admin admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:29:33.568063 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fc9f610-e41b-45a2-8f3b-e12c598d859b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76c5181fc7be45279ac00017bb4640b3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:29:33.568745 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc9f610-e41b-45a2-8f3b-e12c598d859b admin admin] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:29:33.569088 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc9f610-e41b-45a2-8f3b-e12c598d859b admin admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:29:33.569474 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc9f610-e41b-45a2-8f3b-e12c598d859b admin admin] PATH_INFO: `/v3/projects` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:29:33.570234 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fc9f610-e41b-45a2-8f3b-e12c598d859b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:29:33.574255 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fc9f610-e41b-45a2-8f3b-e12c598d859b admin admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:29:33.602645 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 1158/2322] 10.222.0.18 () {60 vars in 1153 bytes} [Tue Sep 13 03:29:33 2022] POST /identity/v3/projects => generated 403 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 03:29:33.607022 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-334318dc-ad3e-473d-a327-4427cae3ab2c None None] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:29:33.619296 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-334318dc-ad3e-473d-a327-4427cae3ab2c None None] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:29:33.688976 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e84996fb-eb46-4134-a56e-2aa94835530c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76c5181fc7be45279ac00017bb4640b3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:29:33.689602 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e84996fb-eb46-4134-a56e-2aa94835530c admin admin] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:29:33.689885 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e84996fb-eb46-4134-a56e-2aa94835530c admin admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:29:33.690165 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e84996fb-eb46-4134-a56e-2aa94835530c admin admin] PATH_INFO: `/v3/users` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:29:33.690869 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e84996fb-eb46-4134-a56e-2aa94835530c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:29:33.695983 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e84996fb-eb46-4134-a56e-2aa94835530c admin admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:29:33.721957 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 1165/2323] 10.222.0.18 () {60 vars in 1144 bytes} [Tue Sep 13 03:29:33 2022] POST /identity/v3/users => generated 308 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 13 03:29:33.723186 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fc9f610-e41b-45a2-8f3b-e12c598d859b admin admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:29:33.734426 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fc9f610-e41b-45a2-8f3b-e12c598d859b admin admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:29:33.739182 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af71f80e-cbb2-44f4-b9aa-c3d2c9551691 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76c5181fc7be45279ac00017bb4640b3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:29:33.739876 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af71f80e-cbb2-44f4-b9aa-c3d2c9551691 admin admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:29:33.740224 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af71f80e-cbb2-44f4-b9aa-c3d2c9551691 admin admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:29:33.740609 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af71f80e-cbb2-44f4-b9aa-c3d2c9551691 admin admin] PATH_INFO: `/v3/roles` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:29:33.741353 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af71f80e-cbb2-44f4-b9aa-c3d2c9551691 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:29:33.743116 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af71f80e-cbb2-44f4-b9aa-c3d2c9551691 admin admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:29:33.753837 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 1159/2324] 10.222.0.18 () {58 vars in 1122 bytes} [Tue Sep 13 03:29:33 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 03:29:33.758171 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e84996fb-eb46-4134-a56e-2aa94835530c admin admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:29:33.768735 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e84996fb-eb46-4134-a56e-2aa94835530c admin admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:29:33.774036 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-470251ed-6c3a-4afe-a1f8-57281cdb6ed4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76c5181fc7be45279ac00017bb4640b3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:29:33.774859 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-470251ed-6c3a-4afe-a1f8-57281cdb6ed4 admin admin] REQUEST_METHOD: `PUT` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:29:33.775596 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-470251ed-6c3a-4afe-a1f8-57281cdb6ed4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:29:33.775875 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-470251ed-6c3a-4afe-a1f8-57281cdb6ed4 admin admin] PATH_INFO: `/v3/projects/d7e8ab2fa80047369486e6cbee9fd496/users/c68c452d5889485cb09e71fe31dacac9/roles/425d4b481c7846e0bc07e7e805d5b15f` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:29:33.792891 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-470251ed-6c3a-4afe-a1f8-57281cdb6ed4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d7e8ab2fa80047369486e6cbee9fd496, user_id=c68c452d5889485cb09e71fe31dacac9, role_id=425d4b481c7846e0bc07e7e805d5b15f)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:29:33.795896 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-470251ed-6c3a-4afe-a1f8-57281cdb6ed4 admin admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:29:33.810880 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 1166/2325] 10.222.0.18 () {60 vars in 1483 bytes} [Tue Sep 13 03:29:33 2022] PUT /identity/v3/projects/d7e8ab2fa80047369486e6cbee9fd496/users/c68c452d5889485cb09e71fe31dacac9/roles/425d4b481c7846e0bc07e7e805d5b15f => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 13 03:29:33.814673 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af71f80e-cbb2-44f4-b9aa-c3d2c9551691 admin admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:29:33.825078 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af71f80e-cbb2-44f4-b9aa-c3d2c9551691 admin admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:29:33.887895 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14d824b3-4769-4578-ab60-eb8a13774ccb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76c5181fc7be45279ac00017bb4640b3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:29:33.888406 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d824b3-4769-4578-ab60-eb8a13774ccb admin admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:29:33.888682 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d824b3-4769-4578-ab60-eb8a13774ccb admin admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:29:33.888951 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d824b3-4769-4578-ab60-eb8a13774ccb admin admin] PATH_INFO: `/v3/roles` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:29:33.889870 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14d824b3-4769-4578-ab60-eb8a13774ccb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:29:33.891223 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14d824b3-4769-4578-ab60-eb8a13774ccb admin admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:29:33.901657 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 1160/2326] 10.222.0.18 () {58 vars in 1122 bytes} [Tue Sep 13 03:29:33 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 03:29:33.905847 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-470251ed-6c3a-4afe-a1f8-57281cdb6ed4 admin admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:29:33.917190 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-470251ed-6c3a-4afe-a1f8-57281cdb6ed4 admin admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:29:33.922114 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5232ada2-fed0-481e-94f7-9559986c114c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76c5181fc7be45279ac00017bb4640b3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:29:33.922810 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5232ada2-fed0-481e-94f7-9559986c114c admin admin] REQUEST_METHOD: `PUT` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:29:33.923092 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5232ada2-fed0-481e-94f7-9559986c114c admin admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:29:33.923390 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5232ada2-fed0-481e-94f7-9559986c114c admin admin] PATH_INFO: `/v3/projects/d7e8ab2fa80047369486e6cbee9fd496/users/c68c452d5889485cb09e71fe31dacac9/roles/425d4b481c7846e0bc07e7e805d5b15f` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:29:33.925287 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5232ada2-fed0-481e-94f7-9559986c114c admin admin] RBAC: Authorizing `identity:create_grant(project_id=d7e8ab2fa80047369486e6cbee9fd496, user_id=c68c452d5889485cb09e71fe31dacac9, role_id=425d4b481c7846e0bc07e7e805d5b15f)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:29:33.928038 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5232ada2-fed0-481e-94f7-9559986c114c admin admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:29:33.939424 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 1167/2327] 10.222.0.18 () {60 vars in 1483 bytes} [Tue Sep 13 03:29:33 2022] PUT /identity/v3/projects/d7e8ab2fa80047369486e6cbee9fd496/users/c68c452d5889485cb09e71fe31dacac9/roles/425d4b481c7846e0bc07e7e805d5b15f => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 03:29:33.949945 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2796841-bf40-458f-93b5-5ca56a5db3a2 None None] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:29:33.950234 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2796841-bf40-458f-93b5-5ca56a5db3a2 None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:29:33.950509 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2796841-bf40-458f-93b5-5ca56a5db3a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:29:33.987973 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.auth.core [None req-c2796841-bf40-458f-93b5-5ca56a5db3a2 None None] MFA Rules not processed for user `c68c452d5889485cb09e71fe31dacac9`. Rule list: `[]` (Enabled: `True`). {{(pid=81400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 03:29:33.989300 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-c2796841-bf40-458f-93b5-5ca56a5db3a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:29:34.058479 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 1161/2328] 10.222.0.18 () {58 vars in 958 bytes} [Tue Sep 13 03:29:33 2022] POST /identity/v3/auth/tokens => generated 3347 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 03:29:34.069853 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5232ada2-fed0-481e-94f7-9559986c114c admin admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:29:34.071586 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-5232ada2-fed0-481e-94f7-9559986c114c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:29:34.149108 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5232ada2-fed0-481e-94f7-9559986c114c admin admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:29:34.152738 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10d2b5b8-6e7f-47de-9da4-db4fa9fb44bf service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '924ee3f608d34a18b0ddc4695f102a56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b9d94f6a68f7470d9494f112175620dc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:29:34.153342 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d2b5b8-6e7f-47de-9da4-db4fa9fb44bf service cinder] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:29:34.153619 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d2b5b8-6e7f-47de-9da4-db4fa9fb44bf service cinder] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:29:34.153890 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d2b5b8-6e7f-47de-9da4-db4fa9fb44bf service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:29:34.163051 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10d2b5b8-6e7f-47de-9da4-db4fa9fb44bf service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:29:34.164726 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10d2b5b8-6e7f-47de-9da4-db4fa9fb44bf service cinder] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:29:34.174331 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 1168/2329] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Sep 13 03:29:34 2022] GET /identity/v3/auth/tokens => generated 3347 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 03:29:34.322783 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2796841-bf40-458f-93b5-5ca56a5db3a2 None None] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:29:34.332538 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:29:34.384808 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2796841-bf40-458f-93b5-5ca56a5db3a2 None None] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:29:34.389272 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7df764f-a793-496e-9a7c-bfcbb3a49cc6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76c5181fc7be45279ac00017bb4640b3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:29:34.389895 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7df764f-a793-496e-9a7c-bfcbb3a49cc6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:29:34.390171 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7df764f-a793-496e-9a7c-bfcbb3a49cc6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:29:34.390441 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7df764f-a793-496e-9a7c-bfcbb3a49cc6 admin admin] PATH_INFO: `/v3/users/77da95d05d8d41e98a040781cea7c813` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:29:34.391648 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7df764f-a793-496e-9a7c-bfcbb3a49cc6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=77da95d05d8d41e98a040781cea7c813)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:29:34.393609 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7df764f-a793-496e-9a7c-bfcbb3a49cc6 admin admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:29:34.438174 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-e7df764f-a793-496e-9a7c-bfcbb3a49cc6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '77da95d05d8d41e98a040781cea7c813'} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:29:34.445164 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-e7df764f-a793-496e-9a7c-bfcbb3a49cc6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '77da95d05d8d41e98a040781cea7c813'} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:29:34.445792 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-e7df764f-a793-496e-9a7c-bfcbb3a49cc6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '77da95d05d8d41e98a040781cea7c813'} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:29:34.457532 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-e7df764f-a793-496e-9a7c-bfcbb3a49cc6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '77da95d05d8d41e98a040781cea7c813'} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:29:34.464340 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-e7df764f-a793-496e-9a7c-bfcbb3a49cc6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '77da95d05d8d41e98a040781cea7c813'} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:29:34.470179 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 1162/2330] 10.222.0.18 () {58 vars in 1224 bytes} [Tue Sep 13 03:29:34 2022] DELETE /identity/v3/users/77da95d05d8d41e98a040781cea7c813 => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 03:29:34.475074 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b1340d-c71d-4bdb-9acb-d8b2f386884f None None] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:29:34.475554 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b1340d-c71d-4bdb-9acb-d8b2f386884f None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:29:34.475663 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b1340d-c71d-4bdb-9acb-d8b2f386884f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:29:34.494863 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.auth.core [None req-d9b1340d-c71d-4bdb-9acb-d8b2f386884f None None] MFA Rules not processed for user `49f280ab086b4610855c4ffa378517d9`. Rule list: `[]` (Enabled: `True`). {{(pid=81399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 03:29:34.496063 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-d9b1340d-c71d-4bdb-9acb-d8b2f386884f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:29:34.519758 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:29:34.555050 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 1169/2331] 10.222.0.18 () {58 vars in 958 bytes} [Tue Sep 13 03:29:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 03:29:34.564611 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7df764f-a793-496e-9a7c-bfcbb3a49cc6 admin admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:29:34.565971 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-e7df764f-a793-496e-9a7c-bfcbb3a49cc6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:29:34.633286 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7df764f-a793-496e-9a7c-bfcbb3a49cc6 admin admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:29:34.636821 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d739f6c0-2421-460f-82ce-7312df6d5d55 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdffbc851086441ebd01022d1e4c2609', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b9d94f6a68f7470d9494f112175620dc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:29:34.637403 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d739f6c0-2421-460f-82ce-7312df6d5d55 service neutron] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:29:34.637676 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d739f6c0-2421-460f-82ce-7312df6d5d55 service neutron] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:29:34.637942 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d739f6c0-2421-460f-82ce-7312df6d5d55 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:29:34.647075 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d739f6c0-2421-460f-82ce-7312df6d5d55 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:29:34.648484 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d739f6c0-2421-460f-82ce-7312df6d5d55 service neutron] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:29:34.657731 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 1163/2332] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Sep 13 03:29:34 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 03:29:34.926266 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9b1340d-c71d-4bdb-9acb-d8b2f386884f None None] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:29:34.928275 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-d9b1340d-c71d-4bdb-9acb-d8b2f386884f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:29:34.940089 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9b1340d-c71d-4bdb-9acb-d8b2f386884f None None] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:29:34.945227 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e4e93be-a3da-44d2-becb-66edba863a5d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76c5181fc7be45279ac00017bb4640b3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:29:34.945865 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e4e93be-a3da-44d2-becb-66edba863a5d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:29:34.946145 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e4e93be-a3da-44d2-becb-66edba863a5d admin admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:29:34.946417 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e4e93be-a3da-44d2-becb-66edba863a5d admin admin] PATH_INFO: `/v3/projects/85482abe8bd640008cfc4765a2eaf35b` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:29:34.947591 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e4e93be-a3da-44d2-becb-66edba863a5d admin admin] RBAC: Authorizing `identity:delete_project(project_id=85482abe8bd640008cfc4765a2eaf35b)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:29:34.949549 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e4e93be-a3da-44d2-becb-66edba863a5d admin admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:29:34.996059 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-8e4e93be-a3da-44d2-becb-66edba863a5d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '85482abe8bd640008cfc4765a2eaf35b', 'request_id': 'req-8e4e93be-a3da-44d2-becb-66edba863a5d'} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:29:35.001688 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-8e4e93be-a3da-44d2-becb-66edba863a5d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '85482abe8bd640008cfc4765a2eaf35b', 'request_id': 'req-8e4e93be-a3da-44d2-becb-66edba863a5d'} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:29:35.007858 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-8e4e93be-a3da-44d2-becb-66edba863a5d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '85482abe8bd640008cfc4765a2eaf35b', 'request_id': 'req-8e4e93be-a3da-44d2-becb-66edba863a5d'} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:29:35.019714 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-8e4e93be-a3da-44d2-becb-66edba863a5d admin admin] The token cache is being invalidate because project 85482abe8bd640008cfc4765a2eaf35b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81399) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 03:29:35.020007 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-8e4e93be-a3da-44d2-becb-66edba863a5d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:29:35.020623 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-8e4e93be-a3da-44d2-becb-66edba863a5d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:29:35.022148 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 1170/2333] 10.222.0.18 () {58 vars in 1233 bytes} [Tue Sep 13 03:29:34 2022] DELETE /identity/v3/projects/85482abe8bd640008cfc4765a2eaf35b => generated 0 bytes in 96 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 03:29:35.026206 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d739f6c0-2421-460f-82ce-7312df6d5d55 service neutron] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:29:35.036023 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d739f6c0-2421-460f-82ce-7312df6d5d55 service neutron] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:29:35.054120 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:29:35.095957 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ed95b5c-dc88-42a8-a963-db7f38e414ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76c5181fc7be45279ac00017bb4640b3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:29:35.096584 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ed95b5c-dc88-42a8-a963-db7f38e414ff admin admin] REQUEST_METHOD: `DELETE` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:29:35.096863 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ed95b5c-dc88-42a8-a963-db7f38e414ff admin admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:29:35.097136 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ed95b5c-dc88-42a8-a963-db7f38e414ff admin admin] PATH_INFO: `/v3/users/c68c452d5889485cb09e71fe31dacac9` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:29:35.098249 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ed95b5c-dc88-42a8-a963-db7f38e414ff admin admin] RBAC: Authorizing `identity:delete_user(user_id=c68c452d5889485cb09e71fe31dacac9)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:29:35.100752 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ed95b5c-dc88-42a8-a963-db7f38e414ff admin admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:29:35.144925 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-3ed95b5c-dc88-42a8-a963-db7f38e414ff admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c68c452d5889485cb09e71fe31dacac9'} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:29:35.151908 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-3ed95b5c-dc88-42a8-a963-db7f38e414ff admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c68c452d5889485cb09e71fe31dacac9'} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:29:35.152509 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-3ed95b5c-dc88-42a8-a963-db7f38e414ff admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c68c452d5889485cb09e71fe31dacac9'} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:29:35.165026 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-3ed95b5c-dc88-42a8-a963-db7f38e414ff admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c68c452d5889485cb09e71fe31dacac9'} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:29:35.171729 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-3ed95b5c-dc88-42a8-a963-db7f38e414ff admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c68c452d5889485cb09e71fe31dacac9'} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:29:35.178092 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 1164/2334] 10.222.0.18 () {58 vars in 1224 bytes} [Tue Sep 13 03:29:35 2022] DELETE /identity/v3/users/c68c452d5889485cb09e71fe31dacac9 => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 03:29:35.461922 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e4e93be-a3da-44d2-becb-66edba863a5d admin admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:29:35.463380 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-8e4e93be-a3da-44d2-becb-66edba863a5d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:29:35.489470 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e4e93be-a3da-44d2-becb-66edba863a5d admin admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:29:35.547211 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57ab7101-d545-4891-b8d0-eabeaa8daf47 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76c5181fc7be45279ac00017bb4640b3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:29:35.547964 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ab7101-d545-4891-b8d0-eabeaa8daf47 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:29:35.548118 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ab7101-d545-4891-b8d0-eabeaa8daf47 admin admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:29:35.548382 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ab7101-d545-4891-b8d0-eabeaa8daf47 admin admin] PATH_INFO: `/v3/projects/d7e8ab2fa80047369486e6cbee9fd496` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:29:35.549492 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57ab7101-d545-4891-b8d0-eabeaa8daf47 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d7e8ab2fa80047369486e6cbee9fd496)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:29:35.551420 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57ab7101-d545-4891-b8d0-eabeaa8daf47 admin admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:29:35.599161 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-57ab7101-d545-4891-b8d0-eabeaa8daf47 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd7e8ab2fa80047369486e6cbee9fd496', 'request_id': 'req-57ab7101-d545-4891-b8d0-eabeaa8daf47'} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:29:35.603461 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:29:35.605882 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-57ab7101-d545-4891-b8d0-eabeaa8daf47 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd7e8ab2fa80047369486e6cbee9fd496', 'request_id': 'req-57ab7101-d545-4891-b8d0-eabeaa8daf47'} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:29:35.613860 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-57ab7101-d545-4891-b8d0-eabeaa8daf47 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd7e8ab2fa80047369486e6cbee9fd496', 'request_id': 'req-57ab7101-d545-4891-b8d0-eabeaa8daf47'} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:29:35.626442 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-57ab7101-d545-4891-b8d0-eabeaa8daf47 admin admin] The token cache is being invalidate because project d7e8ab2fa80047369486e6cbee9fd496 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81399) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 03:29:35.626763 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-57ab7101-d545-4891-b8d0-eabeaa8daf47 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:29:35.627379 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-57ab7101-d545-4891-b8d0-eabeaa8daf47 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:29:35.628886 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 1171/2335] 10.222.0.18 () {58 vars in 1233 bytes} [Tue Sep 13 03:29:35 2022] DELETE /identity/v3/projects/d7e8ab2fa80047369486e6cbee9fd496 => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 03:29:35.635404 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ed5520-61d1-49bc-a5bb-5680d98c385a None None] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:29:35.635883 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ed5520-61d1-49bc-a5bb-5680d98c385a None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:29:35.636012 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ed5520-61d1-49bc-a5bb-5680d98c385a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:29:35.655973 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.auth.core [None req-62ed5520-61d1-49bc-a5bb-5680d98c385a None None] MFA Rules not processed for user `49f280ab086b4610855c4ffa378517d9`. Rule list: `[]` (Enabled: `True`). {{(pid=81400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 03:29:35.657155 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-62ed5520-61d1-49bc-a5bb-5680d98c385a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:29:35.714579 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 1165/2336] 10.222.0.18 () {58 vars in 958 bytes} [Tue Sep 13 03:29:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 03:29:35.723691 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c36148-a26d-44fe-a363-96464e8c7747 None None] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:29:35.723979 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c36148-a26d-44fe-a363-96464e8c7747 None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:29:35.724252 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c36148-a26d-44fe-a363-96464e8c7747 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:29:35.744524 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.auth.core [None req-82c36148-a26d-44fe-a363-96464e8c7747 None None] MFA Rules not processed for user `49f280ab086b4610855c4ffa378517d9`. Rule list: `[]` (Enabled: `True`). {{(pid=81399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 03:29:35.745713 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-82c36148-a26d-44fe-a363-96464e8c7747 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:29:35.753572 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 1172/2337] 10.222.0.18 () {58 vars in 958 bytes} [Tue Sep 13 03:29:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 03:29:35.758358 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62ed5520-61d1-49bc-a5bb-5680d98c385a None None] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:29:35.768003 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62ed5520-61d1-49bc-a5bb-5680d98c385a None None] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:29:35.773224 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dd8aba2-15eb-4df4-bb7e-ece4097c7a57 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76c5181fc7be45279ac00017bb4640b3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:29:35.773812 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd8aba2-15eb-4df4-bb7e-ece4097c7a57 admin admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:29:35.774118 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd8aba2-15eb-4df4-bb7e-ece4097c7a57 admin admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:29:35.774391 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd8aba2-15eb-4df4-bb7e-ece4097c7a57 admin admin] PATH_INFO: `/v3/domains` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:29:35.775063 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dd8aba2-15eb-4df4-bb7e-ece4097c7a57 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 03:29:35.775370 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dd8aba2-15eb-4df4-bb7e-ece4097c7a57 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:29:35.776956 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dd8aba2-15eb-4df4-bb7e-ece4097c7a57 admin admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:29:35.788209 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 1166/2338] 10.222.0.18 () {58 vars in 1153 bytes} [Tue Sep 13 03:29:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 03:29:35.792511 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82c36148-a26d-44fe-a363-96464e8c7747 None None] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:29:35.801536 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82c36148-a26d-44fe-a363-96464e8c7747 None None] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:29:35.806030 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fce1c90-8fe8-4443-93a5-86d564b5dd15 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76c5181fc7be45279ac00017bb4640b3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:29:35.806641 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fce1c90-8fe8-4443-93a5-86d564b5dd15 admin admin] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:29:35.806906 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fce1c90-8fe8-4443-93a5-86d564b5dd15 admin admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:29:35.807168 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fce1c90-8fe8-4443-93a5-86d564b5dd15 admin admin] PATH_INFO: `/v3/projects` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:29:35.807779 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fce1c90-8fe8-4443-93a5-86d564b5dd15 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:29:35.809646 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fce1c90-8fe8-4443-93a5-86d564b5dd15 admin admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:29:35.837012 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 1173/2339] 10.222.0.18 () {60 vars in 1153 bytes} [Tue Sep 13 03:29:35 2022] POST /identity/v3/projects => generated 389 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 03:29:35.841865 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dd8aba2-15eb-4df4-bb7e-ece4097c7a57 admin admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:29:35.852013 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dd8aba2-15eb-4df4-bb7e-ece4097c7a57 admin admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:29:35.875595 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:29:35.910227 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad7e29bb-c723-4648-9890-9d1720674bfd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76c5181fc7be45279ac00017bb4640b3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:29:35.910854 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad7e29bb-c723-4648-9890-9d1720674bfd admin admin] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:29:35.911176 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad7e29bb-c723-4648-9890-9d1720674bfd admin admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:29:35.911445 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad7e29bb-c723-4648-9890-9d1720674bfd admin admin] PATH_INFO: `/v3/users` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:29:35.912068 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad7e29bb-c723-4648-9890-9d1720674bfd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:29:35.913977 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad7e29bb-c723-4648-9890-9d1720674bfd admin admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:29:35.935870 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 1167/2340] 10.222.0.18 () {60 vars in 1144 bytes} [Tue Sep 13 03:29:35 2022] POST /identity/v3/users => generated 295 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 03:29:35.940058 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fce1c90-8fe8-4443-93a5-86d564b5dd15 admin admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:29:35.951091 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fce1c90-8fe8-4443-93a5-86d564b5dd15 admin admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:29:35.955722 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-246175ca-0dce-44d0-be6c-0195cbf4d89f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76c5181fc7be45279ac00017bb4640b3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:29:35.956311 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-246175ca-0dce-44d0-be6c-0195cbf4d89f admin admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:29:35.956587 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-246175ca-0dce-44d0-be6c-0195cbf4d89f admin admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:29:35.956886 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-246175ca-0dce-44d0-be6c-0195cbf4d89f admin admin] PATH_INFO: `/v3/roles` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:29:35.957517 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-246175ca-0dce-44d0-be6c-0195cbf4d89f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:29:35.959132 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-246175ca-0dce-44d0-be6c-0195cbf4d89f admin admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:29:35.969342 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 1174/2341] 10.222.0.18 () {58 vars in 1122 bytes} [Tue Sep 13 03:29:35 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 03:29:35.973305 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad7e29bb-c723-4648-9890-9d1720674bfd admin admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:29:35.983516 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad7e29bb-c723-4648-9890-9d1720674bfd admin admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:29:35.988140 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfba0a48-0bdc-41cf-820b-51fa9479122c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76c5181fc7be45279ac00017bb4640b3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:29:35.988731 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfba0a48-0bdc-41cf-820b-51fa9479122c admin admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:29:35.989008 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfba0a48-0bdc-41cf-820b-51fa9479122c admin admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:29:35.989781 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfba0a48-0bdc-41cf-820b-51fa9479122c admin admin] PATH_INFO: `/v3/roles` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:29:35.990417 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfba0a48-0bdc-41cf-820b-51fa9479122c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:29:35.992036 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfba0a48-0bdc-41cf-820b-51fa9479122c admin admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:29:36.002484 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 1168/2342] 10.222.0.18 () {58 vars in 1122 bytes} [Tue Sep 13 03:29:35 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 03:29:36.007188 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-246175ca-0dce-44d0-be6c-0195cbf4d89f admin admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:29:36.017129 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-246175ca-0dce-44d0-be6c-0195cbf4d89f admin admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:29:36.021790 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-671b95a8-5c0f-4902-b9b2-c30b6b1f5572 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76c5181fc7be45279ac00017bb4640b3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:29:36.022437 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-671b95a8-5c0f-4902-b9b2-c30b6b1f5572 admin admin] REQUEST_METHOD: `PUT` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:29:36.022743 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-671b95a8-5c0f-4902-b9b2-c30b6b1f5572 admin admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:29:36.023016 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-671b95a8-5c0f-4902-b9b2-c30b6b1f5572 admin admin] PATH_INFO: `/v3/projects/bc836c4a771a4582b191d4232d9c6c6d/users/e7f0dfa4ec6143d59b6d2dce0cb48ea9/roles/1eb7a5b5203945c29e6d22d653f47847` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:29:36.037146 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-671b95a8-5c0f-4902-b9b2-c30b6b1f5572 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bc836c4a771a4582b191d4232d9c6c6d, user_id=e7f0dfa4ec6143d59b6d2dce0cb48ea9, role_id=1eb7a5b5203945c29e6d22d653f47847)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:29:36.040096 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-671b95a8-5c0f-4902-b9b2-c30b6b1f5572 admin admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:29:36.053223 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 1175/2343] 10.222.0.18 () {60 vars in 1483 bytes} [Tue Sep 13 03:29:36 2022] PUT /identity/v3/projects/bc836c4a771a4582b191d4232d9c6c6d/users/e7f0dfa4ec6143d59b6d2dce0cb48ea9/roles/1eb7a5b5203945c29e6d22d653f47847 => generated 0 bytes in 46 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 03:29:36.063810 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3311fb76-f0c5-4e62-97a8-a921ad77f003 None None] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:29:36.064096 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3311fb76-f0c5-4e62-97a8-a921ad77f003 None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:29:36.064365 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3311fb76-f0c5-4e62-97a8-a921ad77f003 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:29:36.092186 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.auth.core [None req-3311fb76-f0c5-4e62-97a8-a921ad77f003 None None] MFA Rules not processed for user `e7f0dfa4ec6143d59b6d2dce0cb48ea9`. Rule list: `[]` (Enabled: `True`). {{(pid=81400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 03:29:36.093373 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-3311fb76-f0c5-4e62-97a8-a921ad77f003 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:29:36.155857 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 1169/2344] 10.222.0.18 () {58 vars in 958 bytes} [Tue Sep 13 03:29:36 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 03:29:36.160586 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-671b95a8-5c0f-4902-b9b2-c30b6b1f5572 admin admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:29:36.221169 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-671b95a8-5c0f-4902-b9b2-c30b6b1f5572 admin admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:29:36.225781 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3508661b-adbc-4084-a7b8-df32b8f2b81a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76c5181fc7be45279ac00017bb4640b3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:29:36.226368 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3508661b-adbc-4084-a7b8-df32b8f2b81a admin admin] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:29:36.226665 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3508661b-adbc-4084-a7b8-df32b8f2b81a admin admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:29:36.226936 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3508661b-adbc-4084-a7b8-df32b8f2b81a admin admin] PATH_INFO: `/v3/projects` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:29:36.227558 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3508661b-adbc-4084-a7b8-df32b8f2b81a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:29:36.229437 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3508661b-adbc-4084-a7b8-df32b8f2b81a admin admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:29:36.255714 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 1176/2345] 10.222.0.18 () {60 vars in 1153 bytes} [Tue Sep 13 03:29:36 2022] POST /identity/v3/projects => generated 387 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 13 03:29:36.259744 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3311fb76-f0c5-4e62-97a8-a921ad77f003 None None] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:29:36.270222 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3311fb76-f0c5-4e62-97a8-a921ad77f003 None None] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:29:36.327273 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-803834d9-d3a9-4c2d-8365-44dbe3c3130b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76c5181fc7be45279ac00017bb4640b3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:29:36.327800 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-803834d9-d3a9-4c2d-8365-44dbe3c3130b admin admin] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:29:36.328078 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-803834d9-d3a9-4c2d-8365-44dbe3c3130b admin admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:29:36.328349 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-803834d9-d3a9-4c2d-8365-44dbe3c3130b admin admin] PATH_INFO: `/v3/users` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:29:36.328981 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-803834d9-d3a9-4c2d-8365-44dbe3c3130b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:29:36.330930 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-803834d9-d3a9-4c2d-8365-44dbe3c3130b admin admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:29:36.351009 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 1170/2346] 10.222.0.18 () {60 vars in 1144 bytes} [Tue Sep 13 03:29:36 2022] POST /identity/v3/users => generated 300 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 03:29:36.355016 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3508661b-adbc-4084-a7b8-df32b8f2b81a admin admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:29:36.366042 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3508661b-adbc-4084-a7b8-df32b8f2b81a admin admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:29:36.370596 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86a18403-b06a-47f2-8cc0-359522738a11 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76c5181fc7be45279ac00017bb4640b3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:29:36.371295 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a18403-b06a-47f2-8cc0-359522738a11 admin admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:29:36.371577 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a18403-b06a-47f2-8cc0-359522738a11 admin admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:29:36.371848 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a18403-b06a-47f2-8cc0-359522738a11 admin admin] PATH_INFO: `/v3/roles` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:29:36.372483 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86a18403-b06a-47f2-8cc0-359522738a11 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:29:36.374073 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86a18403-b06a-47f2-8cc0-359522738a11 admin admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:29:36.384530 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 1177/2347] 10.222.0.18 () {58 vars in 1122 bytes} [Tue Sep 13 03:29:36 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 03:29:36.388451 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-803834d9-d3a9-4c2d-8365-44dbe3c3130b admin admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:29:36.398251 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-803834d9-d3a9-4c2d-8365-44dbe3c3130b admin admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:29:36.402827 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f730c8d-743f-4f1c-95c2-bfafb706b5e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76c5181fc7be45279ac00017bb4640b3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:29:36.403460 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f730c8d-743f-4f1c-95c2-bfafb706b5e3 admin admin] REQUEST_METHOD: `PUT` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:29:36.403732 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f730c8d-743f-4f1c-95c2-bfafb706b5e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:29:36.403998 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f730c8d-743f-4f1c-95c2-bfafb706b5e3 admin admin] PATH_INFO: `/v3/projects/1a5ede706c214d93b7fad27228963351/users/bc778e721208439d946a5d194db0a5c4/roles/425d4b481c7846e0bc07e7e805d5b15f` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:29:36.419071 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f730c8d-743f-4f1c-95c2-bfafb706b5e3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1a5ede706c214d93b7fad27228963351, user_id=bc778e721208439d946a5d194db0a5c4, role_id=425d4b481c7846e0bc07e7e805d5b15f)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:29:36.421719 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f730c8d-743f-4f1c-95c2-bfafb706b5e3 admin admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:29:36.430876 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:29:36.436039 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 1171/2348] 10.222.0.18 () {60 vars in 1483 bytes} [Tue Sep 13 03:29:36 2022] PUT /identity/v3/projects/1a5ede706c214d93b7fad27228963351/users/bc778e721208439d946a5d194db0a5c4/roles/425d4b481c7846e0bc07e7e805d5b15f => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 03:29:36.439896 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86a18403-b06a-47f2-8cc0-359522738a11 admin admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:29:36.450125 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86a18403-b06a-47f2-8cc0-359522738a11 admin admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:29:36.496987 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:DELETE=2,keystone:SELECT=94,keystone:INSERT=4 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:29:36.507271 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e82e42d-413a-4633-9bd6-58bbb5c77a26 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76c5181fc7be45279ac00017bb4640b3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:29:36.507861 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e82e42d-413a-4633-9bd6-58bbb5c77a26 admin admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:29:36.508137 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e82e42d-413a-4633-9bd6-58bbb5c77a26 admin admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:29:36.508402 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e82e42d-413a-4633-9bd6-58bbb5c77a26 admin admin] PATH_INFO: `/v3/roles` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:29:36.509032 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e82e42d-413a-4633-9bd6-58bbb5c77a26 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:29:36.510626 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e82e42d-413a-4633-9bd6-58bbb5c77a26 admin admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:29:36.520113 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 1178/2349] 10.222.0.18 () {58 vars in 1122 bytes} [Tue Sep 13 03:29:36 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 03:29:36.524309 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f730c8d-743f-4f1c-95c2-bfafb706b5e3 admin admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:29:36.534403 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f730c8d-743f-4f1c-95c2-bfafb706b5e3 admin admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:29:36.539797 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ddb6c9e-f2f7-450d-aef0-377540b95ca2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76c5181fc7be45279ac00017bb4640b3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:29:36.540436 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ddb6c9e-f2f7-450d-aef0-377540b95ca2 admin admin] REQUEST_METHOD: `PUT` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:29:36.540715 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ddb6c9e-f2f7-450d-aef0-377540b95ca2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:29:36.540984 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ddb6c9e-f2f7-450d-aef0-377540b95ca2 admin admin] PATH_INFO: `/v3/projects/1a5ede706c214d93b7fad27228963351/users/bc778e721208439d946a5d194db0a5c4/roles/425d4b481c7846e0bc07e7e805d5b15f` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:29:36.542929 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ddb6c9e-f2f7-450d-aef0-377540b95ca2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1a5ede706c214d93b7fad27228963351, user_id=bc778e721208439d946a5d194db0a5c4, role_id=425d4b481c7846e0bc07e7e805d5b15f)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:29:36.545587 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ddb6c9e-f2f7-450d-aef0-377540b95ca2 admin admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:29:36.556162 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 1172/2350] 10.222.0.18 () {60 vars in 1483 bytes} [Tue Sep 13 03:29:36 2022] PUT /identity/v3/projects/1a5ede706c214d93b7fad27228963351/users/bc778e721208439d946a5d194db0a5c4/roles/425d4b481c7846e0bc07e7e805d5b15f => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 03:29:36.567856 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb89ea55-28b7-4118-b760-29e6ddb2530f None None] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:29:36.568148 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb89ea55-28b7-4118-b760-29e6ddb2530f None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:29:36.568425 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb89ea55-28b7-4118-b760-29e6ddb2530f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:29:36.596623 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.auth.core [None req-eb89ea55-28b7-4118-b760-29e6ddb2530f None None] MFA Rules not processed for user `bc778e721208439d946a5d194db0a5c4`. Rule list: `[]` (Enabled: `True`). {{(pid=81399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 03:29:36.597853 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-eb89ea55-28b7-4118-b760-29e6ddb2530f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:29:36.668151 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 1179/2351] 10.222.0.18 () {58 vars in 958 bytes} [Tue Sep 13 03:29:36 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 03:29:36.678926 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ddb6c9e-f2f7-450d-aef0-377540b95ca2 admin admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:29:36.680734 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-2ddb6c9e-f2f7-450d-aef0-377540b95ca2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:29:36.741219 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ddb6c9e-f2f7-450d-aef0-377540b95ca2 admin admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:29:36.745767 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ebf57e2-1b20-48f7-bc03-cf4e163737ea service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '924ee3f608d34a18b0ddc4695f102a56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b9d94f6a68f7470d9494f112175620dc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:29:36.746374 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ebf57e2-1b20-48f7-bc03-cf4e163737ea service cinder] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:29:36.746678 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ebf57e2-1b20-48f7-bc03-cf4e163737ea service cinder] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:29:36.746950 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ebf57e2-1b20-48f7-bc03-cf4e163737ea service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:29:36.761979 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ebf57e2-1b20-48f7-bc03-cf4e163737ea service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:29:36.763462 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ebf57e2-1b20-48f7-bc03-cf4e163737ea service cinder] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:29:36.773525 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 1173/2352] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Sep 13 03:29:36 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 03:29:36.917453 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb89ea55-28b7-4118-b760-29e6ddb2530f None None] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:29:36.978498 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb89ea55-28b7-4118-b760-29e6ddb2530f None None] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:29:36.982992 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-280c5932-3d73-411b-a8ff-9b25f66f081c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76c5181fc7be45279ac00017bb4640b3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:29:36.983607 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-280c5932-3d73-411b-a8ff-9b25f66f081c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:29:36.983882 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-280c5932-3d73-411b-a8ff-9b25f66f081c admin admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:29:36.984152 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-280c5932-3d73-411b-a8ff-9b25f66f081c admin admin] PATH_INFO: `/v3/users/e7f0dfa4ec6143d59b6d2dce0cb48ea9` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:29:36.985257 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-280c5932-3d73-411b-a8ff-9b25f66f081c admin admin] RBAC: Authorizing `identity:delete_user(user_id=e7f0dfa4ec6143d59b6d2dce0cb48ea9)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:29:36.987198 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-280c5932-3d73-411b-a8ff-9b25f66f081c admin admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:29:37.032158 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-280c5932-3d73-411b-a8ff-9b25f66f081c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e7f0dfa4ec6143d59b6d2dce0cb48ea9'} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:29:37.040153 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-280c5932-3d73-411b-a8ff-9b25f66f081c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e7f0dfa4ec6143d59b6d2dce0cb48ea9'} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:29:37.052466 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-280c5932-3d73-411b-a8ff-9b25f66f081c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e7f0dfa4ec6143d59b6d2dce0cb48ea9'} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:29:37.053095 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-280c5932-3d73-411b-a8ff-9b25f66f081c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e7f0dfa4ec6143d59b6d2dce0cb48ea9'} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:29:37.057582 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:29:37.063177 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-280c5932-3d73-411b-a8ff-9b25f66f081c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e7f0dfa4ec6143d59b6d2dce0cb48ea9'} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:29:37.071993 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 1180/2353] 10.222.0.18 () {58 vars in 1224 bytes} [Tue Sep 13 03:29:36 2022] DELETE /identity/v3/users/e7f0dfa4ec6143d59b6d2dce0cb48ea9 => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 03:29:37.076292 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fcb3893-ffbe-4f8b-8155-e96f85820a90 None None] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:29:37.076574 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fcb3893-ffbe-4f8b-8155-e96f85820a90 None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:29:37.076845 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fcb3893-ffbe-4f8b-8155-e96f85820a90 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:29:37.097167 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.auth.core [None req-2fcb3893-ffbe-4f8b-8155-e96f85820a90 None None] MFA Rules not processed for user `49f280ab086b4610855c4ffa378517d9`. Rule list: `[]` (Enabled: `True`). {{(pid=81400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 03:29:37.098288 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-2fcb3893-ffbe-4f8b-8155-e96f85820a90 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:29:37.157663 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 1174/2354] 10.222.0.18 () {58 vars in 958 bytes} [Tue Sep 13 03:29:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 03:29:37.167644 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-280c5932-3d73-411b-a8ff-9b25f66f081c admin admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:29:37.169006 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-280c5932-3d73-411b-a8ff-9b25f66f081c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:29:37.227390 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-280c5932-3d73-411b-a8ff-9b25f66f081c admin admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:29:37.231020 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36391554-b260-4587-a82a-69622cfeaa69 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdffbc851086441ebd01022d1e4c2609', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b9d94f6a68f7470d9494f112175620dc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:29:37.231618 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36391554-b260-4587-a82a-69622cfeaa69 service neutron] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:29:37.231897 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36391554-b260-4587-a82a-69622cfeaa69 service neutron] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:29:37.232166 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36391554-b260-4587-a82a-69622cfeaa69 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:29:37.240884 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36391554-b260-4587-a82a-69622cfeaa69 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:29:37.242314 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36391554-b260-4587-a82a-69622cfeaa69 service neutron] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:29:37.251577 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 1181/2355] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Sep 13 03:29:37 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 03:29:37.523937 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fcb3893-ffbe-4f8b-8155-e96f85820a90 None None] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:29:37.525363 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-2fcb3893-ffbe-4f8b-8155-e96f85820a90 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:29:37.537249 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fcb3893-ffbe-4f8b-8155-e96f85820a90 None None] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:29:37.542159 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a722d254-0213-4f95-874a-ca91289cf814 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76c5181fc7be45279ac00017bb4640b3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:29:37.542818 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a722d254-0213-4f95-874a-ca91289cf814 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:29:37.543107 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a722d254-0213-4f95-874a-ca91289cf814 admin admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:29:37.543361 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a722d254-0213-4f95-874a-ca91289cf814 admin admin] PATH_INFO: `/v3/projects/bc836c4a771a4582b191d4232d9c6c6d` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:29:37.544489 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a722d254-0213-4f95-874a-ca91289cf814 admin admin] RBAC: Authorizing `identity:delete_project(project_id=bc836c4a771a4582b191d4232d9c6c6d)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:29:37.546473 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a722d254-0213-4f95-874a-ca91289cf814 admin admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:29:37.595235 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-a722d254-0213-4f95-874a-ca91289cf814 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bc836c4a771a4582b191d4232d9c6c6d', 'request_id': 'req-a722d254-0213-4f95-874a-ca91289cf814'} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:29:37.600754 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-a722d254-0213-4f95-874a-ca91289cf814 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bc836c4a771a4582b191d4232d9c6c6d', 'request_id': 'req-a722d254-0213-4f95-874a-ca91289cf814'} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:29:37.606945 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-a722d254-0213-4f95-874a-ca91289cf814 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bc836c4a771a4582b191d4232d9c6c6d', 'request_id': 'req-a722d254-0213-4f95-874a-ca91289cf814'} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:29:37.621423 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-a722d254-0213-4f95-874a-ca91289cf814 admin admin] The token cache is being invalidate because project bc836c4a771a4582b191d4232d9c6c6d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81400) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 03:29:37.621635 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-a722d254-0213-4f95-874a-ca91289cf814 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:29:37.622258 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-a722d254-0213-4f95-874a-ca91289cf814 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:29:37.624021 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 1175/2356] 10.222.0.18 () {58 vars in 1233 bytes} [Tue Sep 13 03:29:37 2022] DELETE /identity/v3/projects/bc836c4a771a4582b191d4232d9c6c6d => generated 0 bytes in 100 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 03:29:37.627871 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36391554-b260-4587-a82a-69622cfeaa69 service neutron] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:29:37.638559 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36391554-b260-4587-a82a-69622cfeaa69 service neutron] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:29:37.696291 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e389a34d-dabf-4e5e-bece-c8207c562d38 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76c5181fc7be45279ac00017bb4640b3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:29:37.696929 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e389a34d-dabf-4e5e-bece-c8207c562d38 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:29:37.697213 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e389a34d-dabf-4e5e-bece-c8207c562d38 admin admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:29:37.697490 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e389a34d-dabf-4e5e-bece-c8207c562d38 admin admin] PATH_INFO: `/v3/users/bc778e721208439d946a5d194db0a5c4` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:29:37.698632 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e389a34d-dabf-4e5e-bece-c8207c562d38 admin admin] RBAC: Authorizing `identity:delete_user(user_id=bc778e721208439d946a5d194db0a5c4)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:29:37.700595 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e389a34d-dabf-4e5e-bece-c8207c562d38 admin admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:29:37.749271 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-e389a34d-dabf-4e5e-bece-c8207c562d38 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bc778e721208439d946a5d194db0a5c4'} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:29:37.759363 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-e389a34d-dabf-4e5e-bece-c8207c562d38 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bc778e721208439d946a5d194db0a5c4'} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:29:37.773832 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-e389a34d-dabf-4e5e-bece-c8207c562d38 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bc778e721208439d946a5d194db0a5c4'} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:29:37.774914 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-e389a34d-dabf-4e5e-bece-c8207c562d38 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bc778e721208439d946a5d194db0a5c4'} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:29:37.785338 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-e389a34d-dabf-4e5e-bece-c8207c562d38 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bc778e721208439d946a5d194db0a5c4'} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:29:37.797683 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 1182/2357] 10.222.0.18 () {58 vars in 1224 bytes} [Tue Sep 13 03:29:37 2022] DELETE /identity/v3/users/bc778e721208439d946a5d194db0a5c4 => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 03:29:38.091919 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a722d254-0213-4f95-874a-ca91289cf814 admin admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:29:38.094061 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-a722d254-0213-4f95-874a-ca91289cf814 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:29:38.113631 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a722d254-0213-4f95-874a-ca91289cf814 admin admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:29:38.128910 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:29:38.188361 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f453fc7-85ca-4fa7-9df7-4e406bfb6fea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76c5181fc7be45279ac00017bb4640b3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:29:38.189005 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f453fc7-85ca-4fa7-9df7-4e406bfb6fea admin admin] REQUEST_METHOD: `DELETE` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:29:38.189284 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f453fc7-85ca-4fa7-9df7-4e406bfb6fea admin admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:29:38.189560 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f453fc7-85ca-4fa7-9df7-4e406bfb6fea admin admin] PATH_INFO: `/v3/projects/1a5ede706c214d93b7fad27228963351` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:29:38.190773 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f453fc7-85ca-4fa7-9df7-4e406bfb6fea admin admin] RBAC: Authorizing `identity:delete_project(project_id=1a5ede706c214d93b7fad27228963351)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:29:38.192698 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f453fc7-85ca-4fa7-9df7-4e406bfb6fea admin admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:29:38.240602 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-1f453fc7-85ca-4fa7-9df7-4e406bfb6fea admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1a5ede706c214d93b7fad27228963351', 'request_id': 'req-1f453fc7-85ca-4fa7-9df7-4e406bfb6fea'} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:29:38.246237 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-1f453fc7-85ca-4fa7-9df7-4e406bfb6fea admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1a5ede706c214d93b7fad27228963351', 'request_id': 'req-1f453fc7-85ca-4fa7-9df7-4e406bfb6fea'} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:29:38.252954 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-1f453fc7-85ca-4fa7-9df7-4e406bfb6fea admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1a5ede706c214d93b7fad27228963351', 'request_id': 'req-1f453fc7-85ca-4fa7-9df7-4e406bfb6fea'} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:29:38.266143 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-1f453fc7-85ca-4fa7-9df7-4e406bfb6fea admin admin] The token cache is being invalidate because project 1a5ede706c214d93b7fad27228963351 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81400) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 03:29:38.266436 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-1f453fc7-85ca-4fa7-9df7-4e406bfb6fea admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:29:38.267089 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-1f453fc7-85ca-4fa7-9df7-4e406bfb6fea admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:29:38.268807 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 1176/2358] 10.222.0.18 () {58 vars in 1233 bytes} [Tue Sep 13 03:29:38 2022] DELETE /identity/v3/projects/1a5ede706c214d93b7fad27228963351 => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 03:29:38.273858 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd4f9cb9-1803-4220-b8ff-d1d5b04c66e9 None None] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:29:38.274141 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd4f9cb9-1803-4220-b8ff-d1d5b04c66e9 None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:29:38.274411 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd4f9cb9-1803-4220-b8ff-d1d5b04c66e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:29:38.302481 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.auth.core [None req-dd4f9cb9-1803-4220-b8ff-d1d5b04c66e9 None None] MFA Rules not processed for user `49f280ab086b4610855c4ffa378517d9`. Rule list: `[]` (Enabled: `True`). {{(pid=81399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 03:29:38.303759 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-dd4f9cb9-1803-4220-b8ff-d1d5b04c66e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:29:38.315400 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:29:38.381868 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 1183/2359] 10.222.0.18 () {58 vars in 958 bytes} [Tue Sep 13 03:29:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 13 03:29:38.391091 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e15f9d8b-115a-4273-ac5c-5469a8d07ba0 None None] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:29:38.391369 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e15f9d8b-115a-4273-ac5c-5469a8d07ba0 None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:29:38.391767 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e15f9d8b-115a-4273-ac5c-5469a8d07ba0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:29:38.416202 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.auth.core [None req-e15f9d8b-115a-4273-ac5c-5469a8d07ba0 None None] MFA Rules not processed for user `49f280ab086b4610855c4ffa378517d9`. Rule list: `[]` (Enabled: `True`). {{(pid=81400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 03:29:38.417709 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-e15f9d8b-115a-4273-ac5c-5469a8d07ba0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:29:38.427171 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 1177/2360] 10.222.0.18 () {58 vars in 958 bytes} [Tue Sep 13 03:29:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 03:29:38.433575 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd4f9cb9-1803-4220-b8ff-d1d5b04c66e9 None None] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:29:38.448859 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd4f9cb9-1803-4220-b8ff-d1d5b04c66e9 None None] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:29:38.456569 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84061d00-fc04-41f9-8066-3a3902446640 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76c5181fc7be45279ac00017bb4640b3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:29:38.457366 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84061d00-fc04-41f9-8066-3a3902446640 admin admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:29:38.458449 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84061d00-fc04-41f9-8066-3a3902446640 admin admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:29:38.458823 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84061d00-fc04-41f9-8066-3a3902446640 admin admin] PATH_INFO: `/v3/domains` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:29:38.459758 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84061d00-fc04-41f9-8066-3a3902446640 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 03:29:38.460151 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84061d00-fc04-41f9-8066-3a3902446640 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:29:38.461848 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84061d00-fc04-41f9-8066-3a3902446640 admin admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:29:38.478657 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 1184/2361] 10.222.0.18 () {58 vars in 1153 bytes} [Tue Sep 13 03:29:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 03:29:38.481494 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e15f9d8b-115a-4273-ac5c-5469a8d07ba0 None None] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:29:38.491716 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e15f9d8b-115a-4273-ac5c-5469a8d07ba0 None None] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:29:38.498747 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6d3b624-5c44-415c-9fd9-0dc5d358cddb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76c5181fc7be45279ac00017bb4640b3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:29:38.499369 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d3b624-5c44-415c-9fd9-0dc5d358cddb admin admin] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:29:38.499642 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d3b624-5c44-415c-9fd9-0dc5d358cddb admin admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:29:38.499912 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d3b624-5c44-415c-9fd9-0dc5d358cddb admin admin] PATH_INFO: `/v3/projects` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:29:38.500798 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6d3b624-5c44-415c-9fd9-0dc5d358cddb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:29:38.504844 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6d3b624-5c44-415c-9fd9-0dc5d358cddb admin admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:29:38.538667 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 1178/2362] 10.222.0.18 () {60 vars in 1153 bytes} [Tue Sep 13 03:29:38 2022] POST /identity/v3/projects => generated 373 bytes in 56 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 03:29:38.541069 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84061d00-fc04-41f9-8066-3a3902446640 admin admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:29:38.552453 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84061d00-fc04-41f9-8066-3a3902446640 admin admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:29:38.615989 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4765ddef-e08e-4198-b770-db251e96be77 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76c5181fc7be45279ac00017bb4640b3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:29:38.616598 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4765ddef-e08e-4198-b770-db251e96be77 admin admin] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:29:38.616881 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4765ddef-e08e-4198-b770-db251e96be77 admin admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:29:38.617140 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4765ddef-e08e-4198-b770-db251e96be77 admin admin] PATH_INFO: `/v3/users` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:29:38.617760 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4765ddef-e08e-4198-b770-db251e96be77 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:29:38.619720 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4765ddef-e08e-4198-b770-db251e96be77 admin admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:29:38.644086 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 1185/2363] 10.222.0.18 () {60 vars in 1144 bytes} [Tue Sep 13 03:29:38 2022] POST /identity/v3/users => generated 287 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 03:29:38.649781 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6d3b624-5c44-415c-9fd9-0dc5d358cddb admin admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:29:38.661421 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6d3b624-5c44-415c-9fd9-0dc5d358cddb admin admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:29:38.666689 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c84d454d-ddbb-422a-8f4c-f6bdc4adf7d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76c5181fc7be45279ac00017bb4640b3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:29:38.667473 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c84d454d-ddbb-422a-8f4c-f6bdc4adf7d5 admin admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:29:38.667753 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c84d454d-ddbb-422a-8f4c-f6bdc4adf7d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:29:38.668028 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c84d454d-ddbb-422a-8f4c-f6bdc4adf7d5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:29:38.668675 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c84d454d-ddbb-422a-8f4c-f6bdc4adf7d5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:29:38.670525 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c84d454d-ddbb-422a-8f4c-f6bdc4adf7d5 admin admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:29:38.682006 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 1179/2364] 10.222.0.18 () {58 vars in 1122 bytes} [Tue Sep 13 03:29:38 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 03:29:38.685962 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4765ddef-e08e-4198-b770-db251e96be77 admin admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:29:38.696841 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4765ddef-e08e-4198-b770-db251e96be77 admin admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:29:38.701565 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d4e401c-efe6-431b-bcbf-4dc8bf4e314e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76c5181fc7be45279ac00017bb4640b3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:29:38.702177 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d4e401c-efe6-431b-bcbf-4dc8bf4e314e admin admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:29:38.702454 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d4e401c-efe6-431b-bcbf-4dc8bf4e314e admin admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:29:38.702767 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d4e401c-efe6-431b-bcbf-4dc8bf4e314e admin admin] PATH_INFO: `/v3/roles` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:29:38.703501 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d4e401c-efe6-431b-bcbf-4dc8bf4e314e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:29:38.705110 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d4e401c-efe6-431b-bcbf-4dc8bf4e314e admin admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:29:38.720325 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 1186/2365] 10.222.0.18 () {58 vars in 1122 bytes} [Tue Sep 13 03:29:38 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 03:29:38.722731 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c84d454d-ddbb-422a-8f4c-f6bdc4adf7d5 admin admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:29:38.732384 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c84d454d-ddbb-422a-8f4c-f6bdc4adf7d5 admin admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:29:38.738839 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92f9f8dc-b90e-4e55-9f8d-e615a9a88115 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76c5181fc7be45279ac00017bb4640b3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:29:38.739537 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f9f8dc-b90e-4e55-9f8d-e615a9a88115 admin admin] REQUEST_METHOD: `PUT` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:29:38.739809 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f9f8dc-b90e-4e55-9f8d-e615a9a88115 admin admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:29:38.740076 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f9f8dc-b90e-4e55-9f8d-e615a9a88115 admin admin] PATH_INFO: `/v3/projects/dfe19a2f55f84e0ab097866392b40e96/users/58421d36134149e0a911a54b05e579b5/roles/1eb7a5b5203945c29e6d22d653f47847` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:29:38.760070 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92f9f8dc-b90e-4e55-9f8d-e615a9a88115 admin admin] RBAC: Authorizing `identity:create_grant(project_id=dfe19a2f55f84e0ab097866392b40e96, user_id=58421d36134149e0a911a54b05e579b5, role_id=1eb7a5b5203945c29e6d22d653f47847)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:29:38.762768 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92f9f8dc-b90e-4e55-9f8d-e615a9a88115 admin admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:29:38.786139 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 1180/2366] 10.222.0.18 () {60 vars in 1483 bytes} [Tue Sep 13 03:29:38 2022] PUT /identity/v3/projects/dfe19a2f55f84e0ab097866392b40e96/users/58421d36134149e0a911a54b05e579b5/roles/1eb7a5b5203945c29e6d22d653f47847 => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 03:29:38.789201 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e6e88e9-3145-483a-b949-c7ad0e8e6650 None None] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:29:38.790396 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e6e88e9-3145-483a-b949-c7ad0e8e6650 None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:29:38.790711 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e6e88e9-3145-483a-b949-c7ad0e8e6650 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:29:38.823728 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.auth.core [None req-9e6e88e9-3145-483a-b949-c7ad0e8e6650 None None] MFA Rules not processed for user `58421d36134149e0a911a54b05e579b5`. Rule list: `[]` (Enabled: `True`). {{(pid=81399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 03:29:38.824979 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-9e6e88e9-3145-483a-b949-c7ad0e8e6650 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:29:38.900663 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 1187/2367] 10.222.0.18 () {58 vars in 958 bytes} [Tue Sep 13 03:29:38 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 03:29:38.913412 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92f9f8dc-b90e-4e55-9f8d-e615a9a88115 admin admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:29:38.915656 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-92f9f8dc-b90e-4e55-9f8d-e615a9a88115 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:29:38.961694 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:29:38.981195 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92f9f8dc-b90e-4e55-9f8d-e615a9a88115 admin admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:29:38.984872 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b802a7f-46d8-48bd-9eef-ed6c498c282b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '924ee3f608d34a18b0ddc4695f102a56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b9d94f6a68f7470d9494f112175620dc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:29:38.985471 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b802a7f-46d8-48bd-9eef-ed6c498c282b service cinder] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:29:38.985746 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b802a7f-46d8-48bd-9eef-ed6c498c282b service cinder] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:29:38.986010 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b802a7f-46d8-48bd-9eef-ed6c498c282b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:29:38.994967 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b802a7f-46d8-48bd-9eef-ed6c498c282b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:29:38.996410 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b802a7f-46d8-48bd-9eef-ed6c498c282b service cinder] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:29:39.007086 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 1181/2368] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Sep 13 03:29:38 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 03:29:39.029497 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e6e88e9-3145-483a-b949-c7ad0e8e6650 None None] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:29:39.047565 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:29:39.119388 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e6e88e9-3145-483a-b949-c7ad0e8e6650 None None] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:29:39.124872 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-282869d8-535e-46ac-93c4-b83985e96945 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76c5181fc7be45279ac00017bb4640b3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:29:39.125594 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-282869d8-535e-46ac-93c4-b83985e96945 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:29:39.125942 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-282869d8-535e-46ac-93c4-b83985e96945 admin admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:29:39.126280 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-282869d8-535e-46ac-93c4-b83985e96945 admin admin] PATH_INFO: `/v3/users/58421d36134149e0a911a54b05e579b5` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:29:39.127759 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-282869d8-535e-46ac-93c4-b83985e96945 admin admin] RBAC: Authorizing `identity:delete_user(user_id=58421d36134149e0a911a54b05e579b5)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:29:39.129767 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-282869d8-535e-46ac-93c4-b83985e96945 admin admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:29:39.178587 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-282869d8-535e-46ac-93c4-b83985e96945 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '58421d36134149e0a911a54b05e579b5'} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:29:39.188056 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-282869d8-535e-46ac-93c4-b83985e96945 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '58421d36134149e0a911a54b05e579b5'} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:29:39.205943 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-282869d8-535e-46ac-93c4-b83985e96945 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '58421d36134149e0a911a54b05e579b5'} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:29:39.206712 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-282869d8-535e-46ac-93c4-b83985e96945 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '58421d36134149e0a911a54b05e579b5'} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:29:39.214110 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-282869d8-535e-46ac-93c4-b83985e96945 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '58421d36134149e0a911a54b05e579b5'} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:29:39.223102 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 1188/2369] 10.222.0.18 () {58 vars in 1224 bytes} [Tue Sep 13 03:29:39 2022] DELETE /identity/v3/users/58421d36134149e0a911a54b05e579b5 => generated 0 bytes in 194 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 03:29:39.227788 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86c25a96-b729-49c7-9204-84314b6dab59 None None] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:29:39.228261 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86c25a96-b729-49c7-9204-84314b6dab59 None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:29:39.228461 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86c25a96-b729-49c7-9204-84314b6dab59 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:29:39.249211 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.auth.core [None req-86c25a96-b729-49c7-9204-84314b6dab59 None None] MFA Rules not processed for user `49f280ab086b4610855c4ffa378517d9`. Rule list: `[]` (Enabled: `True`). {{(pid=81400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 03:29:39.250542 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-86c25a96-b729-49c7-9204-84314b6dab59 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:29:39.315782 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 1182/2370] 10.222.0.18 () {58 vars in 958 bytes} [Tue Sep 13 03:29:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 13 03:29:39.325372 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-282869d8-535e-46ac-93c4-b83985e96945 admin admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:29:39.326883 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-282869d8-535e-46ac-93c4-b83985e96945 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:29:39.392986 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-282869d8-535e-46ac-93c4-b83985e96945 admin admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:29:39.397369 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20322702-6c0e-401e-b674-9fec5243a257 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdffbc851086441ebd01022d1e4c2609', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b9d94f6a68f7470d9494f112175620dc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:29:39.398021 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20322702-6c0e-401e-b674-9fec5243a257 service neutron] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:29:39.398297 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20322702-6c0e-401e-b674-9fec5243a257 service neutron] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:29:39.398564 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20322702-6c0e-401e-b674-9fec5243a257 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:29:39.410741 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20322702-6c0e-401e-b674-9fec5243a257 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:29:39.413150 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20322702-6c0e-401e-b674-9fec5243a257 service neutron] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:29:39.424061 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 1189/2371] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Sep 13 03:29:39 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 03:29:39.695824 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86c25a96-b729-49c7-9204-84314b6dab59 None None] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:29:39.697356 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-86c25a96-b729-49c7-9204-84314b6dab59 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:29:39.710512 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86c25a96-b729-49c7-9204-84314b6dab59 None None] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:29:39.715186 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-941c6d4d-a28c-4e96-96b1-6f5cde484264 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76c5181fc7be45279ac00017bb4640b3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:29:39.715815 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-941c6d4d-a28c-4e96-96b1-6f5cde484264 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:29:39.716221 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-941c6d4d-a28c-4e96-96b1-6f5cde484264 admin admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:29:39.716495 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-941c6d4d-a28c-4e96-96b1-6f5cde484264 admin admin] PATH_INFO: `/v3/projects/dfe19a2f55f84e0ab097866392b40e96` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:29:39.717617 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-941c6d4d-a28c-4e96-96b1-6f5cde484264 admin admin] RBAC: Authorizing `identity:delete_project(project_id=dfe19a2f55f84e0ab097866392b40e96)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:29:39.719573 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-941c6d4d-a28c-4e96-96b1-6f5cde484264 admin admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:29:39.765887 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-941c6d4d-a28c-4e96-96b1-6f5cde484264 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'dfe19a2f55f84e0ab097866392b40e96', 'request_id': 'req-941c6d4d-a28c-4e96-96b1-6f5cde484264'} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:29:39.772048 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-941c6d4d-a28c-4e96-96b1-6f5cde484264 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'dfe19a2f55f84e0ab097866392b40e96', 'request_id': 'req-941c6d4d-a28c-4e96-96b1-6f5cde484264'} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:29:39.778255 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-941c6d4d-a28c-4e96-96b1-6f5cde484264 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'dfe19a2f55f84e0ab097866392b40e96', 'request_id': 'req-941c6d4d-a28c-4e96-96b1-6f5cde484264'} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:29:39.791397 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-941c6d4d-a28c-4e96-96b1-6f5cde484264 admin admin] The token cache is being invalidate because project dfe19a2f55f84e0ab097866392b40e96 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81400) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 03:29:39.791709 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-941c6d4d-a28c-4e96-96b1-6f5cde484264 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:29:39.792321 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-941c6d4d-a28c-4e96-96b1-6f5cde484264 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:29:39.793793 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 1183/2372] 10.222.0.18 () {58 vars in 1233 bytes} [Tue Sep 13 03:29:39 2022] DELETE /identity/v3/projects/dfe19a2f55f84e0ab097866392b40e96 => generated 0 bytes in 99 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 03:29:39.798791 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f47bde4-23f1-464c-8cf6-c99a0ec5114f None None] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:29:39.799262 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f47bde4-23f1-464c-8cf6-c99a0ec5114f None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:29:39.799383 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f47bde4-23f1-464c-8cf6-c99a0ec5114f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:29:39.818420 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.auth.core [None req-8f47bde4-23f1-464c-8cf6-c99a0ec5114f None None] MFA Rules not processed for user `49f280ab086b4610855c4ffa378517d9`. Rule list: `[]` (Enabled: `True`). {{(pid=81399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 03:29:39.819657 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-8f47bde4-23f1-464c-8cf6-c99a0ec5114f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:29:39.850032 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:29:39.881453 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 1190/2373] 10.222.0.18 () {58 vars in 958 bytes} [Tue Sep 13 03:29:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 03:29:39.890031 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-479cb6b7-209d-4153-9a34-fc595efb9121 None None] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:29:39.890311 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-479cb6b7-209d-4153-9a34-fc595efb9121 None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:29:39.890583 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-479cb6b7-209d-4153-9a34-fc595efb9121 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:29:39.911822 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.auth.core [None req-479cb6b7-209d-4153-9a34-fc595efb9121 None None] MFA Rules not processed for user `49f280ab086b4610855c4ffa378517d9`. Rule list: `[]` (Enabled: `True`). {{(pid=81400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 03:29:39.913004 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-479cb6b7-209d-4153-9a34-fc595efb9121 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:29:39.920975 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 1184/2374] 10.222.0.18 () {58 vars in 958 bytes} [Tue Sep 13 03:29:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 03:29:39.925049 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f47bde4-23f1-464c-8cf6-c99a0ec5114f None None] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:29:39.934470 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f47bde4-23f1-464c-8cf6-c99a0ec5114f None None] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:29:39.939016 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3670f5b2-b07c-48d3-abd4-b01a490e3a74 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76c5181fc7be45279ac00017bb4640b3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:29:39.939601 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3670f5b2-b07c-48d3-abd4-b01a490e3a74 admin admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:29:39.939877 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3670f5b2-b07c-48d3-abd4-b01a490e3a74 admin admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:29:39.940258 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3670f5b2-b07c-48d3-abd4-b01a490e3a74 admin admin] PATH_INFO: `/v3/domains` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:29:39.940895 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3670f5b2-b07c-48d3-abd4-b01a490e3a74 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 03:29:39.941222 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3670f5b2-b07c-48d3-abd4-b01a490e3a74 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:29:39.942815 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3670f5b2-b07c-48d3-abd4-b01a490e3a74 admin admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:29:39.954061 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 1191/2375] 10.222.0.18 () {58 vars in 1153 bytes} [Tue Sep 13 03:29:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 03:29:39.958573 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-479cb6b7-209d-4153-9a34-fc595efb9121 None None] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:29:39.967884 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-479cb6b7-209d-4153-9a34-fc595efb9121 None None] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:29:39.972489 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eed26b5b-c2e4-4a8c-baa0-34a77588c450 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76c5181fc7be45279ac00017bb4640b3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:29:39.973079 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed26b5b-c2e4-4a8c-baa0-34a77588c450 admin admin] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:29:39.973354 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed26b5b-c2e4-4a8c-baa0-34a77588c450 admin admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:29:39.974298 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed26b5b-c2e4-4a8c-baa0-34a77588c450 admin admin] PATH_INFO: `/v3/projects` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:29:39.974997 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eed26b5b-c2e4-4a8c-baa0-34a77588c450 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:29:39.976920 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eed26b5b-c2e4-4a8c-baa0-34a77588c450 admin admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:29:40.002780 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 1185/2376] 10.222.0.18 () {60 vars in 1153 bytes} [Tue Sep 13 03:29:39 2022] POST /identity/v3/projects => generated 385 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 03:29:40.006907 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3670f5b2-b07c-48d3-abd4-b01a490e3a74 admin admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:29:40.017350 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3670f5b2-b07c-48d3-abd4-b01a490e3a74 admin admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:29:40.074819 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5d193d8-7a49-48a4-bec0-66f8b26a09f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76c5181fc7be45279ac00017bb4640b3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:29:40.075423 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d193d8-7a49-48a4-bec0-66f8b26a09f1 admin admin] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:29:40.075697 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d193d8-7a49-48a4-bec0-66f8b26a09f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:29:40.075962 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d193d8-7a49-48a4-bec0-66f8b26a09f1 admin admin] PATH_INFO: `/v3/users` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:29:40.076584 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5d193d8-7a49-48a4-bec0-66f8b26a09f1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:29:40.078503 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5d193d8-7a49-48a4-bec0-66f8b26a09f1 admin admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:29:40.100381 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 1192/2377] 10.222.0.18 () {60 vars in 1144 bytes} [Tue Sep 13 03:29:40 2022] POST /identity/v3/users => generated 293 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 13 03:29:40.104542 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eed26b5b-c2e4-4a8c-baa0-34a77588c450 admin admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:29:40.114894 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eed26b5b-c2e4-4a8c-baa0-34a77588c450 admin admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:29:40.119538 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fce96fb-e073-4cd2-91c3-7c2f95d1bc9c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76c5181fc7be45279ac00017bb4640b3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:29:40.120129 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fce96fb-e073-4cd2-91c3-7c2f95d1bc9c admin admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:29:40.120406 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fce96fb-e073-4cd2-91c3-7c2f95d1bc9c admin admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:29:40.120675 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fce96fb-e073-4cd2-91c3-7c2f95d1bc9c admin admin] PATH_INFO: `/v3/roles` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:29:40.121309 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fce96fb-e073-4cd2-91c3-7c2f95d1bc9c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:29:40.122956 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fce96fb-e073-4cd2-91c3-7c2f95d1bc9c admin admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:29:40.136213 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 1186/2378] 10.222.0.18 () {58 vars in 1122 bytes} [Tue Sep 13 03:29:40 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 03:29:40.138041 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5d193d8-7a49-48a4-bec0-66f8b26a09f1 admin admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:29:40.149384 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5d193d8-7a49-48a4-bec0-66f8b26a09f1 admin admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:29:40.153920 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5de84097-7817-4c87-a3c0-0872926a3567 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76c5181fc7be45279ac00017bb4640b3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:29:40.154591 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de84097-7817-4c87-a3c0-0872926a3567 admin admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:29:40.154981 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de84097-7817-4c87-a3c0-0872926a3567 admin admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:29:40.155385 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de84097-7817-4c87-a3c0-0872926a3567 admin admin] PATH_INFO: `/v3/roles` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:29:40.156126 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5de84097-7817-4c87-a3c0-0872926a3567 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:29:40.157738 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5de84097-7817-4c87-a3c0-0872926a3567 admin admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:29:40.176392 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 1193/2379] 10.222.0.18 () {58 vars in 1122 bytes} [Tue Sep 13 03:29:40 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 03:29:40.177447 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5de84097-7817-4c87-a3c0-0872926a3567 admin admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:29:40.178090 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd6f51c-9179-4545-afb0-753fa97787a9 None None] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:29:40.178375 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd6f51c-9179-4545-afb0-753fa97787a9 None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:29:40.178734 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd6f51c-9179-4545-afb0-753fa97787a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:29:40.187352 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5de84097-7817-4c87-a3c0-0872926a3567 admin admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:29:40.192032 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-587886f9-17ae-4c5e-a7bd-ec75e6fea465 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76c5181fc7be45279ac00017bb4640b3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:29:40.193174 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-587886f9-17ae-4c5e-a7bd-ec75e6fea465 admin admin] REQUEST_METHOD: `PUT` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:29:40.193449 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-587886f9-17ae-4c5e-a7bd-ec75e6fea465 admin admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:29:40.193838 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-587886f9-17ae-4c5e-a7bd-ec75e6fea465 admin admin] PATH_INFO: `/v3/projects/39bb499a38674a57a81227d07c012e1d/users/80b3e44e4afc447f9e90417098723582/roles/1eb7a5b5203945c29e6d22d653f47847` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:29:40.197612 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.auth.core [None req-9dd6f51c-9179-4545-afb0-753fa97787a9 None None] MFA Rules not processed for user `47fa1f7ce54145ceb3564b9ba7b4964f`. Rule list: `[]` (Enabled: `True`). {{(pid=81400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 03:29:40.198841 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-9dd6f51c-9179-4545-afb0-753fa97787a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:29:40.208715 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-587886f9-17ae-4c5e-a7bd-ec75e6fea465 admin admin] RBAC: Authorizing `identity:create_grant(project_id=39bb499a38674a57a81227d07c012e1d, user_id=80b3e44e4afc447f9e90417098723582, role_id=1eb7a5b5203945c29e6d22d653f47847)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:29:40.211203 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-587886f9-17ae-4c5e-a7bd-ec75e6fea465 admin admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:29:40.219599 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:29:40.224896 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 1194/2380] 10.222.0.18 () {60 vars in 1483 bytes} [Tue Sep 13 03:29:40 2022] PUT /identity/v3/projects/39bb499a38674a57a81227d07c012e1d/users/80b3e44e4afc447f9e90417098723582/roles/1eb7a5b5203945c29e6d22d653f47847 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 03:29:40.235809 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98db2602-b6c1-4736-b1fd-2980d529e462 None None] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:29:40.236918 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98db2602-b6c1-4736-b1fd-2980d529e462 None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:29:40.237205 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98db2602-b6c1-4736-b1fd-2980d529e462 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:29:40.257415 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 1187/2381] 10.222.0.18 () {58 vars in 958 bytes} [Tue Sep 13 03:29:40 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 03:29:40.265678 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.auth.core [None req-98db2602-b6c1-4736-b1fd-2980d529e462 None None] MFA Rules not processed for user `80b3e44e4afc447f9e90417098723582`. Rule list: `[]` (Enabled: `True`). {{(pid=81399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 03:29:40.267491 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-98db2602-b6c1-4736-b1fd-2980d529e462 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:29:40.334958 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 1195/2382] 10.222.0.18 () {58 vars in 958 bytes} [Tue Sep 13 03:29:40 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 13 03:29:40.345724 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dd6f51c-9179-4545-afb0-753fa97787a9 None None] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:29:40.347361 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-9dd6f51c-9179-4545-afb0-753fa97787a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:29:40.416876 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dd6f51c-9179-4545-afb0-753fa97787a9 None None] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:29:40.466986 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e701a387-2aba-4a0b-ac2c-c1e5aa3b04d4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '924ee3f608d34a18b0ddc4695f102a56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b9d94f6a68f7470d9494f112175620dc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:29:40.468030 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e701a387-2aba-4a0b-ac2c-c1e5aa3b04d4 service cinder] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:29:40.468030 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e701a387-2aba-4a0b-ac2c-c1e5aa3b04d4 service cinder] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:29:40.468234 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e701a387-2aba-4a0b-ac2c-c1e5aa3b04d4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:29:40.476616 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e701a387-2aba-4a0b-ac2c-c1e5aa3b04d4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:29:40.478174 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e701a387-2aba-4a0b-ac2c-c1e5aa3b04d4 service cinder] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:29:40.487133 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 1188/2383] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Sep 13 03:29:40 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 142 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 03:29:49.927809 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98db2602-b6c1-4736-b1fd-2980d529e462 None None] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:29:49.933233 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:29:49.989548 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98db2602-b6c1-4736-b1fd-2980d529e462 None None] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:29:49.993996 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f293220a-4f62-4979-8cb6-430634ff6641 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76c5181fc7be45279ac00017bb4640b3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:29:49.994625 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f293220a-4f62-4979-8cb6-430634ff6641 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:29:49.994907 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f293220a-4f62-4979-8cb6-430634ff6641 admin admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:29:49.995180 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f293220a-4f62-4979-8cb6-430634ff6641 admin admin] PATH_INFO: `/v3/users/80b3e44e4afc447f9e90417098723582` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:29:49.996273 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f293220a-4f62-4979-8cb6-430634ff6641 admin admin] RBAC: Authorizing `identity:delete_user(user_id=80b3e44e4afc447f9e90417098723582)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:29:49.998176 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f293220a-4f62-4979-8cb6-430634ff6641 admin admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:29:50.048443 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-f293220a-4f62-4979-8cb6-430634ff6641 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '80b3e44e4afc447f9e90417098723582'} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:29:50.057725 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-f293220a-4f62-4979-8cb6-430634ff6641 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '80b3e44e4afc447f9e90417098723582'} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:29:50.070674 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-f293220a-4f62-4979-8cb6-430634ff6641 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '80b3e44e4afc447f9e90417098723582'} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:29:50.071358 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-f293220a-4f62-4979-8cb6-430634ff6641 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '80b3e44e4afc447f9e90417098723582'} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:29:50.078516 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-f293220a-4f62-4979-8cb6-430634ff6641 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '80b3e44e4afc447f9e90417098723582'} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:29:50.084815 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 1196/2384] 10.222.0.18 () {58 vars in 1224 bytes} [Tue Sep 13 03:29:49 2022] DELETE /identity/v3/users/80b3e44e4afc447f9e90417098723582 => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 03:29:50.089268 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98c1c928-fc13-4bd4-b716-e30f6597c241 None None] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:29:50.089758 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98c1c928-fc13-4bd4-b716-e30f6597c241 None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:29:50.089854 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98c1c928-fc13-4bd4-b716-e30f6597c241 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:29:50.108844 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.auth.core [None req-98c1c928-fc13-4bd4-b716-e30f6597c241 None None] MFA Rules not processed for user `49f280ab086b4610855c4ffa378517d9`. Rule list: `[]` (Enabled: `True`). {{(pid=81400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 03:29:50.110024 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-98c1c928-fc13-4bd4-b716-e30f6597c241 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:29:50.168311 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 1189/2385] 10.222.0.18 () {58 vars in 958 bytes} [Tue Sep 13 03:29:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 03:29:50.178413 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f293220a-4f62-4979-8cb6-430634ff6641 admin admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:29:50.179825 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-f293220a-4f62-4979-8cb6-430634ff6641 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:29:50.236170 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f293220a-4f62-4979-8cb6-430634ff6641 admin admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:29:50.239825 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5da0db2-abdc-43bf-b478-0ecf7507bbaa service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdffbc851086441ebd01022d1e4c2609', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b9d94f6a68f7470d9494f112175620dc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:29:50.240432 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5da0db2-abdc-43bf-b478-0ecf7507bbaa service neutron] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:29:50.240709 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5da0db2-abdc-43bf-b478-0ecf7507bbaa service neutron] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:29:50.240980 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5da0db2-abdc-43bf-b478-0ecf7507bbaa service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:29:50.249838 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5da0db2-abdc-43bf-b478-0ecf7507bbaa service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:29:50.251314 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5da0db2-abdc-43bf-b478-0ecf7507bbaa service neutron] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:29:50.261275 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 1197/2386] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Sep 13 03:29:50 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 03:29:50.607300 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98c1c928-fc13-4bd4-b716-e30f6597c241 None None] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:29:50.608731 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-98c1c928-fc13-4bd4-b716-e30f6597c241 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:29:50.621011 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98c1c928-fc13-4bd4-b716-e30f6597c241 None None] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:29:50.625812 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcbc0b47-3530-4c35-8a10-b71fcfb32956 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76c5181fc7be45279ac00017bb4640b3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:29:50.626464 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcbc0b47-3530-4c35-8a10-b71fcfb32956 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:29:50.626768 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcbc0b47-3530-4c35-8a10-b71fcfb32956 admin admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:29:50.627039 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcbc0b47-3530-4c35-8a10-b71fcfb32956 admin admin] PATH_INFO: `/v3/projects/39bb499a38674a57a81227d07c012e1d` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:29:50.628162 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcbc0b47-3530-4c35-8a10-b71fcfb32956 admin admin] RBAC: Authorizing `identity:delete_project(project_id=39bb499a38674a57a81227d07c012e1d)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:29:50.630097 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcbc0b47-3530-4c35-8a10-b71fcfb32956 admin admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:29:50.661170 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=99,keystone:DELETE=1 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:29:50.679793 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-bcbc0b47-3530-4c35-8a10-b71fcfb32956 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '39bb499a38674a57a81227d07c012e1d', 'request_id': 'req-bcbc0b47-3530-4c35-8a10-b71fcfb32956'} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:29:50.685477 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-bcbc0b47-3530-4c35-8a10-b71fcfb32956 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '39bb499a38674a57a81227d07c012e1d', 'request_id': 'req-bcbc0b47-3530-4c35-8a10-b71fcfb32956'} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:29:50.691671 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-bcbc0b47-3530-4c35-8a10-b71fcfb32956 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '39bb499a38674a57a81227d07c012e1d', 'request_id': 'req-bcbc0b47-3530-4c35-8a10-b71fcfb32956'} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:29:50.704716 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-bcbc0b47-3530-4c35-8a10-b71fcfb32956 admin admin] The token cache is being invalidate because project 39bb499a38674a57a81227d07c012e1d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81400) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 03:29:50.704997 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-bcbc0b47-3530-4c35-8a10-b71fcfb32956 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:29:50.705611 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-bcbc0b47-3530-4c35-8a10-b71fcfb32956 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:29:50.707111 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 1190/2387] 10.222.0.18 () {58 vars in 1233 bytes} [Tue Sep 13 03:29:50 2022] DELETE /identity/v3/projects/39bb499a38674a57a81227d07c012e1d => generated 0 bytes in 100 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 03:29:50.711901 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac438df-df6a-4ee5-a447-b671578a891a None None] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:29:50.712187 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac438df-df6a-4ee5-a447-b671578a891a None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:29:50.712464 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac438df-df6a-4ee5-a447-b671578a891a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:29:50.731398 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.auth.core [None req-9ac438df-df6a-4ee5-a447-b671578a891a None None] MFA Rules not processed for user `49f280ab086b4610855c4ffa378517d9`. Rule list: `[]` (Enabled: `True`). {{(pid=81399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 03:29:50.732586 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-9ac438df-df6a-4ee5-a447-b671578a891a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:29:50.752630 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:29:50.793177 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 1198/2388] 10.222.0.18 () {58 vars in 958 bytes} [Tue Sep 13 03:29:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 03:29:50.801170 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0030098-6e9b-4cc3-8b33-78f2941413bf None None] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:29:50.801450 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0030098-6e9b-4cc3-8b33-78f2941413bf None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:29:50.801720 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0030098-6e9b-4cc3-8b33-78f2941413bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:29:50.820986 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.auth.core [None req-a0030098-6e9b-4cc3-8b33-78f2941413bf None None] MFA Rules not processed for user `49f280ab086b4610855c4ffa378517d9`. Rule list: `[]` (Enabled: `True`). {{(pid=81400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 03:29:50.822336 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-a0030098-6e9b-4cc3-8b33-78f2941413bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:29:50.830001 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 1191/2389] 10.222.0.18 () {58 vars in 958 bytes} [Tue Sep 13 03:29:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 03:29:50.834380 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ac438df-df6a-4ee5-a447-b671578a891a None None] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:29:50.844664 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ac438df-df6a-4ee5-a447-b671578a891a None None] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:29:50.850125 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-add4d4f1-d9a2-444f-8c34-60a170a3ac04 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76c5181fc7be45279ac00017bb4640b3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:29:50.850751 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add4d4f1-d9a2-444f-8c34-60a170a3ac04 admin admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:29:50.851027 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add4d4f1-d9a2-444f-8c34-60a170a3ac04 admin admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:29:50.851293 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add4d4f1-d9a2-444f-8c34-60a170a3ac04 admin admin] PATH_INFO: `/v3/domains` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:29:50.851926 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-add4d4f1-d9a2-444f-8c34-60a170a3ac04 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 03:29:50.852225 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-add4d4f1-d9a2-444f-8c34-60a170a3ac04 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:29:50.853808 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-add4d4f1-d9a2-444f-8c34-60a170a3ac04 admin admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:29:50.865335 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 1199/2390] 10.222.0.18 () {58 vars in 1153 bytes} [Tue Sep 13 03:29:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 03:29:50.869535 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0030098-6e9b-4cc3-8b33-78f2941413bf None None] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:29:50.878496 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0030098-6e9b-4cc3-8b33-78f2941413bf None None] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:29:50.883053 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be7f0778-6b28-4a2e-994e-3926433d02b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76c5181fc7be45279ac00017bb4640b3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:29:50.883636 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be7f0778-6b28-4a2e-994e-3926433d02b5 admin admin] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:29:50.883907 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be7f0778-6b28-4a2e-994e-3926433d02b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:29:50.884175 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be7f0778-6b28-4a2e-994e-3926433d02b5 admin admin] PATH_INFO: `/v3/projects` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:29:50.884791 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be7f0778-6b28-4a2e-994e-3926433d02b5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:29:50.886696 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be7f0778-6b28-4a2e-994e-3926433d02b5 admin admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:29:50.913383 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 1192/2391] 10.222.0.18 () {60 vars in 1153 bytes} [Tue Sep 13 03:29:50 2022] POST /identity/v3/projects => generated 383 bytes in 44 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 03:29:50.917742 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-add4d4f1-d9a2-444f-8c34-60a170a3ac04 admin admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:29:50.927738 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-add4d4f1-d9a2-444f-8c34-60a170a3ac04 admin admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:29:50.982656 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8678db89-cd3d-4b0b-add8-f0f759a6e63c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76c5181fc7be45279ac00017bb4640b3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:29:50.983304 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8678db89-cd3d-4b0b-add8-f0f759a6e63c admin admin] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:29:50.983443 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8678db89-cd3d-4b0b-add8-f0f759a6e63c admin admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:29:50.983716 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8678db89-cd3d-4b0b-add8-f0f759a6e63c admin admin] PATH_INFO: `/v3/users` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:29:50.984336 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8678db89-cd3d-4b0b-add8-f0f759a6e63c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:29:50.986243 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8678db89-cd3d-4b0b-add8-f0f759a6e63c admin admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:29:51.007059 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 1200/2392] 10.222.0.18 () {60 vars in 1144 bytes} [Tue Sep 13 03:29:50 2022] POST /identity/v3/users => generated 292 bytes in 90 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 03:29:51.011083 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be7f0778-6b28-4a2e-994e-3926433d02b5 admin admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:29:51.021827 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be7f0778-6b28-4a2e-994e-3926433d02b5 admin admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:29:51.026435 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c64a0faa-77b0-4231-baf5-9094f14f6eff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76c5181fc7be45279ac00017bb4640b3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:29:51.027060 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c64a0faa-77b0-4231-baf5-9094f14f6eff admin admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:29:51.027334 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c64a0faa-77b0-4231-baf5-9094f14f6eff admin admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:29:51.027598 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c64a0faa-77b0-4231-baf5-9094f14f6eff admin admin] PATH_INFO: `/v3/roles` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:29:51.028233 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c64a0faa-77b0-4231-baf5-9094f14f6eff admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:29:51.029824 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c64a0faa-77b0-4231-baf5-9094f14f6eff admin admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:29:51.039805 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 1193/2393] 10.222.0.18 () {58 vars in 1122 bytes} [Tue Sep 13 03:29:51 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 03:29:51.043412 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8678db89-cd3d-4b0b-add8-f0f759a6e63c admin admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:29:51.053171 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8678db89-cd3d-4b0b-add8-f0f759a6e63c admin admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:29:51.057868 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eef0bc12-983c-45fd-9a7e-343cae11753f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76c5181fc7be45279ac00017bb4640b3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:29:51.058464 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef0bc12-983c-45fd-9a7e-343cae11753f admin admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:29:51.058778 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef0bc12-983c-45fd-9a7e-343cae11753f admin admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:29:51.059052 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef0bc12-983c-45fd-9a7e-343cae11753f admin admin] PATH_INFO: `/v3/roles` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:29:51.059688 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eef0bc12-983c-45fd-9a7e-343cae11753f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:29:51.061290 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eef0bc12-983c-45fd-9a7e-343cae11753f admin admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:29:51.074901 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 1201/2394] 10.222.0.18 () {58 vars in 1122 bytes} [Tue Sep 13 03:29:51 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 03:29:51.079235 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c64a0faa-77b0-4231-baf5-9094f14f6eff admin admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:29:51.089756 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c64a0faa-77b0-4231-baf5-9094f14f6eff admin admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:29:51.094332 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1b5d50d-100c-45b7-8d78-983c6f972350 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76c5181fc7be45279ac00017bb4640b3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:29:51.095008 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b5d50d-100c-45b7-8d78-983c6f972350 admin admin] REQUEST_METHOD: `PUT` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:29:51.095288 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b5d50d-100c-45b7-8d78-983c6f972350 admin admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:29:51.095561 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b5d50d-100c-45b7-8d78-983c6f972350 admin admin] PATH_INFO: `/v3/projects/d3071e0b97c3439bb412568e9e98f3d2/users/b130d5a8cd5440a79ca2b075ce635d3c/roles/1eb7a5b5203945c29e6d22d653f47847` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:29:51.109982 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1b5d50d-100c-45b7-8d78-983c6f972350 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d3071e0b97c3439bb412568e9e98f3d2, user_id=b130d5a8cd5440a79ca2b075ce635d3c, role_id=1eb7a5b5203945c29e6d22d653f47847)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:29:51.112723 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1b5d50d-100c-45b7-8d78-983c6f972350 admin admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:29:51.125751 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 1194/2395] 10.222.0.18 () {60 vars in 1483 bytes} [Tue Sep 13 03:29:51 2022] PUT /identity/v3/projects/d3071e0b97c3439bb412568e9e98f3d2/users/b130d5a8cd5440a79ca2b075ce635d3c/roles/1eb7a5b5203945c29e6d22d653f47847 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 03:29:51.137480 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e7eddc-5f64-4ef1-928e-9df9ff83342b None None] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:29:51.137763 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e7eddc-5f64-4ef1-928e-9df9ff83342b None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:29:51.138033 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e7eddc-5f64-4ef1-928e-9df9ff83342b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:29:51.167071 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.auth.core [None req-79e7eddc-5f64-4ef1-928e-9df9ff83342b None None] MFA Rules not processed for user `b130d5a8cd5440a79ca2b075ce635d3c`. Rule list: `[]` (Enabled: `True`). {{(pid=81399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 03:29:51.168271 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-79e7eddc-5f64-4ef1-928e-9df9ff83342b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:29:51.235076 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 1202/2396] 10.222.0.18 () {58 vars in 958 bytes} [Tue Sep 13 03:29:51 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 03:29:51.239625 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1b5d50d-100c-45b7-8d78-983c6f972350 admin admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:29:51.302841 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1b5d50d-100c-45b7-8d78-983c6f972350 admin admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:29:51.307860 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d1de344-2b1b-4511-b5a5-cc5d0ec036e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76c5181fc7be45279ac00017bb4640b3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:29:51.308457 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d1de344-2b1b-4511-b5a5-cc5d0ec036e8 admin admin] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:29:51.308735 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d1de344-2b1b-4511-b5a5-cc5d0ec036e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:29:51.309004 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d1de344-2b1b-4511-b5a5-cc5d0ec036e8 admin admin] PATH_INFO: `/v3/projects` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:29:51.309639 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d1de344-2b1b-4511-b5a5-cc5d0ec036e8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:29:51.311606 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d1de344-2b1b-4511-b5a5-cc5d0ec036e8 admin admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:29:51.337976 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 1195/2397] 10.222.0.18 () {60 vars in 1153 bytes} [Tue Sep 13 03:29:51 2022] POST /identity/v3/projects => generated 385 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 03:29:51.342631 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79e7eddc-5f64-4ef1-928e-9df9ff83342b None None] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:29:51.353887 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79e7eddc-5f64-4ef1-928e-9df9ff83342b None None] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:29:51.372557 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:29:51.410395 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e81a375-114e-482a-988c-942506f12d37 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76c5181fc7be45279ac00017bb4640b3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:29:51.411031 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e81a375-114e-482a-988c-942506f12d37 admin admin] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:29:51.411310 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e81a375-114e-482a-988c-942506f12d37 admin admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:29:51.411582 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e81a375-114e-482a-988c-942506f12d37 admin admin] PATH_INFO: `/v3/users` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:29:51.412211 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e81a375-114e-482a-988c-942506f12d37 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:29:51.414139 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e81a375-114e-482a-988c-942506f12d37 admin admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:29:51.434926 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 1203/2398] 10.222.0.18 () {60 vars in 1144 bytes} [Tue Sep 13 03:29:51 2022] POST /identity/v3/users => generated 293 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 03:29:51.438808 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d1de344-2b1b-4511-b5a5-cc5d0ec036e8 admin admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:29:51.449545 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d1de344-2b1b-4511-b5a5-cc5d0ec036e8 admin admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:29:51.454077 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4d8332c-c253-43ec-9404-c91cb5a5360f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76c5181fc7be45279ac00017bb4640b3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:29:51.454699 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d8332c-c253-43ec-9404-c91cb5a5360f admin admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:29:51.454974 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d8332c-c253-43ec-9404-c91cb5a5360f admin admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:29:51.455240 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d8332c-c253-43ec-9404-c91cb5a5360f admin admin] PATH_INFO: `/v3/roles` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:29:51.455864 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4d8332c-c253-43ec-9404-c91cb5a5360f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:29:51.457447 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4d8332c-c253-43ec-9404-c91cb5a5360f admin admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:29:51.468173 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 1196/2399] 10.222.0.18 () {58 vars in 1122 bytes} [Tue Sep 13 03:29:51 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 03:29:51.471803 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e81a375-114e-482a-988c-942506f12d37 admin admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:29:51.481532 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e81a375-114e-482a-988c-942506f12d37 admin admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:29:51.486171 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-712ddde2-cc84-49a2-be5c-f9b163744eff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76c5181fc7be45279ac00017bb4640b3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:29:51.486734 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-712ddde2-cc84-49a2-be5c-f9b163744eff admin admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:29:51.487015 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-712ddde2-cc84-49a2-be5c-f9b163744eff admin admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:29:51.487288 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-712ddde2-cc84-49a2-be5c-f9b163744eff admin admin] PATH_INFO: `/v3/roles` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:29:51.487918 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-712ddde2-cc84-49a2-be5c-f9b163744eff admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:29:51.490045 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-712ddde2-cc84-49a2-be5c-f9b163744eff admin admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:29:51.499790 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 1204/2400] 10.222.0.18 () {58 vars in 1122 bytes} [Tue Sep 13 03:29:51 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 03:29:51.503509 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4d8332c-c253-43ec-9404-c91cb5a5360f admin admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:29:51.514604 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4d8332c-c253-43ec-9404-c91cb5a5360f admin admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:29:51.519370 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7f41e26-2cf0-4b0e-869b-1df043a8e40c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76c5181fc7be45279ac00017bb4640b3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:29:51.520015 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7f41e26-2cf0-4b0e-869b-1df043a8e40c admin admin] REQUEST_METHOD: `PUT` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:29:51.520285 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7f41e26-2cf0-4b0e-869b-1df043a8e40c admin admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:29:51.520765 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7f41e26-2cf0-4b0e-869b-1df043a8e40c admin admin] PATH_INFO: `/v3/projects/82b2d48fd14148eb82ffd497ed37bfdd/users/da7c0c8ef4a043cea652c78b81cb25b8/roles/1eb7a5b5203945c29e6d22d653f47847` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:29:51.535407 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7f41e26-2cf0-4b0e-869b-1df043a8e40c admin admin] RBAC: Authorizing `identity:create_grant(project_id=82b2d48fd14148eb82ffd497ed37bfdd, user_id=da7c0c8ef4a043cea652c78b81cb25b8, role_id=1eb7a5b5203945c29e6d22d653f47847)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:29:51.538329 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7f41e26-2cf0-4b0e-869b-1df043a8e40c admin admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:29:51.548271 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:DELETE=6,keystone:SELECT=89,keystone:INSERT=5 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:29:51.553612 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 1197/2401] 10.222.0.18 () {60 vars in 1483 bytes} [Tue Sep 13 03:29:51 2022] PUT /identity/v3/projects/82b2d48fd14148eb82ffd497ed37bfdd/users/da7c0c8ef4a043cea652c78b81cb25b8/roles/1eb7a5b5203945c29e6d22d653f47847 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 03:29:51.564214 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b497e1f-0719-4648-be43-39bffc005674 None None] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:29:51.564497 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b497e1f-0719-4648-be43-39bffc005674 None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:29:51.564841 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b497e1f-0719-4648-be43-39bffc005674 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:29:51.595321 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.auth.core [None req-3b497e1f-0719-4648-be43-39bffc005674 None None] MFA Rules not processed for user `da7c0c8ef4a043cea652c78b81cb25b8`. Rule list: `[]` (Enabled: `True`). {{(pid=81399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 03:29:51.596663 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-3b497e1f-0719-4648-be43-39bffc005674 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:29:51.666284 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 1205/2402] 10.222.0.18 () {58 vars in 958 bytes} [Tue Sep 13 03:29:51 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 03:29:51.671648 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7f41e26-2cf0-4b0e-869b-1df043a8e40c admin admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:29:51.735695 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7f41e26-2cf0-4b0e-869b-1df043a8e40c admin admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:29:51.740918 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c96add1-b1a3-45e3-8953-16972ae98626 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76c5181fc7be45279ac00017bb4640b3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:29:51.741458 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c96add1-b1a3-45e3-8953-16972ae98626 admin admin] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:29:51.741730 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c96add1-b1a3-45e3-8953-16972ae98626 admin admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:29:51.741998 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c96add1-b1a3-45e3-8953-16972ae98626 admin admin] PATH_INFO: `/v3/projects` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:29:51.742680 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c96add1-b1a3-45e3-8953-16972ae98626 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:29:51.744601 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c96add1-b1a3-45e3-8953-16972ae98626 admin admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:29:51.776820 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 1198/2403] 10.222.0.18 () {60 vars in 1153 bytes} [Tue Sep 13 03:29:51 2022] POST /identity/v3/projects => generated 383 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 03:29:51.778524 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b497e1f-0719-4648-be43-39bffc005674 None None] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:29:51.792585 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b497e1f-0719-4648-be43-39bffc005674 None None] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:29:51.850213 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b838a3f5-e010-46e0-8503-306c78b764be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76c5181fc7be45279ac00017bb4640b3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:29:51.850843 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b838a3f5-e010-46e0-8503-306c78b764be admin admin] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:29:51.851120 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b838a3f5-e010-46e0-8503-306c78b764be admin admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:29:51.851388 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b838a3f5-e010-46e0-8503-306c78b764be admin admin] PATH_INFO: `/v3/users` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:29:51.852012 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b838a3f5-e010-46e0-8503-306c78b764be admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:29:51.853927 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b838a3f5-e010-46e0-8503-306c78b764be admin admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:29:51.876720 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 1206/2404] 10.222.0.18 () {60 vars in 1144 bytes} [Tue Sep 13 03:29:51 2022] POST /identity/v3/users => generated 298 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 03:29:51.881976 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c96add1-b1a3-45e3-8953-16972ae98626 admin admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:29:51.892098 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c96add1-b1a3-45e3-8953-16972ae98626 admin admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:29:51.896671 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e374e485-b844-4efd-b50e-0b2dc24f8965 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76c5181fc7be45279ac00017bb4640b3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:29:51.897256 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e374e485-b844-4efd-b50e-0b2dc24f8965 admin admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:29:51.897531 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e374e485-b844-4efd-b50e-0b2dc24f8965 admin admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:29:51.897799 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e374e485-b844-4efd-b50e-0b2dc24f8965 admin admin] PATH_INFO: `/v3/roles` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:29:51.898430 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e374e485-b844-4efd-b50e-0b2dc24f8965 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:29:51.900069 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e374e485-b844-4efd-b50e-0b2dc24f8965 admin admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:29:51.910339 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 1199/2405] 10.222.0.18 () {58 vars in 1122 bytes} [Tue Sep 13 03:29:51 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 03:29:51.915059 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b838a3f5-e010-46e0-8503-306c78b764be admin admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:29:51.924991 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b838a3f5-e010-46e0-8503-306c78b764be admin admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:29:51.930252 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad94ec74-e16c-4ff0-8d03-15516d148130 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76c5181fc7be45279ac00017bb4640b3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:29:51.930935 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad94ec74-e16c-4ff0-8d03-15516d148130 admin admin] REQUEST_METHOD: `PUT` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:29:51.931213 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad94ec74-e16c-4ff0-8d03-15516d148130 admin admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:29:51.931484 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad94ec74-e16c-4ff0-8d03-15516d148130 admin admin] PATH_INFO: `/v3/projects/34b6715483f24045a5bc906c69bdd96e/users/e3f76bc6f6ae40249c1a55cd849ff620/roles/425d4b481c7846e0bc07e7e805d5b15f` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:29:51.945905 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad94ec74-e16c-4ff0-8d03-15516d148130 admin admin] RBAC: Authorizing `identity:create_grant(project_id=34b6715483f24045a5bc906c69bdd96e, user_id=e3f76bc6f6ae40249c1a55cd849ff620, role_id=425d4b481c7846e0bc07e7e805d5b15f)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:29:51.948576 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad94ec74-e16c-4ff0-8d03-15516d148130 admin admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:29:51.951560 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:29:51.962380 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 1207/2406] 10.222.0.18 () {60 vars in 1483 bytes} [Tue Sep 13 03:29:51 2022] PUT /identity/v3/projects/34b6715483f24045a5bc906c69bdd96e/users/e3f76bc6f6ae40249c1a55cd849ff620/roles/425d4b481c7846e0bc07e7e805d5b15f => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 13 03:29:51.966358 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e374e485-b844-4efd-b50e-0b2dc24f8965 admin admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:29:51.977107 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e374e485-b844-4efd-b50e-0b2dc24f8965 admin admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:29:52.030873 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f35058ff-2f9f-4b7b-837d-4867e7211c2b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76c5181fc7be45279ac00017bb4640b3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:29:52.031396 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f35058ff-2f9f-4b7b-837d-4867e7211c2b admin admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:29:52.031667 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f35058ff-2f9f-4b7b-837d-4867e7211c2b admin admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:29:52.031935 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f35058ff-2f9f-4b7b-837d-4867e7211c2b admin admin] PATH_INFO: `/v3/roles` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:29:52.032554 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f35058ff-2f9f-4b7b-837d-4867e7211c2b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:29:52.034245 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f35058ff-2f9f-4b7b-837d-4867e7211c2b admin admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:29:52.044605 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 1200/2407] 10.222.0.18 () {58 vars in 1122 bytes} [Tue Sep 13 03:29:51 2022] GET /identity/v3/roles => generated 1420 bytes in 79 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 03:29:52.048596 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad94ec74-e16c-4ff0-8d03-15516d148130 admin admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:29:52.059375 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad94ec74-e16c-4ff0-8d03-15516d148130 admin admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:29:52.063960 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1cdc357-33ee-40bb-92d7-56a54fd01ad4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76c5181fc7be45279ac00017bb4640b3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:29:52.064608 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1cdc357-33ee-40bb-92d7-56a54fd01ad4 admin admin] REQUEST_METHOD: `PUT` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:29:52.064882 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1cdc357-33ee-40bb-92d7-56a54fd01ad4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:29:52.065152 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1cdc357-33ee-40bb-92d7-56a54fd01ad4 admin admin] PATH_INFO: `/v3/projects/34b6715483f24045a5bc906c69bdd96e/users/e3f76bc6f6ae40249c1a55cd849ff620/roles/425d4b481c7846e0bc07e7e805d5b15f` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:29:52.067113 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1cdc357-33ee-40bb-92d7-56a54fd01ad4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=34b6715483f24045a5bc906c69bdd96e, user_id=e3f76bc6f6ae40249c1a55cd849ff620, role_id=425d4b481c7846e0bc07e7e805d5b15f)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:29:52.069753 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1cdc357-33ee-40bb-92d7-56a54fd01ad4 admin admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:29:52.080545 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 1208/2408] 10.222.0.18 () {60 vars in 1483 bytes} [Tue Sep 13 03:29:52 2022] PUT /identity/v3/projects/34b6715483f24045a5bc906c69bdd96e/users/e3f76bc6f6ae40249c1a55cd849ff620/roles/425d4b481c7846e0bc07e7e805d5b15f => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 13 03:29:52.092142 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2791a98-4008-438c-97ca-0a2ecc6c77b3 None None] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:29:52.092651 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2791a98-4008-438c-97ca-0a2ecc6c77b3 None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:29:52.092767 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2791a98-4008-438c-97ca-0a2ecc6c77b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:29:52.120335 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.auth.core [None req-d2791a98-4008-438c-97ca-0a2ecc6c77b3 None None] MFA Rules not processed for user `e3f76bc6f6ae40249c1a55cd849ff620`. Rule list: `[]` (Enabled: `True`). {{(pid=81400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 03:29:52.122174 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-d2791a98-4008-438c-97ca-0a2ecc6c77b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:29:52.180801 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:29:52.190601 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 1201/2409] 10.222.0.18 () {58 vars in 958 bytes} [Tue Sep 13 03:29:52 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 03:29:52.201895 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1cdc357-33ee-40bb-92d7-56a54fd01ad4 admin admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:29:52.203565 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-a1cdc357-33ee-40bb-92d7-56a54fd01ad4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:29:52.260307 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1cdc357-33ee-40bb-92d7-56a54fd01ad4 admin admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:29:52.264122 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70f630c8-c210-4d69-a5ac-c7777db4cef5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '924ee3f608d34a18b0ddc4695f102a56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b9d94f6a68f7470d9494f112175620dc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:29:52.264734 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f630c8-c210-4d69-a5ac-c7777db4cef5 service cinder] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:29:52.265022 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f630c8-c210-4d69-a5ac-c7777db4cef5 service cinder] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:29:52.265295 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f630c8-c210-4d69-a5ac-c7777db4cef5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:29:52.320309 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70f630c8-c210-4d69-a5ac-c7777db4cef5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:29:52.321663 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70f630c8-c210-4d69-a5ac-c7777db4cef5 service cinder] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:29:52.330219 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 1209/2410] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Sep 13 03:29:52 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 129 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 03:29:53.822884 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2791a98-4008-438c-97ca-0a2ecc6c77b3 None None] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:29:53.824382 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-d2791a98-4008-438c-97ca-0a2ecc6c77b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:29:53.836260 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2791a98-4008-438c-97ca-0a2ecc6c77b3 None None] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:29:53.839828 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-707901ae-e071-47b9-8cd2-ab4ed2546bf6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '924ee3f608d34a18b0ddc4695f102a56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b9d94f6a68f7470d9494f112175620dc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:29:53.840851 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-707901ae-e071-47b9-8cd2-ab4ed2546bf6 service cinder] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:29:53.841187 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-707901ae-e071-47b9-8cd2-ab4ed2546bf6 service cinder] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:29:53.841607 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-707901ae-e071-47b9-8cd2-ab4ed2546bf6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:29:53.897916 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-707901ae-e071-47b9-8cd2-ab4ed2546bf6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:29:53.899459 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-707901ae-e071-47b9-8cd2-ab4ed2546bf6 service cinder] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:29:53.908168 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 1202/2411] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Sep 13 03:29:53 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 03:29:54.488960 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70f630c8-c210-4d69-a5ac-c7777db4cef5 service cinder] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:29:54.497890 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70f630c8-c210-4d69-a5ac-c7777db4cef5 service cinder] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:29:54.501501 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b7af794-1448-4700-a14c-18a9f4745e1b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '924ee3f608d34a18b0ddc4695f102a56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b9d94f6a68f7470d9494f112175620dc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:29:54.502098 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b7af794-1448-4700-a14c-18a9f4745e1b service cinder] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:29:54.502374 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b7af794-1448-4700-a14c-18a9f4745e1b service cinder] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:29:54.502680 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b7af794-1448-4700-a14c-18a9f4745e1b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:29:54.512551 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b7af794-1448-4700-a14c-18a9f4745e1b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:29:54.513972 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b7af794-1448-4700-a14c-18a9f4745e1b service cinder] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:29:54.523203 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 1210/2412] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Sep 13 03:29:54 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 03:29:58.377180 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-707901ae-e071-47b9-8cd2-ab4ed2546bf6 service cinder] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:29:58.437619 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-707901ae-e071-47b9-8cd2-ab4ed2546bf6 service cinder] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:29:58.442012 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93d356e9-0fde-4270-ac94-9fcf9ceb1d10 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76c5181fc7be45279ac00017bb4640b3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:29:58.443164 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93d356e9-0fde-4270-ac94-9fcf9ceb1d10 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:29:58.443435 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93d356e9-0fde-4270-ac94-9fcf9ceb1d10 admin admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:29:58.443701 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93d356e9-0fde-4270-ac94-9fcf9ceb1d10 admin admin] PATH_INFO: `/v3/users/b130d5a8cd5440a79ca2b075ce635d3c` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:29:58.444788 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93d356e9-0fde-4270-ac94-9fcf9ceb1d10 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b130d5a8cd5440a79ca2b075ce635d3c)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:29:58.446719 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93d356e9-0fde-4270-ac94-9fcf9ceb1d10 admin admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:29:58.491109 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-93d356e9-0fde-4270-ac94-9fcf9ceb1d10 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b130d5a8cd5440a79ca2b075ce635d3c'} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:29:58.497723 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-93d356e9-0fde-4270-ac94-9fcf9ceb1d10 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b130d5a8cd5440a79ca2b075ce635d3c'} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:29:58.498321 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-93d356e9-0fde-4270-ac94-9fcf9ceb1d10 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b130d5a8cd5440a79ca2b075ce635d3c'} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:29:58.510355 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-93d356e9-0fde-4270-ac94-9fcf9ceb1d10 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b130d5a8cd5440a79ca2b075ce635d3c'} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:29:58.517098 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-93d356e9-0fde-4270-ac94-9fcf9ceb1d10 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b130d5a8cd5440a79ca2b075ce635d3c'} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:29:58.523584 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 1203/2413] 10.222.0.18 () {58 vars in 1224 bytes} [Tue Sep 13 03:29:58 2022] DELETE /identity/v3/users/b130d5a8cd5440a79ca2b075ce635d3c => generated 0 bytes in 147 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 03:29:58.527799 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec08bae-1a4f-4ef7-92af-df3e7fc9251c None None] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:29:58.528296 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec08bae-1a4f-4ef7-92af-df3e7fc9251c None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:29:58.528394 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec08bae-1a4f-4ef7-92af-df3e7fc9251c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:29:58.546907 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.auth.core [None req-8ec08bae-1a4f-4ef7-92af-df3e7fc9251c None None] MFA Rules not processed for user `49f280ab086b4610855c4ffa378517d9`. Rule list: `[]` (Enabled: `True`). {{(pid=81399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 03:29:58.548076 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-8ec08bae-1a4f-4ef7-92af-df3e7fc9251c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:29:58.604411 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 1211/2414] 10.222.0.18 () {58 vars in 958 bytes} [Tue Sep 13 03:29:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 03:29:58.614085 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93d356e9-0fde-4270-ac94-9fcf9ceb1d10 admin admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:29:58.615527 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-93d356e9-0fde-4270-ac94-9fcf9ceb1d10 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:29:58.629394 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:29:58.673802 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93d356e9-0fde-4270-ac94-9fcf9ceb1d10 admin admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:29:58.677418 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8755fc08-af9e-4d60-a23f-3086e5babc13 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdffbc851086441ebd01022d1e4c2609', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b9d94f6a68f7470d9494f112175620dc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:29:58.678005 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8755fc08-af9e-4d60-a23f-3086e5babc13 service neutron] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:29:58.678276 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8755fc08-af9e-4d60-a23f-3086e5babc13 service neutron] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:29:58.678541 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8755fc08-af9e-4d60-a23f-3086e5babc13 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:29:58.687259 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8755fc08-af9e-4d60-a23f-3086e5babc13 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:29:58.688677 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8755fc08-af9e-4d60-a23f-3086e5babc13 service neutron] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:29:58.698497 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 1204/2415] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Sep 13 03:29:58 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 03:29:59.013235 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ec08bae-1a4f-4ef7-92af-df3e7fc9251c None None] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:29:59.014868 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-8ec08bae-1a4f-4ef7-92af-df3e7fc9251c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:29:59.029873 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ec08bae-1a4f-4ef7-92af-df3e7fc9251c None None] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:29:59.035220 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bce1d48-0d3d-4bca-aad1-7c02306b2444 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76c5181fc7be45279ac00017bb4640b3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:29:59.035760 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bce1d48-0d3d-4bca-aad1-7c02306b2444 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:29:59.036094 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bce1d48-0d3d-4bca-aad1-7c02306b2444 admin admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:29:59.036420 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bce1d48-0d3d-4bca-aad1-7c02306b2444 admin admin] PATH_INFO: `/v3/projects/d3071e0b97c3439bb412568e9e98f3d2` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:29:59.037572 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bce1d48-0d3d-4bca-aad1-7c02306b2444 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d3071e0b97c3439bb412568e9e98f3d2)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:29:59.040185 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bce1d48-0d3d-4bca-aad1-7c02306b2444 admin admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:29:59.065335 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:29:59.098337 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-7bce1d48-0d3d-4bca-aad1-7c02306b2444 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd3071e0b97c3439bb412568e9e98f3d2', 'request_id': 'req-7bce1d48-0d3d-4bca-aad1-7c02306b2444'} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:29:59.103857 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-7bce1d48-0d3d-4bca-aad1-7c02306b2444 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd3071e0b97c3439bb412568e9e98f3d2', 'request_id': 'req-7bce1d48-0d3d-4bca-aad1-7c02306b2444'} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:29:59.110152 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-7bce1d48-0d3d-4bca-aad1-7c02306b2444 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd3071e0b97c3439bb412568e9e98f3d2', 'request_id': 'req-7bce1d48-0d3d-4bca-aad1-7c02306b2444'} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:29:59.123210 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-7bce1d48-0d3d-4bca-aad1-7c02306b2444 admin admin] The token cache is being invalidate because project d3071e0b97c3439bb412568e9e98f3d2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81399) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 03:29:59.123525 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-7bce1d48-0d3d-4bca-aad1-7c02306b2444 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:29:59.124149 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-7bce1d48-0d3d-4bca-aad1-7c02306b2444 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:29:59.125627 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 1212/2416] 10.222.0.18 () {58 vars in 1233 bytes} [Tue Sep 13 03:29:59 2022] DELETE /identity/v3/projects/d3071e0b97c3439bb412568e9e98f3d2 => generated 0 bytes in 113 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 03:29:59.130423 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8755fc08-af9e-4d60-a23f-3086e5babc13 service neutron] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:29:59.140388 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8755fc08-af9e-4d60-a23f-3086e5babc13 service neutron] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:29:59.201959 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3385e11e-38e5-4c59-a4a0-a83e248b6060 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76c5181fc7be45279ac00017bb4640b3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:29:59.202590 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3385e11e-38e5-4c59-a4a0-a83e248b6060 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:29:59.202893 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3385e11e-38e5-4c59-a4a0-a83e248b6060 admin admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:29:59.203163 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3385e11e-38e5-4c59-a4a0-a83e248b6060 admin admin] PATH_INFO: `/v3/users/da7c0c8ef4a043cea652c78b81cb25b8` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:29:59.204291 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3385e11e-38e5-4c59-a4a0-a83e248b6060 admin admin] RBAC: Authorizing `identity:delete_user(user_id=da7c0c8ef4a043cea652c78b81cb25b8)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:29:59.206248 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3385e11e-38e5-4c59-a4a0-a83e248b6060 admin admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:29:59.254292 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-3385e11e-38e5-4c59-a4a0-a83e248b6060 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'da7c0c8ef4a043cea652c78b81cb25b8'} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:29:59.263567 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-3385e11e-38e5-4c59-a4a0-a83e248b6060 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'da7c0c8ef4a043cea652c78b81cb25b8'} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:29:59.264750 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-3385e11e-38e5-4c59-a4a0-a83e248b6060 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'da7c0c8ef4a043cea652c78b81cb25b8'} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:29:59.279587 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-3385e11e-38e5-4c59-a4a0-a83e248b6060 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'da7c0c8ef4a043cea652c78b81cb25b8'} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:29:59.289167 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-3385e11e-38e5-4c59-a4a0-a83e248b6060 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'da7c0c8ef4a043cea652c78b81cb25b8'} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:29:59.296673 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 1205/2417] 10.222.0.18 () {58 vars in 1224 bytes} [Tue Sep 13 03:29:59 2022] DELETE /identity/v3/users/da7c0c8ef4a043cea652c78b81cb25b8 => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 03:29:59.573239 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bce1d48-0d3d-4bca-aad1-7c02306b2444 admin admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:29:59.574883 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-7bce1d48-0d3d-4bca-aad1-7c02306b2444 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:29:59.588076 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bce1d48-0d3d-4bca-aad1-7c02306b2444 admin admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:29:59.643495 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-001c8726-b660-4f38-a979-3343d9d293c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76c5181fc7be45279ac00017bb4640b3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:29:59.644058 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-001c8726-b660-4f38-a979-3343d9d293c1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:29:59.644342 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-001c8726-b660-4f38-a979-3343d9d293c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:29:59.644610 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-001c8726-b660-4f38-a979-3343d9d293c1 admin admin] PATH_INFO: `/v3/projects/82b2d48fd14148eb82ffd497ed37bfdd` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:29:59.646202 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-001c8726-b660-4f38-a979-3343d9d293c1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=82b2d48fd14148eb82ffd497ed37bfdd)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:29:59.648169 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-001c8726-b660-4f38-a979-3343d9d293c1 admin admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:29:59.693073 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-001c8726-b660-4f38-a979-3343d9d293c1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '82b2d48fd14148eb82ffd497ed37bfdd', 'request_id': 'req-001c8726-b660-4f38-a979-3343d9d293c1'} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:29:59.698529 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-001c8726-b660-4f38-a979-3343d9d293c1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '82b2d48fd14148eb82ffd497ed37bfdd', 'request_id': 'req-001c8726-b660-4f38-a979-3343d9d293c1'} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:29:59.704505 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-001c8726-b660-4f38-a979-3343d9d293c1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '82b2d48fd14148eb82ffd497ed37bfdd', 'request_id': 'req-001c8726-b660-4f38-a979-3343d9d293c1'} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:29:59.716857 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-001c8726-b660-4f38-a979-3343d9d293c1 admin admin] The token cache is being invalidate because project 82b2d48fd14148eb82ffd497ed37bfdd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81399) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 03:29:59.717148 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-001c8726-b660-4f38-a979-3343d9d293c1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:29:59.717763 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-001c8726-b660-4f38-a979-3343d9d293c1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:29:59.719255 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 1213/2418] 10.222.0.18 () {58 vars in 1233 bytes} [Tue Sep 13 03:29:59 2022] DELETE /identity/v3/projects/82b2d48fd14148eb82ffd497ed37bfdd => generated 0 bytes in 147 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 03:29:59.723292 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3385e11e-38e5-4c59-a4a0-a83e248b6060 admin admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:29:59.725311 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-3385e11e-38e5-4c59-a4a0-a83e248b6060 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:29:59.738531 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3385e11e-38e5-4c59-a4a0-a83e248b6060 admin admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:29:59.769658 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:29:59.798236 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-125f21b6-3b18-4858-8a14-7b4e7ae7e472 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76c5181fc7be45279ac00017bb4640b3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:29:59.798923 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-125f21b6-3b18-4858-8a14-7b4e7ae7e472 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:29:59.799254 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-125f21b6-3b18-4858-8a14-7b4e7ae7e472 admin admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:29:59.799523 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-125f21b6-3b18-4858-8a14-7b4e7ae7e472 admin admin] PATH_INFO: `/v3/users/e3f76bc6f6ae40249c1a55cd849ff620` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:29:59.800687 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-125f21b6-3b18-4858-8a14-7b4e7ae7e472 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e3f76bc6f6ae40249c1a55cd849ff620)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:29:59.802599 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-125f21b6-3b18-4858-8a14-7b4e7ae7e472 admin admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:29:59.851082 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-125f21b6-3b18-4858-8a14-7b4e7ae7e472 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e3f76bc6f6ae40249c1a55cd849ff620'} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:29:59.858837 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-125f21b6-3b18-4858-8a14-7b4e7ae7e472 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e3f76bc6f6ae40249c1a55cd849ff620'} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:29:59.859438 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-125f21b6-3b18-4858-8a14-7b4e7ae7e472 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e3f76bc6f6ae40249c1a55cd849ff620'} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:29:59.871499 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-125f21b6-3b18-4858-8a14-7b4e7ae7e472 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e3f76bc6f6ae40249c1a55cd849ff620'} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:29:59.878289 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-125f21b6-3b18-4858-8a14-7b4e7ae7e472 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e3f76bc6f6ae40249c1a55cd849ff620'} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:29:59.884265 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 1206/2419] 10.222.0.18 () {58 vars in 1224 bytes} [Tue Sep 13 03:29:59 2022] DELETE /identity/v3/users/e3f76bc6f6ae40249c1a55cd849ff620 => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 03:30:00.233289 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-001c8726-b660-4f38-a979-3343d9d293c1 admin admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:30:00.234745 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-001c8726-b660-4f38-a979-3343d9d293c1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:30:00.252644 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-001c8726-b660-4f38-a979-3343d9d293c1 admin admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:30:00.286433 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:30:00.311058 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-551adc5b-8150-42a8-9571-2293d1929025 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76c5181fc7be45279ac00017bb4640b3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:30:00.311735 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-551adc5b-8150-42a8-9571-2293d1929025 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:30:00.312001 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-551adc5b-8150-42a8-9571-2293d1929025 admin admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:30:00.312270 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-551adc5b-8150-42a8-9571-2293d1929025 admin admin] PATH_INFO: `/v3/projects/34b6715483f24045a5bc906c69bdd96e` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:30:00.313382 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-551adc5b-8150-42a8-9571-2293d1929025 admin admin] RBAC: Authorizing `identity:delete_project(project_id=34b6715483f24045a5bc906c69bdd96e)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:30:00.315357 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-551adc5b-8150-42a8-9571-2293d1929025 admin admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:30:00.360917 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-551adc5b-8150-42a8-9571-2293d1929025 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '34b6715483f24045a5bc906c69bdd96e', 'request_id': 'req-551adc5b-8150-42a8-9571-2293d1929025'} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:30:00.367681 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-551adc5b-8150-42a8-9571-2293d1929025 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '34b6715483f24045a5bc906c69bdd96e', 'request_id': 'req-551adc5b-8150-42a8-9571-2293d1929025'} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:30:00.373931 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-551adc5b-8150-42a8-9571-2293d1929025 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '34b6715483f24045a5bc906c69bdd96e', 'request_id': 'req-551adc5b-8150-42a8-9571-2293d1929025'} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:30:00.386545 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-551adc5b-8150-42a8-9571-2293d1929025 admin admin] The token cache is being invalidate because project 34b6715483f24045a5bc906c69bdd96e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81399) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 03:30:00.386865 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-551adc5b-8150-42a8-9571-2293d1929025 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:30:00.387475 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-551adc5b-8150-42a8-9571-2293d1929025 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:30:00.388948 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 1214/2420] 10.222.0.18 () {58 vars in 1233 bytes} [Tue Sep 13 03:30:00 2022] DELETE /identity/v3/projects/34b6715483f24045a5bc906c69bdd96e => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 03:30:00.393830 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eca7de9-965a-46b4-a9b7-499b29059313 None None] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:30:00.394101 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eca7de9-965a-46b4-a9b7-499b29059313 None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:30:00.394373 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eca7de9-965a-46b4-a9b7-499b29059313 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:30:00.413648 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.auth.core [None req-8eca7de9-965a-46b4-a9b7-499b29059313 None None] MFA Rules not processed for user `49f280ab086b4610855c4ffa378517d9`. Rule list: `[]` (Enabled: `True`). {{(pid=81400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 03:30:00.415203 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-8eca7de9-965a-46b4-a9b7-499b29059313 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:30:00.475776 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 1207/2421] 10.222.0.18 () {58 vars in 958 bytes} [Tue Sep 13 03:30:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 03:30:00.483541 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46cb0d42-dd91-48c4-b721-d16fcb8485fe None None] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:30:00.483824 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46cb0d42-dd91-48c4-b721-d16fcb8485fe None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:30:00.484094 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46cb0d42-dd91-48c4-b721-d16fcb8485fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:30:00.504094 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.auth.core [None req-46cb0d42-dd91-48c4-b721-d16fcb8485fe None None] MFA Rules not processed for user `49f280ab086b4610855c4ffa378517d9`. Rule list: `[]` (Enabled: `True`). {{(pid=81399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 03:30:00.505853 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-46cb0d42-dd91-48c4-b721-d16fcb8485fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:30:00.514038 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 1215/2422] 10.222.0.18 () {58 vars in 958 bytes} [Tue Sep 13 03:30:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 03:30:00.518132 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eca7de9-965a-46b4-a9b7-499b29059313 None None] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:30:00.528076 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eca7de9-965a-46b4-a9b7-499b29059313 None None] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:30:00.532584 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-785cd6ac-fdf8-44ac-b7c8-0180bd7a02a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76c5181fc7be45279ac00017bb4640b3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:30:00.533187 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-785cd6ac-fdf8-44ac-b7c8-0180bd7a02a2 admin admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:30:00.533464 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-785cd6ac-fdf8-44ac-b7c8-0180bd7a02a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:30:00.533735 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-785cd6ac-fdf8-44ac-b7c8-0180bd7a02a2 admin admin] PATH_INFO: `/v3/domains` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:30:00.534375 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-785cd6ac-fdf8-44ac-b7c8-0180bd7a02a2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 03:30:00.534722 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-785cd6ac-fdf8-44ac-b7c8-0180bd7a02a2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:30:00.536321 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-785cd6ac-fdf8-44ac-b7c8-0180bd7a02a2 admin admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:30:00.548335 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 1208/2423] 10.222.0.18 () {58 vars in 1153 bytes} [Tue Sep 13 03:30:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 03:30:00.553762 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46cb0d42-dd91-48c4-b721-d16fcb8485fe None None] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:30:00.563136 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46cb0d42-dd91-48c4-b721-d16fcb8485fe None None] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:30:00.568646 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5734a44-4653-46a2-928b-90555ea968b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76c5181fc7be45279ac00017bb4640b3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:30:00.569253 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5734a44-4653-46a2-928b-90555ea968b3 admin admin] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:30:00.569531 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5734a44-4653-46a2-928b-90555ea968b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:30:00.569800 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5734a44-4653-46a2-928b-90555ea968b3 admin admin] PATH_INFO: `/v3/projects` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:30:00.570429 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5734a44-4653-46a2-928b-90555ea968b3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:30:00.572385 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5734a44-4653-46a2-928b-90555ea968b3 admin admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:30:00.599286 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 1216/2424] 10.222.0.18 () {60 vars in 1153 bytes} [Tue Sep 13 03:30:00 2022] POST /identity/v3/projects => generated 381 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 03:30:00.603437 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-785cd6ac-fdf8-44ac-b7c8-0180bd7a02a2 admin admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:30:00.613511 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-785cd6ac-fdf8-44ac-b7c8-0180bd7a02a2 admin admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:30:00.655511 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:30:00.673435 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-761ed707-11b2-406f-be5f-bf5a85bc4533 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76c5181fc7be45279ac00017bb4640b3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:30:00.673964 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-761ed707-11b2-406f-be5f-bf5a85bc4533 admin admin] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:30:00.674242 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-761ed707-11b2-406f-be5f-bf5a85bc4533 admin admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:30:00.674513 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-761ed707-11b2-406f-be5f-bf5a85bc4533 admin admin] PATH_INFO: `/v3/users` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:30:00.675184 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-761ed707-11b2-406f-be5f-bf5a85bc4533 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:30:00.677605 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-761ed707-11b2-406f-be5f-bf5a85bc4533 admin admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:30:00.698552 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 1209/2425] 10.222.0.18 () {60 vars in 1144 bytes} [Tue Sep 13 03:30:00 2022] POST /identity/v3/users => generated 291 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 13 03:30:00.702759 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5734a44-4653-46a2-928b-90555ea968b3 admin admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:30:00.713095 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5734a44-4653-46a2-928b-90555ea968b3 admin admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:30:00.718002 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6eba5ee-8f04-4688-8a1e-d261e9e04c51 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76c5181fc7be45279ac00017bb4640b3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:30:00.718725 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6eba5ee-8f04-4688-8a1e-d261e9e04c51 admin admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:30:00.719007 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6eba5ee-8f04-4688-8a1e-d261e9e04c51 admin admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:30:00.719280 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6eba5ee-8f04-4688-8a1e-d261e9e04c51 admin admin] PATH_INFO: `/v3/roles` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:30:00.719923 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6eba5ee-8f04-4688-8a1e-d261e9e04c51 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:30:00.721524 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6eba5ee-8f04-4688-8a1e-d261e9e04c51 admin admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:30:00.732455 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 1217/2426] 10.222.0.18 () {58 vars in 1122 bytes} [Tue Sep 13 03:30:00 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 03:30:00.736456 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-761ed707-11b2-406f-be5f-bf5a85bc4533 admin admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:30:00.747422 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-761ed707-11b2-406f-be5f-bf5a85bc4533 admin admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:30:00.752220 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-279368ae-9bf4-4a8e-aaf3-c110042b327e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76c5181fc7be45279ac00017bb4640b3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:30:00.752917 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-279368ae-9bf4-4a8e-aaf3-c110042b327e admin admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:30:00.753197 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-279368ae-9bf4-4a8e-aaf3-c110042b327e admin admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:30:00.753464 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-279368ae-9bf4-4a8e-aaf3-c110042b327e admin admin] PATH_INFO: `/v3/roles` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:30:00.754095 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-279368ae-9bf4-4a8e-aaf3-c110042b327e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:30:00.755764 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-279368ae-9bf4-4a8e-aaf3-c110042b327e admin admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:30:00.766553 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 1210/2427] 10.222.0.18 () {58 vars in 1122 bytes} [Tue Sep 13 03:30:00 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 03:30:00.770911 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6eba5ee-8f04-4688-8a1e-d261e9e04c51 admin admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:30:00.781033 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6eba5ee-8f04-4688-8a1e-d261e9e04c51 admin admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:30:00.785903 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60705cb3-bf01-494d-8aa1-76b41278e7f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76c5181fc7be45279ac00017bb4640b3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:30:00.787075 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60705cb3-bf01-494d-8aa1-76b41278e7f3 admin admin] REQUEST_METHOD: `PUT` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:30:00.787347 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60705cb3-bf01-494d-8aa1-76b41278e7f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:30:00.787618 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60705cb3-bf01-494d-8aa1-76b41278e7f3 admin admin] PATH_INFO: `/v3/projects/35d0f9365a60474590ea1bb78b1654b2/users/92bba932a70f4a75b1bb1810d8826447/roles/1eb7a5b5203945c29e6d22d653f47847` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:30:00.803134 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60705cb3-bf01-494d-8aa1-76b41278e7f3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=35d0f9365a60474590ea1bb78b1654b2, user_id=92bba932a70f4a75b1bb1810d8826447, role_id=1eb7a5b5203945c29e6d22d653f47847)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:30:00.805792 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60705cb3-bf01-494d-8aa1-76b41278e7f3 admin admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:30:00.820398 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 1218/2428] 10.222.0.18 () {60 vars in 1483 bytes} [Tue Sep 13 03:30:00 2022] PUT /identity/v3/projects/35d0f9365a60474590ea1bb78b1654b2/users/92bba932a70f4a75b1bb1810d8826447/roles/1eb7a5b5203945c29e6d22d653f47847 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 13 03:30:00.825631 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c85a0af7-ed5f-4e8c-8552-6ed99dc72831 None None] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:30:00.825902 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c85a0af7-ed5f-4e8c-8552-6ed99dc72831 None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:30:00.826173 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c85a0af7-ed5f-4e8c-8552-6ed99dc72831 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:30:00.845644 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.auth.core [None req-c85a0af7-ed5f-4e8c-8552-6ed99dc72831 None None] MFA Rules not processed for user `49f280ab086b4610855c4ffa378517d9`. Rule list: `[]` (Enabled: `True`). {{(pid=81400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 03:30:00.846878 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-c85a0af7-ed5f-4e8c-8552-6ed99dc72831 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:30:00.904872 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 1211/2429] 10.222.0.18 () {58 vars in 958 bytes} [Tue Sep 13 03:30:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 03:30:00.915168 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60705cb3-bf01-494d-8aa1-76b41278e7f3 admin admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:30:00.916839 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-60705cb3-bf01-494d-8aa1-76b41278e7f3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:30:00.972056 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60705cb3-bf01-494d-8aa1-76b41278e7f3 admin admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:30:00.976181 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d37c400-5549-4d82-bb1c-a8568f3eb828 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdffbc851086441ebd01022d1e4c2609', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b9d94f6a68f7470d9494f112175620dc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:30:00.976771 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d37c400-5549-4d82-bb1c-a8568f3eb828 service neutron] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:30:00.977048 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d37c400-5549-4d82-bb1c-a8568f3eb828 service neutron] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:30:00.977314 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d37c400-5549-4d82-bb1c-a8568f3eb828 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:30:00.985999 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d37c400-5549-4d82-bb1c-a8568f3eb828 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:30:00.987452 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d37c400-5549-4d82-bb1c-a8568f3eb828 service neutron] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:30:00.998346 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 1219/2430] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Sep 13 03:30:00 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 03:30:05.510128 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6625b52e-3e86-485d-905d-029233a610e7 None None] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:30:05.510650 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6625b52e-3e86-485d-905d-029233a610e7 None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:30:05.510650 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6625b52e-3e86-485d-905d-029233a610e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:30:05.541077 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.auth.core [None req-6625b52e-3e86-485d-905d-029233a610e7 None None] MFA Rules not processed for user `92bba932a70f4a75b1bb1810d8826447`. Rule list: `[]` (Enabled: `True`). {{(pid=81400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 03:30:05.542295 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-6625b52e-3e86-485d-905d-029233a610e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:30:05.611124 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 1212/2431] 10.222.0.18 () {58 vars in 958 bytes} [Tue Sep 13 03:30:05 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 03:30:05.615770 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d37c400-5549-4d82-bb1c-a8568f3eb828 service neutron] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:30:05.625450 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d37c400-5549-4d82-bb1c-a8568f3eb828 service neutron] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:30:05.630074 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfbd0e63-53f2-4469-a66d-2f91cb37267a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76c5181fc7be45279ac00017bb4640b3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:30:05.630697 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfbd0e63-53f2-4469-a66d-2f91cb37267a admin admin] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:30:05.630975 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfbd0e63-53f2-4469-a66d-2f91cb37267a admin admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:30:05.631248 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfbd0e63-53f2-4469-a66d-2f91cb37267a admin admin] PATH_INFO: `/v3/projects` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:30:05.631877 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfbd0e63-53f2-4469-a66d-2f91cb37267a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:30:05.633787 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfbd0e63-53f2-4469-a66d-2f91cb37267a admin admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:30:05.638461 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:30:05.664815 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 1220/2432] 10.222.0.18 () {60 vars in 1153 bytes} [Tue Sep 13 03:30:05 2022] POST /identity/v3/projects => generated 385 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 13 03:30:05.668932 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6625b52e-3e86-485d-905d-029233a610e7 None None] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:30:05.680598 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6625b52e-3e86-485d-905d-029233a610e7 None None] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:30:05.728696 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:30:05.739901 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd82c255-edf3-4d4a-b5df-fd8528a9f4b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76c5181fc7be45279ac00017bb4640b3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:30:05.740511 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd82c255-edf3-4d4a-b5df-fd8528a9f4b2 admin admin] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:30:05.740778 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd82c255-edf3-4d4a-b5df-fd8528a9f4b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:30:05.741044 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd82c255-edf3-4d4a-b5df-fd8528a9f4b2 admin admin] PATH_INFO: `/v3/users` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:30:05.741673 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd82c255-edf3-4d4a-b5df-fd8528a9f4b2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:30:05.743754 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd82c255-edf3-4d4a-b5df-fd8528a9f4b2 admin admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:30:05.768651 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 1213/2433] 10.222.0.18 () {60 vars in 1144 bytes} [Tue Sep 13 03:30:05 2022] POST /identity/v3/users => generated 299 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 03:30:05.770033 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfbd0e63-53f2-4469-a66d-2f91cb37267a admin admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:30:05.782127 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfbd0e63-53f2-4469-a66d-2f91cb37267a admin admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:30:05.788042 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bd9f0c2-c628-495e-bfbd-c1210393b125 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76c5181fc7be45279ac00017bb4640b3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:30:05.788724 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd9f0c2-c628-495e-bfbd-c1210393b125 admin admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:30:05.789067 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd9f0c2-c628-495e-bfbd-c1210393b125 admin admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:30:05.789685 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd9f0c2-c628-495e-bfbd-c1210393b125 admin admin] PATH_INFO: `/v3/roles` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:30:05.790393 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bd9f0c2-c628-495e-bfbd-c1210393b125 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:30:05.792073 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bd9f0c2-c628-495e-bfbd-c1210393b125 admin admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:30:05.802303 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 1221/2434] 10.222.0.18 () {58 vars in 1122 bytes} [Tue Sep 13 03:30:05 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 03:30:05.806532 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd82c255-edf3-4d4a-b5df-fd8528a9f4b2 admin admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:30:05.817944 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd82c255-edf3-4d4a-b5df-fd8528a9f4b2 admin admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:30:05.824346 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04eca66b-8f0c-4da4-bb6e-e81dac4d15d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76c5181fc7be45279ac00017bb4640b3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:30:05.825017 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04eca66b-8f0c-4da4-bb6e-e81dac4d15d8 admin admin] REQUEST_METHOD: `PUT` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:30:05.825318 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04eca66b-8f0c-4da4-bb6e-e81dac4d15d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:30:05.825613 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04eca66b-8f0c-4da4-bb6e-e81dac4d15d8 admin admin] PATH_INFO: `/v3/projects/0baadaacd0434d82af969deb277f2194/users/a4d295248be84eb9901f3415da4b4901/roles/425d4b481c7846e0bc07e7e805d5b15f` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:30:05.845149 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04eca66b-8f0c-4da4-bb6e-e81dac4d15d8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0baadaacd0434d82af969deb277f2194, user_id=a4d295248be84eb9901f3415da4b4901, role_id=425d4b481c7846e0bc07e7e805d5b15f)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:30:05.847872 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04eca66b-8f0c-4da4-bb6e-e81dac4d15d8 admin admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:30:05.862707 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 1214/2435] 10.222.0.18 () {60 vars in 1483 bytes} [Tue Sep 13 03:30:05 2022] PUT /identity/v3/projects/0baadaacd0434d82af969deb277f2194/users/a4d295248be84eb9901f3415da4b4901/roles/425d4b481c7846e0bc07e7e805d5b15f => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 03:30:05.866634 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bd9f0c2-c628-495e-bfbd-c1210393b125 admin admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:30:05.876942 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bd9f0c2-c628-495e-bfbd-c1210393b125 admin admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:30:05.934958 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48cd9cb1-2fc9-4589-84f8-a0e450c844ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76c5181fc7be45279ac00017bb4640b3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:30:05.935571 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48cd9cb1-2fc9-4589-84f8-a0e450c844ce admin admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:30:05.935849 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48cd9cb1-2fc9-4589-84f8-a0e450c844ce admin admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:30:05.936119 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48cd9cb1-2fc9-4589-84f8-a0e450c844ce admin admin] PATH_INFO: `/v3/roles` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:30:05.936774 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48cd9cb1-2fc9-4589-84f8-a0e450c844ce admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:30:05.938411 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48cd9cb1-2fc9-4589-84f8-a0e450c844ce admin admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:30:05.950348 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 1222/2436] 10.222.0.18 () {58 vars in 1122 bytes} [Tue Sep 13 03:30:05 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 03:30:05.953537 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04eca66b-8f0c-4da4-bb6e-e81dac4d15d8 admin admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:30:05.964621 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04eca66b-8f0c-4da4-bb6e-e81dac4d15d8 admin admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:30:05.969369 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d65dfecf-eda7-4989-8092-39050ea66de5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76c5181fc7be45279ac00017bb4640b3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:30:05.970040 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d65dfecf-eda7-4989-8092-39050ea66de5 admin admin] REQUEST_METHOD: `PUT` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:30:05.970319 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d65dfecf-eda7-4989-8092-39050ea66de5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:30:05.970594 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d65dfecf-eda7-4989-8092-39050ea66de5 admin admin] PATH_INFO: `/v3/projects/0baadaacd0434d82af969deb277f2194/users/a4d295248be84eb9901f3415da4b4901/roles/425d4b481c7846e0bc07e7e805d5b15f` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:30:05.972651 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d65dfecf-eda7-4989-8092-39050ea66de5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0baadaacd0434d82af969deb277f2194, user_id=a4d295248be84eb9901f3415da4b4901, role_id=425d4b481c7846e0bc07e7e805d5b15f)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:30:05.975514 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d65dfecf-eda7-4989-8092-39050ea66de5 admin admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:30:05.986593 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 1215/2437] 10.222.0.18 () {60 vars in 1483 bytes} [Tue Sep 13 03:30:05 2022] PUT /identity/v3/projects/0baadaacd0434d82af969deb277f2194/users/a4d295248be84eb9901f3415da4b4901/roles/425d4b481c7846e0bc07e7e805d5b15f => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 13 03:30:10.479783 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa96243c-b027-4ade-9b3e-10ad5ca20457 None None] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:30:10.480273 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa96243c-b027-4ade-9b3e-10ad5ca20457 None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:30:10.480380 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa96243c-b027-4ade-9b3e-10ad5ca20457 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:30:10.511571 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.auth.core [None req-fa96243c-b027-4ade-9b3e-10ad5ca20457 None None] MFA Rules not processed for user `a4d295248be84eb9901f3415da4b4901`. Rule list: `[]` (Enabled: `True`). {{(pid=81399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 03:30:10.512733 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-fa96243c-b027-4ade-9b3e-10ad5ca20457 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:30:10.588841 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 1223/2438] 10.222.0.18 () {58 vars in 958 bytes} [Tue Sep 13 03:30:10 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 03:30:10.600863 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d65dfecf-eda7-4989-8092-39050ea66de5 admin admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:30:10.602592 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-d65dfecf-eda7-4989-8092-39050ea66de5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:30:10.663130 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d65dfecf-eda7-4989-8092-39050ea66de5 admin admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:30:10.666934 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd3bf54b-232d-498c-a8a8-ed43c8cff350 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b5fdc2ff46a14638b409a13e73ba5500', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b9d94f6a68f7470d9494f112175620dc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:30:10.667539 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd3bf54b-232d-498c-a8a8-ed43c8cff350 service glance] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:30:10.667814 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd3bf54b-232d-498c-a8a8-ed43c8cff350 service glance] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:30:10.668082 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd3bf54b-232d-498c-a8a8-ed43c8cff350 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:30:10.725799 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd3bf54b-232d-498c-a8a8-ed43c8cff350 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:30:10.726891 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd3bf54b-232d-498c-a8a8-ed43c8cff350 service glance] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:30:10.736090 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 1216/2439] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Sep 13 03:30:10 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 136 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 03:30:10.881300 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46115a2f-88b0-41ae-ba60-f79f3bf11ab1 None None] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:30:10.881753 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46115a2f-88b0-41ae-ba60-f79f3bf11ab1 None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:30:10.882087 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46115a2f-88b0-41ae-ba60-f79f3bf11ab1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:30:10.919732 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.auth.core [None req-46115a2f-88b0-41ae-ba60-f79f3bf11ab1 None None] MFA Rules not processed for user `4c2d20fb54a54ecc99b47f237be4917c`. Rule list: `[]` (Enabled: `True`). {{(pid=81399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 03:30:10.921036 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-46115a2f-88b0-41ae-ba60-f79f3bf11ab1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:30:10.968004 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:30:10.990258 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 1224/2440] 10.222.0.18 () {58 vars in 1009 bytes} [Tue Sep 13 03:30:10 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 03:30:11.050827 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd3bf54b-232d-498c-a8a8-ed43c8cff350 service glance] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:30:11.052822 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-fd3bf54b-232d-498c-a8a8-ed43c8cff350 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:30:11.135854 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:30:11.154385 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd3bf54b-232d-498c-a8a8-ed43c8cff350 service glance] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:30:11.159795 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c46d4caa-9ff8-4fcb-abae-eb74a44200cc service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '893b0c0d85964b7dbc9705c0a091682c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b9d94f6a68f7470d9494f112175620dc', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:30:11.160482 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c46d4caa-9ff8-4fcb-abae-eb74a44200cc service swift] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:30:11.160881 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c46d4caa-9ff8-4fcb-abae-eb74a44200cc service swift] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:30:11.161236 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c46d4caa-9ff8-4fcb-abae-eb74a44200cc service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:30:11.170594 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c46d4caa-9ff8-4fcb-abae-eb74a44200cc service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:30:11.172416 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c46d4caa-9ff8-4fcb-abae-eb74a44200cc service swift] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:30:11.183671 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 1217/2441] 10.222.0.18 () {60 vars in 1384 bytes} [Tue Sep 13 03:30:11 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 133 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 13 03:30:20.396892 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68ec4086-5196-47b1-87b8-0457dfd31005 None None] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:30:20.397383 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68ec4086-5196-47b1-87b8-0457dfd31005 None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:30:20.397543 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68ec4086-5196-47b1-87b8-0457dfd31005 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:30:20.423011 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.auth.core [None req-68ec4086-5196-47b1-87b8-0457dfd31005 None None] MFA Rules not processed for user `92bba932a70f4a75b1bb1810d8826447`. Rule list: `[]` (Enabled: `True`). {{(pid=81399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 03:30:20.424209 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-68ec4086-5196-47b1-87b8-0457dfd31005 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:30:20.431102 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 1225/2442] 10.222.0.18 () {58 vars in 958 bytes} [Tue Sep 13 03:30:20 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 03:30:21.179631 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=6 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:30:22.485776 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efcb6bd1-dd14-444c-bd0b-eb1290a1c57a None None] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:30:22.486330 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efcb6bd1-dd14-444c-bd0b-eb1290a1c57a None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:30:22.486441 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efcb6bd1-dd14-444c-bd0b-eb1290a1c57a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:30:22.506037 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.auth.core [None req-efcb6bd1-dd14-444c-bd0b-eb1290a1c57a None None] MFA Rules not processed for user `4c2d20fb54a54ecc99b47f237be4917c`. Rule list: `[]` (Enabled: `True`). {{(pid=81400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 03:30:22.507333 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-efcb6bd1-dd14-444c-bd0b-eb1290a1c57a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:30:22.514631 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 1218/2443] 10.222.0.18 () {58 vars in 1009 bytes} [Tue Sep 13 03:30:22 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 03:30:22.526918 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68ec4086-5196-47b1-87b8-0457dfd31005 None None] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:30:22.542596 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68ec4086-5196-47b1-87b8-0457dfd31005 None None] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:30:22.549760 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91d697f9-2cce-49b4-ab1a-4169dcb7d410 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '893b0c0d85964b7dbc9705c0a091682c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b9d94f6a68f7470d9494f112175620dc', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:30:22.550460 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d697f9-2cce-49b4-ab1a-4169dcb7d410 service swift] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:30:22.550861 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d697f9-2cce-49b4-ab1a-4169dcb7d410 service swift] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:30:22.551711 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d697f9-2cce-49b4-ab1a-4169dcb7d410 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:30:22.560155 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91d697f9-2cce-49b4-ab1a-4169dcb7d410 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:30:22.561848 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91d697f9-2cce-49b4-ab1a-4169dcb7d410 service swift] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:30:22.570036 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 1226/2444] 10.222.0.18 () {60 vars in 1384 bytes} [Tue Sep 13 03:30:22 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 13 03:30:32.500583 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=3 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:30:32.567434 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=16 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:30:46.119006 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efcb6bd1-dd14-444c-bd0b-eb1290a1c57a None None] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:30:46.178483 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efcb6bd1-dd14-444c-bd0b-eb1290a1c57a None None] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:30:46.182033 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-749bfe06-0d54-4099-afc1-67867c8933f1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdffbc851086441ebd01022d1e4c2609', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b9d94f6a68f7470d9494f112175620dc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:30:46.182660 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-749bfe06-0d54-4099-afc1-67867c8933f1 service neutron] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:30:46.182938 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-749bfe06-0d54-4099-afc1-67867c8933f1 service neutron] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:30:46.183203 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-749bfe06-0d54-4099-afc1-67867c8933f1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:30:46.184976 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-749bfe06-0d54-4099-afc1-67867c8933f1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:30:46.271595 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-749bfe06-0d54-4099-afc1-67867c8933f1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:30:46.273636 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-749bfe06-0d54-4099-afc1-67867c8933f1 service neutron] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:30:46.296360 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 1219/2445] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Sep 13 03:30:46 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 178 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 03:30:48.302754 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91d697f9-2cce-49b4-ab1a-4169dcb7d410 service swift] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:30:48.312011 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91d697f9-2cce-49b4-ab1a-4169dcb7d410 service swift] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:30:48.315583 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dcf9981-1559-4464-b9f0-91d03ebfa184 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdffbc851086441ebd01022d1e4c2609', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b9d94f6a68f7470d9494f112175620dc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:30:48.316169 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dcf9981-1559-4464-b9f0-91d03ebfa184 service neutron] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:30:48.317071 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dcf9981-1559-4464-b9f0-91d03ebfa184 service neutron] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:30:48.317342 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dcf9981-1559-4464-b9f0-91d03ebfa184 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:30:48.319094 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-4dcf9981-1559-4464-b9f0-91d03ebfa184 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:30:48.381873 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dcf9981-1559-4464-b9f0-91d03ebfa184 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:30:48.383614 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dcf9981-1559-4464-b9f0-91d03ebfa184 service neutron] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:30:48.394358 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 1227/2446] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Sep 13 03:30:48 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 03:30:48.431191 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-749bfe06-0d54-4099-afc1-67867c8933f1 service neutron] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:30:48.432632 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-749bfe06-0d54-4099-afc1-67867c8933f1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:30:48.443662 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-749bfe06-0d54-4099-afc1-67867c8933f1 service neutron] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:30:48.447519 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d5ebe93-00fc-4109-9d54-e31e22ee6594 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdffbc851086441ebd01022d1e4c2609', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b9d94f6a68f7470d9494f112175620dc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:30:48.448118 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d5ebe93-00fc-4109-9d54-e31e22ee6594 service neutron] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:30:48.448396 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d5ebe93-00fc-4109-9d54-e31e22ee6594 service neutron] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:30:48.448666 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d5ebe93-00fc-4109-9d54-e31e22ee6594 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:30:48.450328 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-4d5ebe93-00fc-4109-9d54-e31e22ee6594 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:30:48.465997 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d5ebe93-00fc-4109-9d54-e31e22ee6594 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:30:48.467533 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d5ebe93-00fc-4109-9d54-e31e22ee6594 service neutron] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:30:48.479625 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 1220/2447] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Sep 13 03:30:48 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 03:30:52.614847 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dcf9981-1559-4464-b9f0-91d03ebfa184 service neutron] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:30:52.617013 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-4dcf9981-1559-4464-b9f0-91d03ebfa184 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:30:52.716810 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dcf9981-1559-4464-b9f0-91d03ebfa184 service neutron] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:30:52.722530 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67ac6a5f-356a-482c-9f02-820e746d43c9 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b99f1b2374d4d2f8db78306316180d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b9d94f6a68f7470d9494f112175620dc', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:30:52.723419 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67ac6a5f-356a-482c-9f02-820e746d43c9 service placement] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:30:52.723854 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67ac6a5f-356a-482c-9f02-820e746d43c9 service placement] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:30:52.724289 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67ac6a5f-356a-482c-9f02-820e746d43c9 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:30:52.726321 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-67ac6a5f-356a-482c-9f02-820e746d43c9 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:30:52.738912 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67ac6a5f-356a-482c-9f02-820e746d43c9 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:30:52.740599 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67ac6a5f-356a-482c-9f02-820e746d43c9 service placement] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:30:52.751166 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 1228/2448] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Sep 13 03:30:52 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 137 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 03:30:54.620968 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a608a25-0454-4cc1-9afd-7a41b13d0859 None None] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:30:54.621452 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a608a25-0454-4cc1-9afd-7a41b13d0859 None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:30:54.621566 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a608a25-0454-4cc1-9afd-7a41b13d0859 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:30:54.666672 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.auth.core [None req-4a608a25-0454-4cc1-9afd-7a41b13d0859 None None] MFA Rules not processed for user `4f76a7c6fe9347b0bea5f701bccb6150`. Rule list: `[]` (Enabled: `True`). {{(pid=81400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 03:30:54.667932 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-4a608a25-0454-4cc1-9afd-7a41b13d0859 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:30:54.718112 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:30:54.730860 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 1221/2449] 10.222.0.18 () {58 vars in 958 bytes} [Tue Sep 13 03:30:54 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 03:30:56.628647 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67ac6a5f-356a-482c-9f02-820e746d43c9 service placement] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:30:56.630490 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-67ac6a5f-356a-482c-9f02-820e746d43c9 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:30:56.646753 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67ac6a5f-356a-482c-9f02-820e746d43c9 service placement] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:30:56.651228 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1297b567-bfc0-43e2-a67e-68e1b4f28eaa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76c5181fc7be45279ac00017bb4640b3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:30:56.651880 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1297b567-bfc0-43e2-a67e-68e1b4f28eaa admin admin] REQUEST_METHOD: `DELETE` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:30:56.652153 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1297b567-bfc0-43e2-a67e-68e1b4f28eaa admin admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:30:56.652421 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1297b567-bfc0-43e2-a67e-68e1b4f28eaa admin admin] PATH_INFO: `/v3/users/7ad4d86f4c8f450986870292477c1182` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:30:56.653519 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1297b567-bfc0-43e2-a67e-68e1b4f28eaa admin admin] RBAC: Authorizing `identity:delete_user(user_id=7ad4d86f4c8f450986870292477c1182)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:30:56.655636 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1297b567-bfc0-43e2-a67e-68e1b4f28eaa admin admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:30:56.703079 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-1297b567-bfc0-43e2-a67e-68e1b4f28eaa admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7ad4d86f4c8f450986870292477c1182'} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:30:56.710752 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-1297b567-bfc0-43e2-a67e-68e1b4f28eaa admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7ad4d86f4c8f450986870292477c1182'} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:30:56.723554 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-1297b567-bfc0-43e2-a67e-68e1b4f28eaa admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7ad4d86f4c8f450986870292477c1182'} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:30:56.724136 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-1297b567-bfc0-43e2-a67e-68e1b4f28eaa admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7ad4d86f4c8f450986870292477c1182'} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:30:56.729384 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:30:56.731409 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-1297b567-bfc0-43e2-a67e-68e1b4f28eaa admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7ad4d86f4c8f450986870292477c1182'} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:30:56.738713 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 1229/2450] 10.222.0.18 () {58 vars in 1224 bytes} [Tue Sep 13 03:30:56 2022] DELETE /identity/v3/users/7ad4d86f4c8f450986870292477c1182 => generated 0 bytes in 111 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 03:30:56.986272 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a608a25-0454-4cc1-9afd-7a41b13d0859 None None] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:30:56.997485 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a608a25-0454-4cc1-9afd-7a41b13d0859 None None] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:30:57.089863 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a078ae2-d9d8-4dc9-bd84-b3bd845f4c44 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76c5181fc7be45279ac00017bb4640b3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:30:57.090500 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a078ae2-d9d8-4dc9-bd84-b3bd845f4c44 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:30:57.090814 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a078ae2-d9d8-4dc9-bd84-b3bd845f4c44 admin admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:30:57.091091 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a078ae2-d9d8-4dc9-bd84-b3bd845f4c44 admin admin] PATH_INFO: `/v3/projects/fb8fff2dfb49442492296d4c1701abb1` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:30:57.092233 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a078ae2-d9d8-4dc9-bd84-b3bd845f4c44 admin admin] RBAC: Authorizing `identity:delete_project(project_id=fb8fff2dfb49442492296d4c1701abb1)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:30:57.094151 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a078ae2-d9d8-4dc9-bd84-b3bd845f4c44 admin admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:30:57.153404 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-0a078ae2-d9d8-4dc9-bd84-b3bd845f4c44 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fb8fff2dfb49442492296d4c1701abb1', 'request_id': 'req-0a078ae2-d9d8-4dc9-bd84-b3bd845f4c44'} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:30:57.161602 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-0a078ae2-d9d8-4dc9-bd84-b3bd845f4c44 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fb8fff2dfb49442492296d4c1701abb1', 'request_id': 'req-0a078ae2-d9d8-4dc9-bd84-b3bd845f4c44'} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:30:57.168902 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-0a078ae2-d9d8-4dc9-bd84-b3bd845f4c44 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fb8fff2dfb49442492296d4c1701abb1', 'request_id': 'req-0a078ae2-d9d8-4dc9-bd84-b3bd845f4c44'} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:30:57.182363 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-0a078ae2-d9d8-4dc9-bd84-b3bd845f4c44 admin admin] The token cache is being invalidate because project fb8fff2dfb49442492296d4c1701abb1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81400) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 03:30:57.182672 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-0a078ae2-d9d8-4dc9-bd84-b3bd845f4c44 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:30:57.183287 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-0a078ae2-d9d8-4dc9-bd84-b3bd845f4c44 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:30:57.184785 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 1222/2451] 10.222.0.18 () {58 vars in 1233 bytes} [Tue Sep 13 03:30:56 2022] DELETE /identity/v3/projects/fb8fff2dfb49442492296d4c1701abb1 => generated 0 bytes in 199 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 03:30:57.188789 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1297b567-bfc0-43e2-a67e-68e1b4f28eaa admin admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:30:57.190206 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-1297b567-bfc0-43e2-a67e-68e1b4f28eaa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:30:57.205060 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1297b567-bfc0-43e2-a67e-68e1b4f28eaa admin admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:30:57.264707 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c44ddcd7-b6ef-45ac-adeb-fd76f1466cf6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76c5181fc7be45279ac00017bb4640b3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:30:57.265344 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c44ddcd7-b6ef-45ac-adeb-fd76f1466cf6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:30:57.265617 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c44ddcd7-b6ef-45ac-adeb-fd76f1466cf6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:30:57.265885 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c44ddcd7-b6ef-45ac-adeb-fd76f1466cf6 admin admin] PATH_INFO: `/v3/users/ab6d438d4909481191953b3185d3f0ce` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:30:57.267026 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c44ddcd7-b6ef-45ac-adeb-fd76f1466cf6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ab6d438d4909481191953b3185d3f0ce)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:30:57.269019 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c44ddcd7-b6ef-45ac-adeb-fd76f1466cf6 admin admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:30:57.331502 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-c44ddcd7-b6ef-45ac-adeb-fd76f1466cf6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ab6d438d4909481191953b3185d3f0ce'} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:30:57.339577 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-c44ddcd7-b6ef-45ac-adeb-fd76f1466cf6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ab6d438d4909481191953b3185d3f0ce'} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:30:57.353528 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-c44ddcd7-b6ef-45ac-adeb-fd76f1466cf6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ab6d438d4909481191953b3185d3f0ce'} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:30:57.354226 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-c44ddcd7-b6ef-45ac-adeb-fd76f1466cf6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ab6d438d4909481191953b3185d3f0ce'} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:30:57.361097 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-c44ddcd7-b6ef-45ac-adeb-fd76f1466cf6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ab6d438d4909481191953b3185d3f0ce'} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:30:57.367521 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 1230/2452] 10.222.0.18 () {58 vars in 1224 bytes} [Tue Sep 13 03:30:57 2022] DELETE /identity/v3/users/ab6d438d4909481191953b3185d3f0ce => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 03:30:57.536447 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a078ae2-d9d8-4dc9-bd84-b3bd845f4c44 admin admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:30:57.538069 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-0a078ae2-d9d8-4dc9-bd84-b3bd845f4c44 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:30:57.553478 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a078ae2-d9d8-4dc9-bd84-b3bd845f4c44 admin admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:30:57.608834 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ccfcdcd-6a90-4f9e-8582-d7914774af07 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76c5181fc7be45279ac00017bb4640b3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:30:57.609458 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ccfcdcd-6a90-4f9e-8582-d7914774af07 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:30:57.609733 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ccfcdcd-6a90-4f9e-8582-d7914774af07 admin admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:30:57.610002 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ccfcdcd-6a90-4f9e-8582-d7914774af07 admin admin] PATH_INFO: `/v3/projects/c8bc9301cec443a2b98ec7e663ba6633` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:30:57.611127 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ccfcdcd-6a90-4f9e-8582-d7914774af07 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c8bc9301cec443a2b98ec7e663ba6633)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:30:57.613509 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ccfcdcd-6a90-4f9e-8582-d7914774af07 admin admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:30:57.647996 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:30:57.679016 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-3ccfcdcd-6a90-4f9e-8582-d7914774af07 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c8bc9301cec443a2b98ec7e663ba6633', 'request_id': 'req-3ccfcdcd-6a90-4f9e-8582-d7914774af07'} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:30:57.684863 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-3ccfcdcd-6a90-4f9e-8582-d7914774af07 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c8bc9301cec443a2b98ec7e663ba6633', 'request_id': 'req-3ccfcdcd-6a90-4f9e-8582-d7914774af07'} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:30:57.692847 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-3ccfcdcd-6a90-4f9e-8582-d7914774af07 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c8bc9301cec443a2b98ec7e663ba6633', 'request_id': 'req-3ccfcdcd-6a90-4f9e-8582-d7914774af07'} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:30:57.708388 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-3ccfcdcd-6a90-4f9e-8582-d7914774af07 admin admin] The token cache is being invalidate because project c8bc9301cec443a2b98ec7e663ba6633 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81400) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 03:30:57.708674 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-3ccfcdcd-6a90-4f9e-8582-d7914774af07 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:30:57.709314 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-3ccfcdcd-6a90-4f9e-8582-d7914774af07 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:30:57.710838 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 1223/2453] 10.222.0.18 () {58 vars in 1233 bytes} [Tue Sep 13 03:30:57 2022] DELETE /identity/v3/projects/c8bc9301cec443a2b98ec7e663ba6633 => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 03:30:58.079043 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c44ddcd7-b6ef-45ac-adeb-fd76f1466cf6 admin admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:30:58.081016 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-c44ddcd7-b6ef-45ac-adeb-fd76f1466cf6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:30:58.085603 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ccfcdcd-6a90-4f9e-8582-d7914774af07 admin admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:30:58.087586 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-3ccfcdcd-6a90-4f9e-8582-d7914774af07 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:30:58.149572 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ccfcdcd-6a90-4f9e-8582-d7914774af07 admin admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:30:58.155417 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7923385-84c9-40a1-84ae-a151d65a8b3d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdffbc851086441ebd01022d1e4c2609', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b9d94f6a68f7470d9494f112175620dc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:30:58.156042 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7923385-84c9-40a1-84ae-a151d65a8b3d service neutron] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:30:58.156328 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7923385-84c9-40a1-84ae-a151d65a8b3d service neutron] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:30:58.156600 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7923385-84c9-40a1-84ae-a151d65a8b3d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:30:58.159277 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-e7923385-84c9-40a1-84ae-a151d65a8b3d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:30:58.166847 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c44ddcd7-b6ef-45ac-adeb-fd76f1466cf6 admin admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:30:58.175924 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:30:58.236750 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd2793fe-b9c9-4c3c-a621-36dafd061d5b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7377f5bee4a49959182e1bd34bc97a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b9d94f6a68f7470d9494f112175620dc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:30:58.237456 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd2793fe-b9c9-4c3c-a621-36dafd061d5b service nova] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:30:58.237744 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd2793fe-b9c9-4c3c-a621-36dafd061d5b service nova] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:30:58.238039 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd2793fe-b9c9-4c3c-a621-36dafd061d5b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:30:58.239825 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-dd2793fe-b9c9-4c3c-a621-36dafd061d5b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:30:58.241544 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7923385-84c9-40a1-84ae-a151d65a8b3d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:30:58.243123 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7923385-84c9-40a1-84ae-a151d65a8b3d service neutron] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:30:58.253398 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 1224/2454] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Sep 13 03:30:58 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 168 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 13 03:30:58.336053 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd2793fe-b9c9-4c3c-a621-36dafd061d5b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:30:58.337703 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd2793fe-b9c9-4c3c-a621-36dafd061d5b service nova] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:30:58.358334 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 1231/2455] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Sep 13 03:30:58 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 280 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 03:31:08.248812 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:DELETE=6,keystone:SELECT=64,keystone:INSERT=1 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:31:08.355610 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=63 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:31:11.160309 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2427d25f-c449-458a-b4cd-1fe967ab74da None None] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:31:11.160815 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2427d25f-c449-458a-b4cd-1fe967ab74da None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:31:11.160998 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2427d25f-c449-458a-b4cd-1fe967ab74da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:31:11.182196 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.auth.core [None req-2427d25f-c449-458a-b4cd-1fe967ab74da None None] MFA Rules not processed for user `4c2d20fb54a54ecc99b47f237be4917c`. Rule list: `[]` (Enabled: `True`). {{(pid=81400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 03:31:11.183479 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-2427d25f-c449-458a-b4cd-1fe967ab74da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:31:11.260201 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 1225/2456] 10.222.0.18 () {58 vars in 1009 bytes} [Tue Sep 13 03:31:11 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 03:31:11.285465 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd2793fe-b9c9-4c3c-a621-36dafd061d5b service nova] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:31:11.288069 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-dd2793fe-b9c9-4c3c-a621-36dafd061d5b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:31:11.364210 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd2793fe-b9c9-4c3c-a621-36dafd061d5b service nova] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:31:11.372643 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20f4901e-eadd-4d19-a219-77d6c12e2954 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '893b0c0d85964b7dbc9705c0a091682c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b9d94f6a68f7470d9494f112175620dc', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:31:11.373552 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f4901e-eadd-4d19-a219-77d6c12e2954 service swift] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:31:11.374059 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f4901e-eadd-4d19-a219-77d6c12e2954 service swift] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:31:11.374582 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f4901e-eadd-4d19-a219-77d6c12e2954 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:31:11.384835 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20f4901e-eadd-4d19-a219-77d6c12e2954 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:31:11.386449 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20f4901e-eadd-4d19-a219-77d6c12e2954 service swift] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:31:11.396449 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 1232/2457] 10.222.0.18 () {60 vars in 1384 bytes} [Tue Sep 13 03:31:11 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 13 03:31:15.172834 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2427d25f-c449-458a-b4cd-1fe967ab74da None None] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:31:15.174329 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-2427d25f-c449-458a-b4cd-1fe967ab74da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:31:15.191728 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2427d25f-c449-458a-b4cd-1fe967ab74da None None] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:31:15.197103 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-641f27b8-91cf-41ee-a05e-24248c435267 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7377f5bee4a49959182e1bd34bc97a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b9d94f6a68f7470d9494f112175620dc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:31:15.197751 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-641f27b8-91cf-41ee-a05e-24248c435267 service nova] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:31:15.198222 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-641f27b8-91cf-41ee-a05e-24248c435267 service nova] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:31:15.198490 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-641f27b8-91cf-41ee-a05e-24248c435267 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:31:15.200845 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-641f27b8-91cf-41ee-a05e-24248c435267 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:31:15.287904 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-641f27b8-91cf-41ee-a05e-24248c435267 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:31:15.290118 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-641f27b8-91cf-41ee-a05e-24248c435267 service nova] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:31:15.313083 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 1226/2458] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Sep 13 03:31:15 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 141 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 03:31:21.392802 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=32 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:31:25.307951 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=66 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:31:28.907839 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20f4901e-eadd-4d19-a219-77d6c12e2954 service swift] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:31:28.909283 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-20f4901e-eadd-4d19-a219-77d6c12e2954 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:31:28.977081 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20f4901e-eadd-4d19-a219-77d6c12e2954 service swift] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:31:28.981889 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34d17ab4-de99-43dd-893a-66eaaa7cda55 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b99f1b2374d4d2f8db78306316180d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b9d94f6a68f7470d9494f112175620dc', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:31:28.982477 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d17ab4-de99-43dd-893a-66eaaa7cda55 service placement] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:31:28.982781 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d17ab4-de99-43dd-893a-66eaaa7cda55 service placement] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:31:28.983048 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d17ab4-de99-43dd-893a-66eaaa7cda55 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:31:28.984699 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-34d17ab4-de99-43dd-893a-66eaaa7cda55 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:31:28.996178 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34d17ab4-de99-43dd-893a-66eaaa7cda55 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:31:28.997762 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34d17ab4-de99-43dd-893a-66eaaa7cda55 service placement] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:31:29.008008 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 1233/2459] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Sep 13 03:31:28 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 03:31:30.818872 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8375de3e-de22-4609-bd90-1ea0b2e7df4f None None] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:31:30.819322 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8375de3e-de22-4609-bd90-1ea0b2e7df4f None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:31:30.819428 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8375de3e-de22-4609-bd90-1ea0b2e7df4f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:31:30.868959 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.auth.core [None req-8375de3e-de22-4609-bd90-1ea0b2e7df4f None None] MFA Rules not processed for user `4c2d20fb54a54ecc99b47f237be4917c`. Rule list: `[]` (Enabled: `True`). {{(pid=81400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 03:31:30.870159 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-8375de3e-de22-4609-bd90-1ea0b2e7df4f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:31:30.886407 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 1227/2460] 10.222.0.18 () {58 vars in 1009 bytes} [Tue Sep 13 03:31:30 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 69 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 03:31:30.898275 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34d17ab4-de99-43dd-893a-66eaaa7cda55 service placement] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:31:30.918900 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34d17ab4-de99-43dd-893a-66eaaa7cda55 service placement] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:31:30.923864 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24ea7885-fd57-4638-8997-d7e101f9cbcd service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '893b0c0d85964b7dbc9705c0a091682c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b9d94f6a68f7470d9494f112175620dc', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:31:30.924450 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ea7885-fd57-4638-8997-d7e101f9cbcd service swift] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:31:30.924725 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ea7885-fd57-4638-8997-d7e101f9cbcd service swift] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:31:30.924991 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ea7885-fd57-4638-8997-d7e101f9cbcd service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:31:30.933385 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24ea7885-fd57-4638-8997-d7e101f9cbcd service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:31:30.935011 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24ea7885-fd57-4638-8997-d7e101f9cbcd service swift] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:31:30.942885 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 1234/2461] 10.222.0.18 () {60 vars in 1384 bytes} [Tue Sep 13 03:31:30 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 13 03:31:40.883549 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=18 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:31:40.939611 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=42 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:31:50.311991 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8375de3e-de22-4609-bd90-1ea0b2e7df4f None None] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:31:50.313391 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-8375de3e-de22-4609-bd90-1ea0b2e7df4f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:31:50.325171 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8375de3e-de22-4609-bd90-1ea0b2e7df4f None None] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:31:50.328773 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeeaff4e-cd7e-44e4-83b8-3d63799853e9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdffbc851086441ebd01022d1e4c2609', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b9d94f6a68f7470d9494f112175620dc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:31:50.329321 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeeaff4e-cd7e-44e4-83b8-3d63799853e9 service neutron] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:31:50.329595 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeeaff4e-cd7e-44e4-83b8-3d63799853e9 service neutron] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:31:50.329859 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeeaff4e-cd7e-44e4-83b8-3d63799853e9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:31:50.331702 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-aeeaff4e-cd7e-44e4-83b8-3d63799853e9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:31:50.341757 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeeaff4e-cd7e-44e4-83b8-3d63799853e9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:31:50.343211 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeeaff4e-cd7e-44e4-83b8-3d63799853e9 service neutron] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:31:50.351270 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 1228/2462] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Sep 13 03:31:50 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 03:31:52.509242 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a59833b9-9a26-439d-a812-c8a397a3f7bb None None] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:31:52.509724 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a59833b9-9a26-439d-a812-c8a397a3f7bb None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:31:52.509849 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a59833b9-9a26-439d-a812-c8a397a3f7bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:31:52.528900 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.auth.core [None req-a59833b9-9a26-439d-a812-c8a397a3f7bb None None] MFA Rules not processed for user `4c2d20fb54a54ecc99b47f237be4917c`. Rule list: `[]` (Enabled: `True`). {{(pid=81399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 03:31:52.530097 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-a59833b9-9a26-439d-a812-c8a397a3f7bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:31:52.537823 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 1235/2463] 10.222.0.18 () {58 vars in 967 bytes} [Tue Sep 13 03:31:52 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 03:31:52.550694 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeeaff4e-cd7e-44e4-83b8-3d63799853e9 service neutron] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:31:52.563085 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeeaff4e-cd7e-44e4-83b8-3d63799853e9 service neutron] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:31:52.568014 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53e2f357-eb8d-44b5-ab55-4604cc4ac9fc service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '893b0c0d85964b7dbc9705c0a091682c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b9d94f6a68f7470d9494f112175620dc', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:31:52.568661 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e2f357-eb8d-44b5-ab55-4604cc4ac9fc service swift] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:31:52.568938 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e2f357-eb8d-44b5-ab55-4604cc4ac9fc service swift] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:31:52.569207 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e2f357-eb8d-44b5-ab55-4604cc4ac9fc service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:31:52.577477 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53e2f357-eb8d-44b5-ab55-4604cc4ac9fc service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:31:52.579095 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53e2f357-eb8d-44b5-ab55-4604cc4ac9fc service swift] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:31:52.587722 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 1229/2464] 10.222.0.18 () {60 vars in 1384 bytes} [Tue Sep 13 03:31:52 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 13 03:31:56.440658 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a59833b9-9a26-439d-a812-c8a397a3f7bb None None] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:31:56.454693 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a59833b9-9a26-439d-a812-c8a397a3f7bb None None] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:31:56.460896 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7d9a683-0221-4a23-b2db-a72976b64a36 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7377f5bee4a49959182e1bd34bc97a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b9d94f6a68f7470d9494f112175620dc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:31:56.461741 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7d9a683-0221-4a23-b2db-a72976b64a36 service nova] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:31:56.462283 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7d9a683-0221-4a23-b2db-a72976b64a36 service nova] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:31:56.462472 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7d9a683-0221-4a23-b2db-a72976b64a36 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:31:56.464896 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-d7d9a683-0221-4a23-b2db-a72976b64a36 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:31:56.479223 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7d9a683-0221-4a23-b2db-a72976b64a36 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:31:56.481079 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7d9a683-0221-4a23-b2db-a72976b64a36 service nova] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:31:56.493159 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 1236/2465] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Sep 13 03:31:56 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 53 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 03:32:02.091858 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53e2f357-eb8d-44b5-ab55-4604cc4ac9fc service swift] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:32:02.101834 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53e2f357-eb8d-44b5-ab55-4604cc4ac9fc service swift] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:32:02.105358 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b54a3573-8d15-4d2c-9ccf-2092f440cfee service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdffbc851086441ebd01022d1e4c2609', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b9d94f6a68f7470d9494f112175620dc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:32:02.105940 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b54a3573-8d15-4d2c-9ccf-2092f440cfee service neutron] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:32:02.106214 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b54a3573-8d15-4d2c-9ccf-2092f440cfee service neutron] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:32:02.106479 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b54a3573-8d15-4d2c-9ccf-2092f440cfee service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:32:02.108214 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-b54a3573-8d15-4d2c-9ccf-2092f440cfee service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:32:02.171431 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b54a3573-8d15-4d2c-9ccf-2092f440cfee service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:32:02.172860 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b54a3573-8d15-4d2c-9ccf-2092f440cfee service neutron] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:32:02.182511 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 1230/2466] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Sep 13 03:32:02 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 03:32:05.292178 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7d9a683-0221-4a23-b2db-a72976b64a36 service nova] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:32:05.293829 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-d7d9a683-0221-4a23-b2db-a72976b64a36 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:32:05.315880 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7d9a683-0221-4a23-b2db-a72976b64a36 service nova] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:32:05.321904 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-142a2e0f-e883-46f6-ac53-7ed1fd873e7a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76c5181fc7be45279ac00017bb4640b3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:32:05.322546 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-142a2e0f-e883-46f6-ac53-7ed1fd873e7a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:32:05.322879 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-142a2e0f-e883-46f6-ac53-7ed1fd873e7a admin admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:32:05.323154 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-142a2e0f-e883-46f6-ac53-7ed1fd873e7a admin admin] PATH_INFO: `/v3/users/80f6f9a952d84ccab0de028d58aa6914` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:32:05.324315 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-142a2e0f-e883-46f6-ac53-7ed1fd873e7a admin admin] RBAC: Authorizing `identity:delete_user(user_id=80f6f9a952d84ccab0de028d58aa6914)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:32:05.326467 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-142a2e0f-e883-46f6-ac53-7ed1fd873e7a admin admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:32:05.381505 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-142a2e0f-e883-46f6-ac53-7ed1fd873e7a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '80f6f9a952d84ccab0de028d58aa6914'} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:32:05.391629 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-142a2e0f-e883-46f6-ac53-7ed1fd873e7a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '80f6f9a952d84ccab0de028d58aa6914'} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:32:05.406002 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-142a2e0f-e883-46f6-ac53-7ed1fd873e7a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '80f6f9a952d84ccab0de028d58aa6914'} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:32:05.407229 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-142a2e0f-e883-46f6-ac53-7ed1fd873e7a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '80f6f9a952d84ccab0de028d58aa6914'} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:32:05.415357 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-142a2e0f-e883-46f6-ac53-7ed1fd873e7a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '80f6f9a952d84ccab0de028d58aa6914'} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:32:05.426191 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 1237/2467] 10.222.0.18 () {58 vars in 1224 bytes} [Tue Sep 13 03:32:05 2022] DELETE /identity/v3/users/80f6f9a952d84ccab0de028d58aa6914 => generated 0 bytes in 131 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 03:32:05.599149 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b54a3573-8d15-4d2c-9ccf-2092f440cfee service neutron] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:32:05.611768 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b54a3573-8d15-4d2c-9ccf-2092f440cfee service neutron] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:32:05.671810 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cf900ec-2f20-40e5-bd6e-8d469edc024d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76c5181fc7be45279ac00017bb4640b3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:32:05.672402 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cf900ec-2f20-40e5-bd6e-8d469edc024d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:32:05.672675 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cf900ec-2f20-40e5-bd6e-8d469edc024d admin admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:32:05.672942 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cf900ec-2f20-40e5-bd6e-8d469edc024d admin admin] PATH_INFO: `/v3/projects/bb8f67ac0f3c48b2933b6ae34d4fd838` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:32:05.674062 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cf900ec-2f20-40e5-bd6e-8d469edc024d admin admin] RBAC: Authorizing `identity:delete_project(project_id=bb8f67ac0f3c48b2933b6ae34d4fd838)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:32:05.675976 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cf900ec-2f20-40e5-bd6e-8d469edc024d admin admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:32:05.723589 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-1cf900ec-2f20-40e5-bd6e-8d469edc024d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bb8f67ac0f3c48b2933b6ae34d4fd838', 'request_id': 'req-1cf900ec-2f20-40e5-bd6e-8d469edc024d'} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:32:05.729755 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-1cf900ec-2f20-40e5-bd6e-8d469edc024d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bb8f67ac0f3c48b2933b6ae34d4fd838', 'request_id': 'req-1cf900ec-2f20-40e5-bd6e-8d469edc024d'} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:32:05.736556 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-1cf900ec-2f20-40e5-bd6e-8d469edc024d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bb8f67ac0f3c48b2933b6ae34d4fd838', 'request_id': 'req-1cf900ec-2f20-40e5-bd6e-8d469edc024d'} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:32:05.746495 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:32:05.751589 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-1cf900ec-2f20-40e5-bd6e-8d469edc024d admin admin] The token cache is being invalidate because project bb8f67ac0f3c48b2933b6ae34d4fd838 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81400) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 03:32:05.752313 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-1cf900ec-2f20-40e5-bd6e-8d469edc024d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:32:05.753226 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-1cf900ec-2f20-40e5-bd6e-8d469edc024d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:32:05.755210 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 1231/2468] 10.222.0.18 () {58 vars in 1233 bytes} [Tue Sep 13 03:32:05 2022] DELETE /identity/v3/projects/bb8f67ac0f3c48b2933b6ae34d4fd838 => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 03:32:05.760095 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c69207a0-5abf-4007-aa71-dc9328b2acc7 None None] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:32:05.760370 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c69207a0-5abf-4007-aa71-dc9328b2acc7 None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:32:05.760982 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c69207a0-5abf-4007-aa71-dc9328b2acc7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:32:05.780339 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.auth.core [None req-c69207a0-5abf-4007-aa71-dc9328b2acc7 None None] MFA Rules not processed for user `49f280ab086b4610855c4ffa378517d9`. Rule list: `[]` (Enabled: `True`). {{(pid=81399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 03:32:05.781533 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-c69207a0-5abf-4007-aa71-dc9328b2acc7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:32:05.842693 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 1238/2469] 10.222.0.18 () {58 vars in 958 bytes} [Tue Sep 13 03:32:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 03:32:05.859918 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-960f880a-979a-42fb-94f6-70c76ffb8711 None None] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:32:05.860232 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-960f880a-979a-42fb-94f6-70c76ffb8711 None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:32:05.860459 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-960f880a-979a-42fb-94f6-70c76ffb8711 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:32:05.879826 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.auth.core [None req-960f880a-979a-42fb-94f6-70c76ffb8711 None None] MFA Rules not processed for user `49f280ab086b4610855c4ffa378517d9`. Rule list: `[]` (Enabled: `True`). {{(pid=81400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 03:32:05.881026 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-960f880a-979a-42fb-94f6-70c76ffb8711 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:32:05.889007 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 1232/2470] 10.222.0.18 () {58 vars in 958 bytes} [Tue Sep 13 03:32:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 03:32:05.893328 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c69207a0-5abf-4007-aa71-dc9328b2acc7 None None] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:32:05.903478 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c69207a0-5abf-4007-aa71-dc9328b2acc7 None None] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:32:05.908081 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60429b14-f454-427f-9128-13ed320bd299 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76c5181fc7be45279ac00017bb4640b3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:32:05.908673 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60429b14-f454-427f-9128-13ed320bd299 admin admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:32:05.908950 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60429b14-f454-427f-9128-13ed320bd299 admin admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:32:05.909220 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60429b14-f454-427f-9128-13ed320bd299 admin admin] PATH_INFO: `/v3/domains` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:32:05.909852 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60429b14-f454-427f-9128-13ed320bd299 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 03:32:05.910151 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60429b14-f454-427f-9128-13ed320bd299 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:32:05.911766 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60429b14-f454-427f-9128-13ed320bd299 admin admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:32:05.924226 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 1239/2471] 10.222.0.18 () {58 vars in 1153 bytes} [Tue Sep 13 03:32:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 03:32:05.928344 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-960f880a-979a-42fb-94f6-70c76ffb8711 None None] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:32:05.937569 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-960f880a-979a-42fb-94f6-70c76ffb8711 None None] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:32:05.942234 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11667117-d5c4-447f-be6c-684a0d7d52b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76c5181fc7be45279ac00017bb4640b3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:32:05.942856 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11667117-d5c4-447f-be6c-684a0d7d52b7 admin admin] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:32:05.943132 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11667117-d5c4-447f-be6c-684a0d7d52b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:32:05.943404 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11667117-d5c4-447f-be6c-684a0d7d52b7 admin admin] PATH_INFO: `/v3/projects` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:32:05.944033 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11667117-d5c4-447f-be6c-684a0d7d52b7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:32:05.945924 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11667117-d5c4-447f-be6c-684a0d7d52b7 admin admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:32:05.982432 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 1233/2472] 10.222.0.18 () {60 vars in 1153 bytes} [Tue Sep 13 03:32:05 2022] POST /identity/v3/projects => generated 367 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 03:32:05.986084 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60429b14-f454-427f-9128-13ed320bd299 admin admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:32:05.995919 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60429b14-f454-427f-9128-13ed320bd299 admin admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:32:06.032572 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:32:06.052388 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b05902a3-35bd-4f0c-b16f-3e14177946cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76c5181fc7be45279ac00017bb4640b3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:32:06.052995 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b05902a3-35bd-4f0c-b16f-3e14177946cb admin admin] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:32:06.053275 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b05902a3-35bd-4f0c-b16f-3e14177946cb admin admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:32:06.053548 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b05902a3-35bd-4f0c-b16f-3e14177946cb admin admin] PATH_INFO: `/v3/users` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:32:06.054184 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b05902a3-35bd-4f0c-b16f-3e14177946cb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:32:06.056180 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b05902a3-35bd-4f0c-b16f-3e14177946cb admin admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:32:06.080845 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 1240/2473] 10.222.0.18 () {60 vars in 1144 bytes} [Tue Sep 13 03:32:05 2022] POST /identity/v3/users => generated 284 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 03:32:06.084934 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11667117-d5c4-447f-be6c-684a0d7d52b7 admin admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:32:06.095618 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11667117-d5c4-447f-be6c-684a0d7d52b7 admin admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:32:06.100123 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7416631f-b1fc-4bf8-b53e-84c509d5a713 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76c5181fc7be45279ac00017bb4640b3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:32:06.100760 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7416631f-b1fc-4bf8-b53e-84c509d5a713 admin admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:32:06.101544 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7416631f-b1fc-4bf8-b53e-84c509d5a713 admin admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:32:06.101816 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7416631f-b1fc-4bf8-b53e-84c509d5a713 admin admin] PATH_INFO: `/v3/roles` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:32:06.102450 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7416631f-b1fc-4bf8-b53e-84c509d5a713 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:32:06.104132 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7416631f-b1fc-4bf8-b53e-84c509d5a713 admin admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:32:06.114137 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 1234/2474] 10.222.0.18 () {58 vars in 1122 bytes} [Tue Sep 13 03:32:06 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 03:32:06.117992 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b05902a3-35bd-4f0c-b16f-3e14177946cb admin admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:32:06.127760 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b05902a3-35bd-4f0c-b16f-3e14177946cb admin admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:32:06.132312 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1366a73c-d4ba-4a82-82eb-888d1ea3229b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76c5181fc7be45279ac00017bb4640b3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:32:06.132896 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1366a73c-d4ba-4a82-82eb-888d1ea3229b admin admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:32:06.133172 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1366a73c-d4ba-4a82-82eb-888d1ea3229b admin admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:32:06.133445 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1366a73c-d4ba-4a82-82eb-888d1ea3229b admin admin] PATH_INFO: `/v3/roles` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:32:06.134075 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1366a73c-d4ba-4a82-82eb-888d1ea3229b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:32:06.135719 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1366a73c-d4ba-4a82-82eb-888d1ea3229b admin admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:32:06.146068 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 1241/2475] 10.222.0.18 () {58 vars in 1122 bytes} [Tue Sep 13 03:32:06 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 13 03:32:06.150528 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7416631f-b1fc-4bf8-b53e-84c509d5a713 admin admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:32:06.160690 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7416631f-b1fc-4bf8-b53e-84c509d5a713 admin admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:32:06.166140 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12c98644-ae38-462f-895e-f122ab74f917 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76c5181fc7be45279ac00017bb4640b3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:32:06.166813 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c98644-ae38-462f-895e-f122ab74f917 admin admin] REQUEST_METHOD: `PUT` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:32:06.167089 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c98644-ae38-462f-895e-f122ab74f917 admin admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:32:06.167357 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c98644-ae38-462f-895e-f122ab74f917 admin admin] PATH_INFO: `/v3/projects/7071167bde9e45c08f8a0ddac6638b72/users/fe9405be1b824fbbb5a83e5d778b4218/roles/1eb7a5b5203945c29e6d22d653f47847` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:32:06.182140 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12c98644-ae38-462f-895e-f122ab74f917 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7071167bde9e45c08f8a0ddac6638b72, user_id=fe9405be1b824fbbb5a83e5d778b4218, role_id=1eb7a5b5203945c29e6d22d653f47847)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:32:06.184838 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12c98644-ae38-462f-895e-f122ab74f917 admin admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:32:06.197692 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 1235/2476] 10.222.0.18 () {60 vars in 1483 bytes} [Tue Sep 13 03:32:06 2022] PUT /identity/v3/projects/7071167bde9e45c08f8a0ddac6638b72/users/fe9405be1b824fbbb5a83e5d778b4218/roles/1eb7a5b5203945c29e6d22d653f47847 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 03:32:06.207871 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddaf2e61-b81a-4e64-be79-f8b97debf547 None None] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:32:06.208157 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddaf2e61-b81a-4e64-be79-f8b97debf547 None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:32:06.208428 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddaf2e61-b81a-4e64-be79-f8b97debf547 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:32:06.235781 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.auth.core [None req-ddaf2e61-b81a-4e64-be79-f8b97debf547 None None] MFA Rules not processed for user `fe9405be1b824fbbb5a83e5d778b4218`. Rule list: `[]` (Enabled: `True`). {{(pid=81399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 03:32:06.236977 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-ddaf2e61-b81a-4e64-be79-f8b97debf547 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:32:06.299872 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 1242/2477] 10.222.0.18 () {58 vars in 958 bytes} [Tue Sep 13 03:32:06 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 03:32:06.311113 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12c98644-ae38-462f-895e-f122ab74f917 admin admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:32:06.312817 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-12c98644-ae38-462f-895e-f122ab74f917 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:32:06.369997 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12c98644-ae38-462f-895e-f122ab74f917 admin admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:32:06.373574 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf194127-90dc-4bae-b697-7f23091ab034 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '924ee3f608d34a18b0ddc4695f102a56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b9d94f6a68f7470d9494f112175620dc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:32:06.374174 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf194127-90dc-4bae-b697-7f23091ab034 service cinder] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:32:06.374448 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf194127-90dc-4bae-b697-7f23091ab034 service cinder] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:32:06.374750 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf194127-90dc-4bae-b697-7f23091ab034 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:32:06.383417 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf194127-90dc-4bae-b697-7f23091ab034 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:32:06.385424 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf194127-90dc-4bae-b697-7f23091ab034 service cinder] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:32:06.394111 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 1236/2478] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Sep 13 03:32:06 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 03:32:14.711840 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddaf2e61-b81a-4e64-be79-f8b97debf547 None None] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:32:14.775419 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddaf2e61-b81a-4e64-be79-f8b97debf547 None None] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:32:14.780122 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1c7b006-94b8-4550-bc48-462e1d4547be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76c5181fc7be45279ac00017bb4640b3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:32:14.780763 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1c7b006-94b8-4550-bc48-462e1d4547be admin admin] REQUEST_METHOD: `DELETE` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:32:14.781046 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1c7b006-94b8-4550-bc48-462e1d4547be admin admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:32:14.781320 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1c7b006-94b8-4550-bc48-462e1d4547be admin admin] PATH_INFO: `/v3/users/fe9405be1b824fbbb5a83e5d778b4218` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:32:14.782454 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1c7b006-94b8-4550-bc48-462e1d4547be admin admin] RBAC: Authorizing `identity:delete_user(user_id=fe9405be1b824fbbb5a83e5d778b4218)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:32:14.784583 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1c7b006-94b8-4550-bc48-462e1d4547be admin admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:32:14.832349 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-a1c7b006-94b8-4550-bc48-462e1d4547be admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fe9405be1b824fbbb5a83e5d778b4218'} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:32:14.835335 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=89,keystone:INSERT=4,keystone:DELETE=7 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:32:14.840129 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-a1c7b006-94b8-4550-bc48-462e1d4547be admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fe9405be1b824fbbb5a83e5d778b4218'} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:32:14.854251 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-a1c7b006-94b8-4550-bc48-462e1d4547be admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fe9405be1b824fbbb5a83e5d778b4218'} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:32:14.854872 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-a1c7b006-94b8-4550-bc48-462e1d4547be admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fe9405be1b824fbbb5a83e5d778b4218'} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:32:14.861623 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-a1c7b006-94b8-4550-bc48-462e1d4547be admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fe9405be1b824fbbb5a83e5d778b4218'} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:32:14.867401 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 1243/2479] 10.222.0.18 () {58 vars in 1224 bytes} [Tue Sep 13 03:32:14 2022] DELETE /identity/v3/users/fe9405be1b824fbbb5a83e5d778b4218 => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 03:32:14.871970 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-069c320f-b649-469b-9046-da0addae1253 None None] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:32:14.872424 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-069c320f-b649-469b-9046-da0addae1253 None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:32:14.872537 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-069c320f-b649-469b-9046-da0addae1253 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:32:14.890951 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.auth.core [None req-069c320f-b649-469b-9046-da0addae1253 None None] MFA Rules not processed for user `49f280ab086b4610855c4ffa378517d9`. Rule list: `[]` (Enabled: `True`). {{(pid=81400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 03:32:14.892129 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-069c320f-b649-469b-9046-da0addae1253 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:32:14.949120 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 1237/2480] 10.222.0.18 () {58 vars in 958 bytes} [Tue Sep 13 03:32:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 03:32:14.958686 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1c7b006-94b8-4550-bc48-462e1d4547be admin admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:32:14.960634 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-a1c7b006-94b8-4550-bc48-462e1d4547be admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:32:15.015519 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1c7b006-94b8-4550-bc48-462e1d4547be admin admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:32:15.019143 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e58a25ed-1dc2-42cb-824a-7ab223bb770a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdffbc851086441ebd01022d1e4c2609', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b9d94f6a68f7470d9494f112175620dc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:32:15.019737 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e58a25ed-1dc2-42cb-824a-7ab223bb770a service neutron] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:32:15.020009 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e58a25ed-1dc2-42cb-824a-7ab223bb770a service neutron] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:32:15.020277 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e58a25ed-1dc2-42cb-824a-7ab223bb770a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:32:15.029462 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e58a25ed-1dc2-42cb-824a-7ab223bb770a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:32:15.030898 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e58a25ed-1dc2-42cb-824a-7ab223bb770a service neutron] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:32:15.040868 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 1244/2481] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Sep 13 03:32:14 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 03:32:15.307436 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-069c320f-b649-469b-9046-da0addae1253 None None] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:32:15.308869 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-069c320f-b649-469b-9046-da0addae1253 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:32:15.321777 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-069c320f-b649-469b-9046-da0addae1253 None None] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:32:15.326369 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02fbbf21-f2cb-4b47-9bb7-fc2419479113 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76c5181fc7be45279ac00017bb4640b3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:32:15.327036 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02fbbf21-f2cb-4b47-9bb7-fc2419479113 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:32:15.327314 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02fbbf21-f2cb-4b47-9bb7-fc2419479113 admin admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:32:15.327585 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02fbbf21-f2cb-4b47-9bb7-fc2419479113 admin admin] PATH_INFO: `/v3/projects/7071167bde9e45c08f8a0ddac6638b72` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:32:15.328707 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02fbbf21-f2cb-4b47-9bb7-fc2419479113 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7071167bde9e45c08f8a0ddac6638b72)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:32:15.330723 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02fbbf21-f2cb-4b47-9bb7-fc2419479113 admin admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:32:15.356208 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=97,keystone:INSERT=2,keystone:DELETE=1 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:32:15.379099 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-02fbbf21-f2cb-4b47-9bb7-fc2419479113 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7071167bde9e45c08f8a0ddac6638b72', 'request_id': 'req-02fbbf21-f2cb-4b47-9bb7-fc2419479113'} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:32:15.385081 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-02fbbf21-f2cb-4b47-9bb7-fc2419479113 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7071167bde9e45c08f8a0ddac6638b72', 'request_id': 'req-02fbbf21-f2cb-4b47-9bb7-fc2419479113'} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:32:15.390991 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-02fbbf21-f2cb-4b47-9bb7-fc2419479113 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7071167bde9e45c08f8a0ddac6638b72', 'request_id': 'req-02fbbf21-f2cb-4b47-9bb7-fc2419479113'} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:32:15.405686 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-02fbbf21-f2cb-4b47-9bb7-fc2419479113 admin admin] The token cache is being invalidate because project 7071167bde9e45c08f8a0ddac6638b72 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81400) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 03:32:15.405972 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-02fbbf21-f2cb-4b47-9bb7-fc2419479113 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:32:15.406586 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-02fbbf21-f2cb-4b47-9bb7-fc2419479113 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:32:15.408158 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 1238/2482] 10.222.0.18 () {58 vars in 1233 bytes} [Tue Sep 13 03:32:15 2022] DELETE /identity/v3/projects/7071167bde9e45c08f8a0ddac6638b72 => generated 0 bytes in 102 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 03:32:15.413386 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45fc351b-c4a3-41f8-9771-6118708e0fef None None] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:32:15.413669 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45fc351b-c4a3-41f8-9771-6118708e0fef None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:32:15.413964 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45fc351b-c4a3-41f8-9771-6118708e0fef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:32:15.432964 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.auth.core [None req-45fc351b-c4a3-41f8-9771-6118708e0fef None None] MFA Rules not processed for user `49f280ab086b4610855c4ffa378517d9`. Rule list: `[]` (Enabled: `True`). {{(pid=81399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 03:32:15.434167 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-45fc351b-c4a3-41f8-9771-6118708e0fef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:32:15.492349 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 1245/2483] 10.222.0.18 () {58 vars in 958 bytes} [Tue Sep 13 03:32:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 13 03:32:15.500255 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f4b4484-ebdb-4dbf-ad15-6b5d1d85db43 None None] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:32:15.500543 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f4b4484-ebdb-4dbf-ad15-6b5d1d85db43 None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:32:15.500817 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f4b4484-ebdb-4dbf-ad15-6b5d1d85db43 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:32:15.521736 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.auth.core [None req-4f4b4484-ebdb-4dbf-ad15-6b5d1d85db43 None None] MFA Rules not processed for user `49f280ab086b4610855c4ffa378517d9`. Rule list: `[]` (Enabled: `True`). {{(pid=81400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 03:32:15.523591 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-4f4b4484-ebdb-4dbf-ad15-6b5d1d85db43 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:32:15.532070 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 1239/2484] 10.222.0.18 () {58 vars in 958 bytes} [Tue Sep 13 03:32:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 03:32:15.536421 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45fc351b-c4a3-41f8-9771-6118708e0fef None None] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:32:15.546244 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45fc351b-c4a3-41f8-9771-6118708e0fef None None] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:32:15.550836 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52e8bcb7-017c-4117-86a2-d75139eb4690 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76c5181fc7be45279ac00017bb4640b3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:32:15.551422 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52e8bcb7-017c-4117-86a2-d75139eb4690 admin admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:32:15.551696 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52e8bcb7-017c-4117-86a2-d75139eb4690 admin admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:32:15.551962 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52e8bcb7-017c-4117-86a2-d75139eb4690 admin admin] PATH_INFO: `/v3/domains` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:32:15.552588 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52e8bcb7-017c-4117-86a2-d75139eb4690 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 03:32:15.552902 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52e8bcb7-017c-4117-86a2-d75139eb4690 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:32:15.554977 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52e8bcb7-017c-4117-86a2-d75139eb4690 admin admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:32:15.566102 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 1246/2485] 10.222.0.18 () {58 vars in 1153 bytes} [Tue Sep 13 03:32:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 03:32:15.570924 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f4b4484-ebdb-4dbf-ad15-6b5d1d85db43 None None] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:32:15.580147 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f4b4484-ebdb-4dbf-ad15-6b5d1d85db43 None None] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:32:15.585145 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5c62ef0-241a-4e6c-a4c5-2bccac871731 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76c5181fc7be45279ac00017bb4640b3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:32:15.585736 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5c62ef0-241a-4e6c-a4c5-2bccac871731 admin admin] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:32:15.586011 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5c62ef0-241a-4e6c-a4c5-2bccac871731 admin admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:32:15.586280 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5c62ef0-241a-4e6c-a4c5-2bccac871731 admin admin] PATH_INFO: `/v3/projects` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:32:15.586949 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5c62ef0-241a-4e6c-a4c5-2bccac871731 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:32:15.588956 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5c62ef0-241a-4e6c-a4c5-2bccac871731 admin admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:32:15.615040 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 1240/2486] 10.222.0.18 () {60 vars in 1153 bytes} [Tue Sep 13 03:32:15 2022] POST /identity/v3/projects => generated 357 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 03:32:15.619761 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52e8bcb7-017c-4117-86a2-d75139eb4690 admin admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:32:15.629663 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52e8bcb7-017c-4117-86a2-d75139eb4690 admin admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:32:15.679221 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:DELETE=3,keystone:SELECT=97 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:32:15.685791 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-531c9144-2664-4de9-a758-18f9f5c50975 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76c5181fc7be45279ac00017bb4640b3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:32:15.686391 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-531c9144-2664-4de9-a758-18f9f5c50975 admin admin] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:32:15.686697 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-531c9144-2664-4de9-a758-18f9f5c50975 admin admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:32:15.686972 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-531c9144-2664-4de9-a758-18f9f5c50975 admin admin] PATH_INFO: `/v3/users` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:32:15.687624 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-531c9144-2664-4de9-a758-18f9f5c50975 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:32:15.689529 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-531c9144-2664-4de9-a758-18f9f5c50975 admin admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:32:15.710261 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 1247/2487] 10.222.0.18 () {60 vars in 1144 bytes} [Tue Sep 13 03:32:15 2022] POST /identity/v3/users => generated 279 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 03:32:15.714525 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5c62ef0-241a-4e6c-a4c5-2bccac871731 admin admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:32:15.724657 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5c62ef0-241a-4e6c-a4c5-2bccac871731 admin admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:32:15.729241 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10a9df19-bc95-45cd-b0fe-903f8ec2a329 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76c5181fc7be45279ac00017bb4640b3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:32:15.729830 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a9df19-bc95-45cd-b0fe-903f8ec2a329 admin admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:32:15.730108 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a9df19-bc95-45cd-b0fe-903f8ec2a329 admin admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:32:15.730379 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a9df19-bc95-45cd-b0fe-903f8ec2a329 admin admin] PATH_INFO: `/v3/roles` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:32:15.731056 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10a9df19-bc95-45cd-b0fe-903f8ec2a329 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:32:15.732642 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10a9df19-bc95-45cd-b0fe-903f8ec2a329 admin admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:32:15.742944 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 1241/2488] 10.222.0.18 () {58 vars in 1122 bytes} [Tue Sep 13 03:32:15 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 03:32:15.746806 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-531c9144-2664-4de9-a758-18f9f5c50975 admin admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:32:15.756564 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-531c9144-2664-4de9-a758-18f9f5c50975 admin admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:32:15.761849 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c51506f8-39d6-4193-983d-30774c5fe442 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76c5181fc7be45279ac00017bb4640b3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:32:15.762441 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c51506f8-39d6-4193-983d-30774c5fe442 admin admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:32:15.762760 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c51506f8-39d6-4193-983d-30774c5fe442 admin admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:32:15.763033 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c51506f8-39d6-4193-983d-30774c5fe442 admin admin] PATH_INFO: `/v3/roles` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:32:15.763668 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c51506f8-39d6-4193-983d-30774c5fe442 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:32:15.765240 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c51506f8-39d6-4193-983d-30774c5fe442 admin admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:32:15.775682 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 1248/2489] 10.222.0.18 () {58 vars in 1122 bytes} [Tue Sep 13 03:32:15 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 03:32:15.779673 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10a9df19-bc95-45cd-b0fe-903f8ec2a329 admin admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:32:15.790038 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10a9df19-bc95-45cd-b0fe-903f8ec2a329 admin admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:32:15.794705 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc7ef1a2-de55-405d-98a9-5a011dc250d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76c5181fc7be45279ac00017bb4640b3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:32:15.795273 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc7ef1a2-de55-405d-98a9-5a011dc250d9 admin admin] REQUEST_METHOD: `PUT` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:32:15.795546 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc7ef1a2-de55-405d-98a9-5a011dc250d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:32:15.795810 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc7ef1a2-de55-405d-98a9-5a011dc250d9 admin admin] PATH_INFO: `/v3/projects/37f2940fc6054cb497e76f75b5917edf/users/9742fad068c94342bb8d6c41584b804c/roles/1eb7a5b5203945c29e6d22d653f47847` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:32:15.811937 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc7ef1a2-de55-405d-98a9-5a011dc250d9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=37f2940fc6054cb497e76f75b5917edf, user_id=9742fad068c94342bb8d6c41584b804c, role_id=1eb7a5b5203945c29e6d22d653f47847)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:32:15.814582 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc7ef1a2-de55-405d-98a9-5a011dc250d9 admin admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:32:15.827568 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 1242/2490] 10.222.0.18 () {60 vars in 1483 bytes} [Tue Sep 13 03:32:15 2022] PUT /identity/v3/projects/37f2940fc6054cb497e76f75b5917edf/users/9742fad068c94342bb8d6c41584b804c/roles/1eb7a5b5203945c29e6d22d653f47847 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 03:32:15.839124 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f95b9511-4e8c-434c-a64d-c311f2f73f2c None None] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:32:15.839435 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f95b9511-4e8c-434c-a64d-c311f2f73f2c None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:32:15.839705 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f95b9511-4e8c-434c-a64d-c311f2f73f2c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:32:15.867165 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.auth.core [None req-f95b9511-4e8c-434c-a64d-c311f2f73f2c None None] MFA Rules not processed for user `9742fad068c94342bb8d6c41584b804c`. Rule list: `[]` (Enabled: `True`). {{(pid=81399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 03:32:15.868349 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-f95b9511-4e8c-434c-a64d-c311f2f73f2c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:32:15.930908 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 1249/2491] 10.222.0.18 () {58 vars in 958 bytes} [Tue Sep 13 03:32:15 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 03:32:15.935251 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc7ef1a2-de55-405d-98a9-5a011dc250d9 admin admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:32:15.995838 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc7ef1a2-de55-405d-98a9-5a011dc250d9 admin admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:32:16.000447 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58c7c25d-ecb2-4c54-b8e9-3cdb43346e6e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76c5181fc7be45279ac00017bb4640b3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:32:16.001040 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c7c25d-ecb2-4c54-b8e9-3cdb43346e6e admin admin] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:32:16.001319 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c7c25d-ecb2-4c54-b8e9-3cdb43346e6e admin admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:32:16.001587 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c7c25d-ecb2-4c54-b8e9-3cdb43346e6e admin admin] PATH_INFO: `/v3/projects` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:32:16.002215 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58c7c25d-ecb2-4c54-b8e9-3cdb43346e6e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:32:16.004148 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58c7c25d-ecb2-4c54-b8e9-3cdb43346e6e admin admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:32:16.029311 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 1243/2492] 10.222.0.18 () {60 vars in 1153 bytes} [Tue Sep 13 03:32:15 2022] POST /identity/v3/projects => generated 357 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 13 03:32:16.033299 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f95b9511-4e8c-434c-a64d-c311f2f73f2c None None] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:32:16.044509 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f95b9511-4e8c-434c-a64d-c311f2f73f2c None None] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:32:16.099973 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0952fab1-d77f-45be-8ff3-d545aa36cb5a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76c5181fc7be45279ac00017bb4640b3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:32:16.101099 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0952fab1-d77f-45be-8ff3-d545aa36cb5a admin admin] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:32:16.101382 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0952fab1-d77f-45be-8ff3-d545aa36cb5a admin admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:32:16.101653 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0952fab1-d77f-45be-8ff3-d545aa36cb5a admin admin] PATH_INFO: `/v3/users` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:32:16.102269 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0952fab1-d77f-45be-8ff3-d545aa36cb5a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:32:16.104225 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0952fab1-d77f-45be-8ff3-d545aa36cb5a admin admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:32:16.125003 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 1250/2493] 10.222.0.18 () {60 vars in 1144 bytes} [Tue Sep 13 03:32:16 2022] POST /identity/v3/users => generated 285 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 03:32:16.128938 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58c7c25d-ecb2-4c54-b8e9-3cdb43346e6e admin admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:32:16.139097 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58c7c25d-ecb2-4c54-b8e9-3cdb43346e6e admin admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:32:16.143664 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b3fc450-e3c5-4b71-a344-08fc26ef2a13 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76c5181fc7be45279ac00017bb4640b3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:32:16.144253 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b3fc450-e3c5-4b71-a344-08fc26ef2a13 admin admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:32:16.144528 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b3fc450-e3c5-4b71-a344-08fc26ef2a13 admin admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:32:16.144793 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b3fc450-e3c5-4b71-a344-08fc26ef2a13 admin admin] PATH_INFO: `/v3/roles` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:32:16.145422 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b3fc450-e3c5-4b71-a344-08fc26ef2a13 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:32:16.147121 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b3fc450-e3c5-4b71-a344-08fc26ef2a13 admin admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:32:16.157512 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 1244/2494] 10.222.0.18 () {58 vars in 1122 bytes} [Tue Sep 13 03:32:16 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 03:32:16.161515 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0952fab1-d77f-45be-8ff3-d545aa36cb5a admin admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:32:16.172021 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0952fab1-d77f-45be-8ff3-d545aa36cb5a admin admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:32:16.176956 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b599863d-69d3-4d41-a49a-ebfec30636f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76c5181fc7be45279ac00017bb4640b3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:32:16.177607 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b599863d-69d3-4d41-a49a-ebfec30636f3 admin admin] REQUEST_METHOD: `PUT` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:32:16.177890 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b599863d-69d3-4d41-a49a-ebfec30636f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:32:16.178196 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b599863d-69d3-4d41-a49a-ebfec30636f3 admin admin] PATH_INFO: `/v3/projects/f037dc96d9444e3a8b55bfd6e7c656fe/users/7ff83fe1ec2f41aea378a39abd514a38/roles/425d4b481c7846e0bc07e7e805d5b15f` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:32:16.193426 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b599863d-69d3-4d41-a49a-ebfec30636f3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f037dc96d9444e3a8b55bfd6e7c656fe, user_id=7ff83fe1ec2f41aea378a39abd514a38, role_id=425d4b481c7846e0bc07e7e805d5b15f)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:32:16.196109 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b599863d-69d3-4d41-a49a-ebfec30636f3 admin admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:32:16.208537 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 1251/2495] 10.222.0.18 () {60 vars in 1483 bytes} [Tue Sep 13 03:32:16 2022] PUT /identity/v3/projects/f037dc96d9444e3a8b55bfd6e7c656fe/users/7ff83fe1ec2f41aea378a39abd514a38/roles/425d4b481c7846e0bc07e7e805d5b15f => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 03:32:16.212424 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b3fc450-e3c5-4b71-a344-08fc26ef2a13 admin admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:32:16.222442 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b3fc450-e3c5-4b71-a344-08fc26ef2a13 admin admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:32:16.240561 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=90,keystone:DELETE=6,keystone:INSERT=4 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:32:16.280675 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5bb357c-8b87-4ab4-870c-6abba1a92ce6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76c5181fc7be45279ac00017bb4640b3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:32:16.281273 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5bb357c-8b87-4ab4-870c-6abba1a92ce6 admin admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:32:16.281551 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5bb357c-8b87-4ab4-870c-6abba1a92ce6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:32:16.281823 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5bb357c-8b87-4ab4-870c-6abba1a92ce6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:32:16.282452 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5bb357c-8b87-4ab4-870c-6abba1a92ce6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:32:16.284081 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5bb357c-8b87-4ab4-870c-6abba1a92ce6 admin admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:32:16.294430 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 1245/2496] 10.222.0.18 () {58 vars in 1122 bytes} [Tue Sep 13 03:32:16 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 13 03:32:16.298658 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b599863d-69d3-4d41-a49a-ebfec30636f3 admin admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:32:16.309301 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b599863d-69d3-4d41-a49a-ebfec30636f3 admin admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:32:16.313857 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02c6eaea-c651-4c25-a981-8c0be631cacd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76c5181fc7be45279ac00017bb4640b3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:32:16.314512 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02c6eaea-c651-4c25-a981-8c0be631cacd admin admin] REQUEST_METHOD: `PUT` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:32:16.314809 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02c6eaea-c651-4c25-a981-8c0be631cacd admin admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:32:16.315082 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02c6eaea-c651-4c25-a981-8c0be631cacd admin admin] PATH_INFO: `/v3/projects/f037dc96d9444e3a8b55bfd6e7c656fe/users/7ff83fe1ec2f41aea378a39abd514a38/roles/425d4b481c7846e0bc07e7e805d5b15f` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:32:16.316983 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02c6eaea-c651-4c25-a981-8c0be631cacd admin admin] RBAC: Authorizing `identity:create_grant(project_id=f037dc96d9444e3a8b55bfd6e7c656fe, user_id=7ff83fe1ec2f41aea378a39abd514a38, role_id=425d4b481c7846e0bc07e7e805d5b15f)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:32:16.319638 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02c6eaea-c651-4c25-a981-8c0be631cacd admin admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:32:16.329529 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 1252/2497] 10.222.0.18 () {60 vars in 1483 bytes} [Tue Sep 13 03:32:16 2022] PUT /identity/v3/projects/f037dc96d9444e3a8b55bfd6e7c656fe/users/7ff83fe1ec2f41aea378a39abd514a38/roles/425d4b481c7846e0bc07e7e805d5b15f => generated 0 bytes in 31 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 13 03:32:16.340759 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc5032fa-d7a4-40d2-bff9-44dc678cc072 None None] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:32:16.341044 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc5032fa-d7a4-40d2-bff9-44dc678cc072 None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:32:16.341318 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc5032fa-d7a4-40d2-bff9-44dc678cc072 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:32:16.369172 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.auth.core [None req-bc5032fa-d7a4-40d2-bff9-44dc678cc072 None None] MFA Rules not processed for user `7ff83fe1ec2f41aea378a39abd514a38`. Rule list: `[]` (Enabled: `True`). {{(pid=81400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 03:32:16.370376 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-bc5032fa-d7a4-40d2-bff9-44dc678cc072 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:32:16.437122 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 1246/2498] 10.222.0.18 () {58 vars in 958 bytes} [Tue Sep 13 03:32:16 2022] POST /identity/v3/auth/tokens => generated 3301 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 03:32:16.448461 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02c6eaea-c651-4c25-a981-8c0be631cacd admin admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:32:16.450070 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-02c6eaea-c651-4c25-a981-8c0be631cacd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:32:16.471114 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:32:16.508292 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02c6eaea-c651-4c25-a981-8c0be631cacd admin admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:32:16.511946 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69e13731-69e5-47c7-972d-d2f5b656d30c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '924ee3f608d34a18b0ddc4695f102a56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b9d94f6a68f7470d9494f112175620dc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:32:16.512612 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e13731-69e5-47c7-972d-d2f5b656d30c service cinder] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:32:16.512890 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e13731-69e5-47c7-972d-d2f5b656d30c service cinder] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:32:16.513162 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e13731-69e5-47c7-972d-d2f5b656d30c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:32:16.522139 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69e13731-69e5-47c7-972d-d2f5b656d30c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:32:16.524074 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69e13731-69e5-47c7-972d-d2f5b656d30c service cinder] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:32:16.533334 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 1253/2499] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Sep 13 03:32:16 2022] GET /identity/v3/auth/tokens => generated 3301 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 03:32:16.607353 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc5032fa-d7a4-40d2-bff9-44dc678cc072 None None] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:32:16.616944 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc5032fa-d7a4-40d2-bff9-44dc678cc072 None None] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:32:16.620647 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a4eaf06-cb22-4485-ac00-7f89b0f3d56a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '924ee3f608d34a18b0ddc4695f102a56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b9d94f6a68f7470d9494f112175620dc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:32:16.621240 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a4eaf06-cb22-4485-ac00-7f89b0f3d56a service cinder] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:32:16.621520 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a4eaf06-cb22-4485-ac00-7f89b0f3d56a service cinder] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:32:16.621791 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a4eaf06-cb22-4485-ac00-7f89b0f3d56a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:32:16.678387 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a4eaf06-cb22-4485-ac00-7f89b0f3d56a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:32:16.679909 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a4eaf06-cb22-4485-ac00-7f89b0f3d56a service cinder] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:32:16.688758 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 1247/2500] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Sep 13 03:32:16 2022] GET /identity/v3/auth/tokens => generated 3234 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 03:32:21.556156 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69e13731-69e5-47c7-972d-d2f5b656d30c service cinder] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:32:21.564790 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69e13731-69e5-47c7-972d-d2f5b656d30c service cinder] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:32:21.568367 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7680b7bb-6f7e-4bdf-bad6-eae69a86b9eb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '924ee3f608d34a18b0ddc4695f102a56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b9d94f6a68f7470d9494f112175620dc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:32:21.568959 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7680b7bb-6f7e-4bdf-bad6-eae69a86b9eb service cinder] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:32:21.569235 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7680b7bb-6f7e-4bdf-bad6-eae69a86b9eb service cinder] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:32:21.569503 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7680b7bb-6f7e-4bdf-bad6-eae69a86b9eb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:32:21.571890 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-7680b7bb-6f7e-4bdf-bad6-eae69a86b9eb service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:32:21.630384 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7680b7bb-6f7e-4bdf-bad6-eae69a86b9eb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:32:21.631847 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7680b7bb-6f7e-4bdf-bad6-eae69a86b9eb service cinder] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:32:21.641400 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 1254/2501] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Sep 13 03:32:21 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 03:32:26.684743 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=81 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:32:31.637143 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=46 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:32:32.690970 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf1e8db-4ca4-47b0-8199-cd8447b9f97d None None] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:32:32.691443 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf1e8db-4ca4-47b0-8199-cd8447b9f97d None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:32:32.691531 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf1e8db-4ca4-47b0-8199-cd8447b9f97d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:32:32.711177 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.auth.core [None req-0cf1e8db-4ca4-47b0-8199-cd8447b9f97d None None] MFA Rules not processed for user `47fa1f7ce54145ceb3564b9ba7b4964f`. Rule list: `[]` (Enabled: `True`). {{(pid=81400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 03:32:32.713004 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-0cf1e8db-4ca4-47b0-8199-cd8447b9f97d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:32:32.721176 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 1248/2502] 10.222.0.18 () {58 vars in 958 bytes} [Tue Sep 13 03:32:32 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 03:32:33.039160 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7680b7bb-6f7e-4bdf-bad6-eae69a86b9eb service cinder] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:32:33.040587 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-7680b7bb-6f7e-4bdf-bad6-eae69a86b9eb service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:32:33.217305 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7680b7bb-6f7e-4bdf-bad6-eae69a86b9eb service cinder] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:32:33.220866 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e97ab53-c8a7-4c9d-953d-93f4772a3eef service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdffbc851086441ebd01022d1e4c2609', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b9d94f6a68f7470d9494f112175620dc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:32:33.221460 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e97ab53-c8a7-4c9d-953d-93f4772a3eef service neutron] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:32:33.221731 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e97ab53-c8a7-4c9d-953d-93f4772a3eef service neutron] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:32:33.221995 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e97ab53-c8a7-4c9d-953d-93f4772a3eef service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:32:33.223712 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-2e97ab53-c8a7-4c9d-953d-93f4772a3eef service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:32:33.233338 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e97ab53-c8a7-4c9d-953d-93f4772a3eef service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:32:33.234778 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e97ab53-c8a7-4c9d-953d-93f4772a3eef service neutron] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:32:33.242445 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 1255/2503] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Sep 13 03:32:33 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 204 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 03:32:37.969924 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cf1e8db-4ca4-47b0-8199-cd8447b9f97d None None] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:32:38.032212 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cf1e8db-4ca4-47b0-8199-cd8447b9f97d None None] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:32:38.036778 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74da00d6-7813-4578-9238-740250a78773 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76c5181fc7be45279ac00017bb4640b3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:32:38.037692 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74da00d6-7813-4578-9238-740250a78773 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:32:38.037987 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74da00d6-7813-4578-9238-740250a78773 admin admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:32:38.038255 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74da00d6-7813-4578-9238-740250a78773 admin admin] PATH_INFO: `/v3/users/9742fad068c94342bb8d6c41584b804c` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:32:38.039407 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74da00d6-7813-4578-9238-740250a78773 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9742fad068c94342bb8d6c41584b804c)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:32:38.041968 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74da00d6-7813-4578-9238-740250a78773 admin admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:32:38.086883 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-74da00d6-7813-4578-9238-740250a78773 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9742fad068c94342bb8d6c41584b804c'} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:32:38.094260 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-74da00d6-7813-4578-9238-740250a78773 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9742fad068c94342bb8d6c41584b804c'} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:32:38.094900 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-74da00d6-7813-4578-9238-740250a78773 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9742fad068c94342bb8d6c41584b804c'} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:32:38.107410 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-74da00d6-7813-4578-9238-740250a78773 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9742fad068c94342bb8d6c41584b804c'} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:32:38.114132 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-74da00d6-7813-4578-9238-740250a78773 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9742fad068c94342bb8d6c41584b804c'} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:32:38.119946 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 1249/2504] 10.222.0.18 () {58 vars in 1224 bytes} [Tue Sep 13 03:32:37 2022] DELETE /identity/v3/users/9742fad068c94342bb8d6c41584b804c => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 03:32:38.124407 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05bc6575-2676-48f1-bd50-9c29436142da None None] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:32:38.124886 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05bc6575-2676-48f1-bd50-9c29436142da None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:32:38.125001 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05bc6575-2676-48f1-bd50-9c29436142da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:32:38.143226 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.auth.core [None req-05bc6575-2676-48f1-bd50-9c29436142da None None] MFA Rules not processed for user `49f280ab086b4610855c4ffa378517d9`. Rule list: `[]` (Enabled: `True`). {{(pid=81399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 03:32:38.144406 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-05bc6575-2676-48f1-bd50-9c29436142da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:32:38.201075 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 1256/2505] 10.222.0.18 () {58 vars in 958 bytes} [Tue Sep 13 03:32:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 13 03:32:38.210766 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74da00d6-7813-4578-9238-740250a78773 admin admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:32:38.212082 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-74da00d6-7813-4578-9238-740250a78773 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:32:38.268211 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74da00d6-7813-4578-9238-740250a78773 admin admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:32:38.271824 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c99f8db8-e8b0-4a9c-8e43-831efd227a84 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdffbc851086441ebd01022d1e4c2609', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b9d94f6a68f7470d9494f112175620dc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:32:38.272417 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c99f8db8-e8b0-4a9c-8e43-831efd227a84 service neutron] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:32:38.272695 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c99f8db8-e8b0-4a9c-8e43-831efd227a84 service neutron] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:32:38.272962 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c99f8db8-e8b0-4a9c-8e43-831efd227a84 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:32:38.282771 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c99f8db8-e8b0-4a9c-8e43-831efd227a84 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:32:38.284264 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c99f8db8-e8b0-4a9c-8e43-831efd227a84 service neutron] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:32:38.294742 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 1250/2506] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Sep 13 03:32:38 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 03:32:38.671615 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05bc6575-2676-48f1-bd50-9c29436142da None None] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:32:38.673495 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-05bc6575-2676-48f1-bd50-9c29436142da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:32:38.689516 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05bc6575-2676-48f1-bd50-9c29436142da None None] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:32:38.694827 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba4c77e7-7cb1-46f2-9978-a7076ef68a98 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76c5181fc7be45279ac00017bb4640b3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:32:38.695467 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba4c77e7-7cb1-46f2-9978-a7076ef68a98 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:32:38.695746 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba4c77e7-7cb1-46f2-9978-a7076ef68a98 admin admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:32:38.696201 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba4c77e7-7cb1-46f2-9978-a7076ef68a98 admin admin] PATH_INFO: `/v3/projects/37f2940fc6054cb497e76f75b5917edf` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:32:38.702591 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba4c77e7-7cb1-46f2-9978-a7076ef68a98 admin admin] RBAC: Authorizing `identity:delete_project(project_id=37f2940fc6054cb497e76f75b5917edf)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:32:38.705266 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba4c77e7-7cb1-46f2-9978-a7076ef68a98 admin admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:32:38.754774 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-ba4c77e7-7cb1-46f2-9978-a7076ef68a98 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '37f2940fc6054cb497e76f75b5917edf', 'request_id': 'req-ba4c77e7-7cb1-46f2-9978-a7076ef68a98'} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:32:38.761629 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-ba4c77e7-7cb1-46f2-9978-a7076ef68a98 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '37f2940fc6054cb497e76f75b5917edf', 'request_id': 'req-ba4c77e7-7cb1-46f2-9978-a7076ef68a98'} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:32:38.768428 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-ba4c77e7-7cb1-46f2-9978-a7076ef68a98 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '37f2940fc6054cb497e76f75b5917edf', 'request_id': 'req-ba4c77e7-7cb1-46f2-9978-a7076ef68a98'} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:32:38.781958 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-ba4c77e7-7cb1-46f2-9978-a7076ef68a98 admin admin] The token cache is being invalidate because project 37f2940fc6054cb497e76f75b5917edf was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81399) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 03:32:38.782304 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-ba4c77e7-7cb1-46f2-9978-a7076ef68a98 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:32:38.782949 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-ba4c77e7-7cb1-46f2-9978-a7076ef68a98 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:32:38.784535 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 1257/2507] 10.222.0.18 () {58 vars in 1233 bytes} [Tue Sep 13 03:32:38 2022] DELETE /identity/v3/projects/37f2940fc6054cb497e76f75b5917edf => generated 0 bytes in 113 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 03:32:38.788803 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c99f8db8-e8b0-4a9c-8e43-831efd227a84 service neutron] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:32:38.799798 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c99f8db8-e8b0-4a9c-8e43-831efd227a84 service neutron] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:32:38.812791 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:32:38.861469 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a025109-a314-4d14-9584-ebc50aed2e96 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76c5181fc7be45279ac00017bb4640b3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:32:38.862095 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a025109-a314-4d14-9584-ebc50aed2e96 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:32:38.862366 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a025109-a314-4d14-9584-ebc50aed2e96 admin admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:32:38.862656 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a025109-a314-4d14-9584-ebc50aed2e96 admin admin] PATH_INFO: `/v3/users/7ff83fe1ec2f41aea378a39abd514a38` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:32:38.863774 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a025109-a314-4d14-9584-ebc50aed2e96 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7ff83fe1ec2f41aea378a39abd514a38)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:32:38.865688 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a025109-a314-4d14-9584-ebc50aed2e96 admin admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:32:38.913495 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-3a025109-a314-4d14-9584-ebc50aed2e96 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7ff83fe1ec2f41aea378a39abd514a38'} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:32:38.921362 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-3a025109-a314-4d14-9584-ebc50aed2e96 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7ff83fe1ec2f41aea378a39abd514a38'} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:32:38.922234 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-3a025109-a314-4d14-9584-ebc50aed2e96 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7ff83fe1ec2f41aea378a39abd514a38'} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:32:38.935456 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-3a025109-a314-4d14-9584-ebc50aed2e96 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7ff83fe1ec2f41aea378a39abd514a38'} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:32:38.942501 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-3a025109-a314-4d14-9584-ebc50aed2e96 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7ff83fe1ec2f41aea378a39abd514a38'} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:32:38.949085 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 1251/2508] 10.222.0.18 () {58 vars in 1224 bytes} [Tue Sep 13 03:32:38 2022] DELETE /identity/v3/users/7ff83fe1ec2f41aea378a39abd514a38 => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 03:32:39.220603 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba4c77e7-7cb1-46f2-9978-a7076ef68a98 admin admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:32:39.222044 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-ba4c77e7-7cb1-46f2-9978-a7076ef68a98 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:32:39.235708 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba4c77e7-7cb1-46f2-9978-a7076ef68a98 admin admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:32:39.260718 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:32:39.296222 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bf90472-245f-4f17-9065-7222084e0211 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76c5181fc7be45279ac00017bb4640b3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:32:39.296800 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf90472-245f-4f17-9065-7222084e0211 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:32:39.297081 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf90472-245f-4f17-9065-7222084e0211 admin admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:32:39.297355 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf90472-245f-4f17-9065-7222084e0211 admin admin] PATH_INFO: `/v3/projects/f037dc96d9444e3a8b55bfd6e7c656fe` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:32:39.298487 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bf90472-245f-4f17-9065-7222084e0211 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f037dc96d9444e3a8b55bfd6e7c656fe)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:32:39.300488 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bf90472-245f-4f17-9065-7222084e0211 admin admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:32:39.345501 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-2bf90472-245f-4f17-9065-7222084e0211 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f037dc96d9444e3a8b55bfd6e7c656fe', 'request_id': 'req-2bf90472-245f-4f17-9065-7222084e0211'} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:32:39.351206 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-2bf90472-245f-4f17-9065-7222084e0211 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f037dc96d9444e3a8b55bfd6e7c656fe', 'request_id': 'req-2bf90472-245f-4f17-9065-7222084e0211'} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:32:39.357796 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-2bf90472-245f-4f17-9065-7222084e0211 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f037dc96d9444e3a8b55bfd6e7c656fe', 'request_id': 'req-2bf90472-245f-4f17-9065-7222084e0211'} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:32:39.370590 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-2bf90472-245f-4f17-9065-7222084e0211 admin admin] The token cache is being invalidate because project f037dc96d9444e3a8b55bfd6e7c656fe was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81399) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 03:32:39.370908 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-2bf90472-245f-4f17-9065-7222084e0211 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:32:39.371522 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-2bf90472-245f-4f17-9065-7222084e0211 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:32:39.372993 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 1258/2509] 10.222.0.18 () {58 vars in 1233 bytes} [Tue Sep 13 03:32:39 2022] DELETE /identity/v3/projects/f037dc96d9444e3a8b55bfd6e7c656fe => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 13 03:32:39.377895 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cacf9492-6fca-433e-a5d7-9a164c9ccc5d None None] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:32:39.378393 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cacf9492-6fca-433e-a5d7-9a164c9ccc5d None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:32:39.378501 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cacf9492-6fca-433e-a5d7-9a164c9ccc5d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:32:39.397984 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.auth.core [None req-cacf9492-6fca-433e-a5d7-9a164c9ccc5d None None] MFA Rules not processed for user `49f280ab086b4610855c4ffa378517d9`. Rule list: `[]` (Enabled: `True`). {{(pid=81400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 03:32:39.399214 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-cacf9492-6fca-433e-a5d7-9a164c9ccc5d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:32:39.457703 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 1252/2510] 10.222.0.18 () {58 vars in 958 bytes} [Tue Sep 13 03:32:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 03:32:39.466768 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9718f2ed-cc3f-4048-bb3d-717fbab85c30 None None] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:32:39.467048 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9718f2ed-cc3f-4048-bb3d-717fbab85c30 None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:32:39.467313 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9718f2ed-cc3f-4048-bb3d-717fbab85c30 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:32:39.486635 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.auth.core [None req-9718f2ed-cc3f-4048-bb3d-717fbab85c30 None None] MFA Rules not processed for user `49f280ab086b4610855c4ffa378517d9`. Rule list: `[]` (Enabled: `True`). {{(pid=81399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 03:32:39.487850 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-9718f2ed-cc3f-4048-bb3d-717fbab85c30 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:32:39.496168 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 1259/2511] 10.222.0.18 () {58 vars in 958 bytes} [Tue Sep 13 03:32:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 03:32:39.500216 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cacf9492-6fca-433e-a5d7-9a164c9ccc5d None None] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:32:39.509750 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cacf9492-6fca-433e-a5d7-9a164c9ccc5d None None] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:32:39.514196 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20799dc7-0969-4a26-a77f-1060e3db2674 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76c5181fc7be45279ac00017bb4640b3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:32:39.514809 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20799dc7-0969-4a26-a77f-1060e3db2674 admin admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:32:39.515083 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20799dc7-0969-4a26-a77f-1060e3db2674 admin admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:32:39.515347 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20799dc7-0969-4a26-a77f-1060e3db2674 admin admin] PATH_INFO: `/v3/domains` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:32:39.515977 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20799dc7-0969-4a26-a77f-1060e3db2674 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 03:32:39.516275 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20799dc7-0969-4a26-a77f-1060e3db2674 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:32:39.517861 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20799dc7-0969-4a26-a77f-1060e3db2674 admin admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:32:39.529510 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 1253/2512] 10.222.0.18 () {58 vars in 1153 bytes} [Tue Sep 13 03:32:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 03:32:39.533725 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9718f2ed-cc3f-4048-bb3d-717fbab85c30 None None] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:32:39.543571 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9718f2ed-cc3f-4048-bb3d-717fbab85c30 None None] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:32:39.548120 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-508d248b-6988-4775-850b-aaaeed50e0ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76c5181fc7be45279ac00017bb4640b3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:32:39.548676 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-508d248b-6988-4775-850b-aaaeed50e0ac admin admin] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:32:39.548951 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-508d248b-6988-4775-850b-aaaeed50e0ac admin admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:32:39.549219 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-508d248b-6988-4775-850b-aaaeed50e0ac admin admin] PATH_INFO: `/v3/projects` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:32:39.549833 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-508d248b-6988-4775-850b-aaaeed50e0ac admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:32:39.551800 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-508d248b-6988-4775-850b-aaaeed50e0ac admin admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:32:39.578937 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 1260/2513] 10.222.0.18 () {60 vars in 1153 bytes} [Tue Sep 13 03:32:39 2022] POST /identity/v3/projects => generated 369 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 03:32:39.583793 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20799dc7-0969-4a26-a77f-1060e3db2674 admin admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:32:39.593931 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20799dc7-0969-4a26-a77f-1060e3db2674 admin admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:32:39.613202 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:32:39.653905 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c80c8b81-5de0-4b80-b7da-6d1184cb385e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76c5181fc7be45279ac00017bb4640b3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:32:39.654504 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c80c8b81-5de0-4b80-b7da-6d1184cb385e admin admin] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:32:39.654819 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c80c8b81-5de0-4b80-b7da-6d1184cb385e admin admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:32:39.655087 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c80c8b81-5de0-4b80-b7da-6d1184cb385e admin admin] PATH_INFO: `/v3/users` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:32:39.655706 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c80c8b81-5de0-4b80-b7da-6d1184cb385e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:32:39.657609 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c80c8b81-5de0-4b80-b7da-6d1184cb385e admin admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:32:39.680682 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 1254/2514] 10.222.0.18 () {60 vars in 1144 bytes} [Tue Sep 13 03:32:39 2022] POST /identity/v3/users => generated 285 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 03:32:39.685072 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-508d248b-6988-4775-850b-aaaeed50e0ac admin admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:32:39.695458 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-508d248b-6988-4775-850b-aaaeed50e0ac admin admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:32:39.700057 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2703e5e-9c5e-4a25-829d-b4ee2617318d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76c5181fc7be45279ac00017bb4640b3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:32:39.700662 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2703e5e-9c5e-4a25-829d-b4ee2617318d admin admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:32:39.700934 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2703e5e-9c5e-4a25-829d-b4ee2617318d admin admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:32:39.701191 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2703e5e-9c5e-4a25-829d-b4ee2617318d admin admin] PATH_INFO: `/v3/roles` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:32:39.701820 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2703e5e-9c5e-4a25-829d-b4ee2617318d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:32:39.703443 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2703e5e-9c5e-4a25-829d-b4ee2617318d admin admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:32:39.717204 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 1261/2515] 10.222.0.18 () {58 vars in 1122 bytes} [Tue Sep 13 03:32:39 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 03:32:39.719648 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c80c8b81-5de0-4b80-b7da-6d1184cb385e admin admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:32:39.730990 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c80c8b81-5de0-4b80-b7da-6d1184cb385e admin admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:32:39.735806 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf3430a4-b35d-429d-a999-37ac6a215f6a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76c5181fc7be45279ac00017bb4640b3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:32:39.736402 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf3430a4-b35d-429d-a999-37ac6a215f6a admin admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:32:39.736680 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf3430a4-b35d-429d-a999-37ac6a215f6a admin admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:32:39.736946 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf3430a4-b35d-429d-a999-37ac6a215f6a admin admin] PATH_INFO: `/v3/roles` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:32:39.737579 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf3430a4-b35d-429d-a999-37ac6a215f6a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:32:39.739224 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf3430a4-b35d-429d-a999-37ac6a215f6a admin admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:32:39.749874 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 1255/2516] 10.222.0.18 () {58 vars in 1122 bytes} [Tue Sep 13 03:32:39 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 03:32:39.754077 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2703e5e-9c5e-4a25-829d-b4ee2617318d admin admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:32:39.764215 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2703e5e-9c5e-4a25-829d-b4ee2617318d admin admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:32:39.768711 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57ee14a6-61af-4c73-a204-67e69d85efd1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76c5181fc7be45279ac00017bb4640b3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:32:39.769343 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ee14a6-61af-4c73-a204-67e69d85efd1 admin admin] REQUEST_METHOD: `PUT` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:32:39.769619 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ee14a6-61af-4c73-a204-67e69d85efd1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:32:39.769884 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ee14a6-61af-4c73-a204-67e69d85efd1 admin admin] PATH_INFO: `/v3/projects/385bcc0ea9454746a9d4b5c3bc8ac416/users/3f8f059a0fdf4e1da1f35f5220a7cde4/roles/1eb7a5b5203945c29e6d22d653f47847` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:32:39.784513 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57ee14a6-61af-4c73-a204-67e69d85efd1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=385bcc0ea9454746a9d4b5c3bc8ac416, user_id=3f8f059a0fdf4e1da1f35f5220a7cde4, role_id=1eb7a5b5203945c29e6d22d653f47847)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:32:39.788194 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57ee14a6-61af-4c73-a204-67e69d85efd1 admin admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:32:39.801819 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 1262/2517] 10.222.0.18 () {60 vars in 1483 bytes} [Tue Sep 13 03:32:39 2022] PUT /identity/v3/projects/385bcc0ea9454746a9d4b5c3bc8ac416/users/3f8f059a0fdf4e1da1f35f5220a7cde4/roles/1eb7a5b5203945c29e6d22d653f47847 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 03:32:39.812534 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4eb68f7-b037-4b53-8c41-00c02b1fa32d None None] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:32:39.812817 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4eb68f7-b037-4b53-8c41-00c02b1fa32d None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:32:39.813186 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4eb68f7-b037-4b53-8c41-00c02b1fa32d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:32:39.841784 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.auth.core [None req-a4eb68f7-b037-4b53-8c41-00c02b1fa32d None None] MFA Rules not processed for user `3f8f059a0fdf4e1da1f35f5220a7cde4`. Rule list: `[]` (Enabled: `True`). {{(pid=81400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 03:32:39.843014 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-a4eb68f7-b037-4b53-8c41-00c02b1fa32d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:32:39.910391 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 1256/2518] 10.222.0.18 () {58 vars in 958 bytes} [Tue Sep 13 03:32:39 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 13 03:32:39.915267 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57ee14a6-61af-4c73-a204-67e69d85efd1 admin admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:32:39.967089 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:32:39.979789 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57ee14a6-61af-4c73-a204-67e69d85efd1 admin admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:32:39.985501 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d33775b4-2ed5-473d-8bee-daef2d5a9931 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76c5181fc7be45279ac00017bb4640b3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:32:39.986116 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d33775b4-2ed5-473d-8bee-daef2d5a9931 admin admin] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:32:39.986400 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d33775b4-2ed5-473d-8bee-daef2d5a9931 admin admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:32:39.986705 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d33775b4-2ed5-473d-8bee-daef2d5a9931 admin admin] PATH_INFO: `/v3/projects` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:32:39.987347 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d33775b4-2ed5-473d-8bee-daef2d5a9931 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:32:39.989243 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d33775b4-2ed5-473d-8bee-daef2d5a9931 admin admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:32:40.016539 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 1263/2519] 10.222.0.18 () {60 vars in 1153 bytes} [Tue Sep 13 03:32:39 2022] POST /identity/v3/projects => generated 369 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 13 03:32:40.020551 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4eb68f7-b037-4b53-8c41-00c02b1fa32d None None] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:32:40.031113 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4eb68f7-b037-4b53-8c41-00c02b1fa32d None None] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:32:40.088429 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8c7151a-a5a0-4dfc-9900-2384a44215a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76c5181fc7be45279ac00017bb4640b3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:32:40.089097 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c7151a-a5a0-4dfc-9900-2384a44215a9 admin admin] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:32:40.089381 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c7151a-a5a0-4dfc-9900-2384a44215a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:32:40.089651 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c7151a-a5a0-4dfc-9900-2384a44215a9 admin admin] PATH_INFO: `/v3/users` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:32:40.090313 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8c7151a-a5a0-4dfc-9900-2384a44215a9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:32:40.092271 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8c7151a-a5a0-4dfc-9900-2384a44215a9 admin admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:32:40.113352 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 1257/2520] 10.222.0.18 () {60 vars in 1144 bytes} [Tue Sep 13 03:32:40 2022] POST /identity/v3/users => generated 291 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 03:32:40.117318 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d33775b4-2ed5-473d-8bee-daef2d5a9931 admin admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:32:40.128057 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d33775b4-2ed5-473d-8bee-daef2d5a9931 admin admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:32:40.132587 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-054396be-6efd-4cb3-a05e-03b80062cb93 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76c5181fc7be45279ac00017bb4640b3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:32:40.133177 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-054396be-6efd-4cb3-a05e-03b80062cb93 admin admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:32:40.133454 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-054396be-6efd-4cb3-a05e-03b80062cb93 admin admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:32:40.133721 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-054396be-6efd-4cb3-a05e-03b80062cb93 admin admin] PATH_INFO: `/v3/roles` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:32:40.134352 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-054396be-6efd-4cb3-a05e-03b80062cb93 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:32:40.135966 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-054396be-6efd-4cb3-a05e-03b80062cb93 admin admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:32:40.145802 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 1264/2521] 10.222.0.18 () {58 vars in 1122 bytes} [Tue Sep 13 03:32:40 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 03:32:40.149659 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8c7151a-a5a0-4dfc-9900-2384a44215a9 admin admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:32:40.159514 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8c7151a-a5a0-4dfc-9900-2384a44215a9 admin admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:32:40.164393 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fea47d51-3379-48e7-85c4-556dd3173851 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76c5181fc7be45279ac00017bb4640b3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:32:40.165023 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fea47d51-3379-48e7-85c4-556dd3173851 admin admin] REQUEST_METHOD: `PUT` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:32:40.165297 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fea47d51-3379-48e7-85c4-556dd3173851 admin admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:32:40.165564 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fea47d51-3379-48e7-85c4-556dd3173851 admin admin] PATH_INFO: `/v3/projects/f10d1a6d246f4e6eb137db7a3ec8f6f7/users/a00d4283e9784101bdbc412decbb3384/roles/425d4b481c7846e0bc07e7e805d5b15f` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:32:40.180397 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fea47d51-3379-48e7-85c4-556dd3173851 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f10d1a6d246f4e6eb137db7a3ec8f6f7, user_id=a00d4283e9784101bdbc412decbb3384, role_id=425d4b481c7846e0bc07e7e805d5b15f)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:32:40.183762 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fea47d51-3379-48e7-85c4-556dd3173851 admin admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:32:40.186707 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:32:40.197145 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 1258/2522] 10.222.0.18 () {60 vars in 1483 bytes} [Tue Sep 13 03:32:40 2022] PUT /identity/v3/projects/f10d1a6d246f4e6eb137db7a3ec8f6f7/users/a00d4283e9784101bdbc412decbb3384/roles/425d4b481c7846e0bc07e7e805d5b15f => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 03:32:40.201162 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-054396be-6efd-4cb3-a05e-03b80062cb93 admin admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:32:40.211110 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-054396be-6efd-4cb3-a05e-03b80062cb93 admin admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:32:40.265000 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0afad203-7d39-4817-b6f5-92d2915ca0ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76c5181fc7be45279ac00017bb4640b3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:32:40.265605 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0afad203-7d39-4817-b6f5-92d2915ca0ac admin admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:32:40.265877 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0afad203-7d39-4817-b6f5-92d2915ca0ac admin admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:32:40.266139 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0afad203-7d39-4817-b6f5-92d2915ca0ac admin admin] PATH_INFO: `/v3/roles` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:32:40.266796 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0afad203-7d39-4817-b6f5-92d2915ca0ac admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:32:40.268355 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0afad203-7d39-4817-b6f5-92d2915ca0ac admin admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:32:40.278207 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 1265/2523] 10.222.0.18 () {58 vars in 1122 bytes} [Tue Sep 13 03:32:40 2022] GET /identity/v3/roles => generated 1420 bytes in 77 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 03:32:40.282293 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fea47d51-3379-48e7-85c4-556dd3173851 admin admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:32:40.293322 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fea47d51-3379-48e7-85c4-556dd3173851 admin admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:32:40.297819 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f9d36af-a4e4-43cf-b4d2-dc1d1e77c23b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76c5181fc7be45279ac00017bb4640b3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:32:40.298449 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f9d36af-a4e4-43cf-b4d2-dc1d1e77c23b admin admin] REQUEST_METHOD: `PUT` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:32:40.298755 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f9d36af-a4e4-43cf-b4d2-dc1d1e77c23b admin admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:32:40.299026 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f9d36af-a4e4-43cf-b4d2-dc1d1e77c23b admin admin] PATH_INFO: `/v3/projects/f10d1a6d246f4e6eb137db7a3ec8f6f7/users/a00d4283e9784101bdbc412decbb3384/roles/425d4b481c7846e0bc07e7e805d5b15f` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:32:40.300918 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f9d36af-a4e4-43cf-b4d2-dc1d1e77c23b admin admin] RBAC: Authorizing `identity:create_grant(project_id=f10d1a6d246f4e6eb137db7a3ec8f6f7, user_id=a00d4283e9784101bdbc412decbb3384, role_id=425d4b481c7846e0bc07e7e805d5b15f)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:32:40.303592 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f9d36af-a4e4-43cf-b4d2-dc1d1e77c23b admin admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:32:40.313375 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 1259/2524] 10.222.0.18 () {60 vars in 1483 bytes} [Tue Sep 13 03:32:40 2022] PUT /identity/v3/projects/f10d1a6d246f4e6eb137db7a3ec8f6f7/users/a00d4283e9784101bdbc412decbb3384/roles/425d4b481c7846e0bc07e7e805d5b15f => generated 0 bytes in 31 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 03:32:40.324105 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51018202-5e0f-4fe6-a2b4-60872793e8ec None None] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:32:40.324328 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51018202-5e0f-4fe6-a2b4-60872793e8ec None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:32:40.324595 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51018202-5e0f-4fe6-a2b4-60872793e8ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:32:40.351840 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.auth.core [None req-51018202-5e0f-4fe6-a2b4-60872793e8ec None None] MFA Rules not processed for user `a00d4283e9784101bdbc412decbb3384`. Rule list: `[]` (Enabled: `True`). {{(pid=81399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 03:32:40.353018 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-51018202-5e0f-4fe6-a2b4-60872793e8ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:32:40.418889 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 1266/2525] 10.222.0.18 () {58 vars in 958 bytes} [Tue Sep 13 03:32:40 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 03:32:40.429899 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f9d36af-a4e4-43cf-b4d2-dc1d1e77c23b admin admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:32:40.432266 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-6f9d36af-a4e4-43cf-b4d2-dc1d1e77c23b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:32:40.487132 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f9d36af-a4e4-43cf-b4d2-dc1d1e77c23b admin admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:32:40.490744 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83041d35-5ef3-4495-b483-009f6e200524 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '924ee3f608d34a18b0ddc4695f102a56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b9d94f6a68f7470d9494f112175620dc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:32:40.491327 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83041d35-5ef3-4495-b483-009f6e200524 service cinder] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:32:40.491600 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83041d35-5ef3-4495-b483-009f6e200524 service cinder] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:32:40.491866 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83041d35-5ef3-4495-b483-009f6e200524 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:32:40.501076 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83041d35-5ef3-4495-b483-009f6e200524 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:32:40.502483 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83041d35-5ef3-4495-b483-009f6e200524 service cinder] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:32:40.512272 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 1260/2526] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Sep 13 03:32:40 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 03:32:40.561145 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51018202-5e0f-4fe6-a2b4-60872793e8ec None None] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:32:40.562555 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-51018202-5e0f-4fe6-a2b4-60872793e8ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:32:40.574914 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51018202-5e0f-4fe6-a2b4-60872793e8ec None None] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:32:40.578381 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a53afce7-4e9a-4b8f-a24d-5ad2b535c5b9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '924ee3f608d34a18b0ddc4695f102a56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b9d94f6a68f7470d9494f112175620dc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:32:40.579002 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a53afce7-4e9a-4b8f-a24d-5ad2b535c5b9 service cinder] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:32:40.579282 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a53afce7-4e9a-4b8f-a24d-5ad2b535c5b9 service cinder] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:32:40.579555 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a53afce7-4e9a-4b8f-a24d-5ad2b535c5b9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:32:40.623838 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:32:40.635953 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a53afce7-4e9a-4b8f-a24d-5ad2b535c5b9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:32:40.637386 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a53afce7-4e9a-4b8f-a24d-5ad2b535c5b9 service cinder] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:32:40.646092 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 1267/2527] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Sep 13 03:32:40 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 03:32:43.946598 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83041d35-5ef3-4495-b483-009f6e200524 service cinder] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:32:44.007565 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83041d35-5ef3-4495-b483-009f6e200524 service cinder] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:32:44.013006 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68b5fcd9-0c54-4885-ab48-3daa5d42c52e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76c5181fc7be45279ac00017bb4640b3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:32:44.013284 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68b5fcd9-0c54-4885-ab48-3daa5d42c52e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:32:44.013537 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68b5fcd9-0c54-4885-ab48-3daa5d42c52e admin admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:32:44.013804 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68b5fcd9-0c54-4885-ab48-3daa5d42c52e admin admin] PATH_INFO: `/v3/users/3f8f059a0fdf4e1da1f35f5220a7cde4` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:32:44.014942 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68b5fcd9-0c54-4885-ab48-3daa5d42c52e admin admin] RBAC: Authorizing `identity:delete_user(user_id=3f8f059a0fdf4e1da1f35f5220a7cde4)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:32:44.016844 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68b5fcd9-0c54-4885-ab48-3daa5d42c52e admin admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:32:44.062473 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-68b5fcd9-0c54-4885-ab48-3daa5d42c52e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3f8f059a0fdf4e1da1f35f5220a7cde4'} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:32:44.075301 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-68b5fcd9-0c54-4885-ab48-3daa5d42c52e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3f8f059a0fdf4e1da1f35f5220a7cde4'} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:32:44.076029 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-68b5fcd9-0c54-4885-ab48-3daa5d42c52e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3f8f059a0fdf4e1da1f35f5220a7cde4'} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:32:44.088301 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-68b5fcd9-0c54-4885-ab48-3daa5d42c52e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3f8f059a0fdf4e1da1f35f5220a7cde4'} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:32:44.095850 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-68b5fcd9-0c54-4885-ab48-3daa5d42c52e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3f8f059a0fdf4e1da1f35f5220a7cde4'} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:32:44.102847 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 1261/2528] 10.222.0.18 () {58 vars in 1224 bytes} [Tue Sep 13 03:32:43 2022] DELETE /identity/v3/users/3f8f059a0fdf4e1da1f35f5220a7cde4 => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 03:32:44.108664 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f9e3d1-8782-4328-a0d1-2ccae9f5c57c None None] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:32:44.109124 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f9e3d1-8782-4328-a0d1-2ccae9f5c57c None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:32:44.109230 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f9e3d1-8782-4328-a0d1-2ccae9f5c57c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:32:44.130664 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.auth.core [None req-00f9e3d1-8782-4328-a0d1-2ccae9f5c57c None None] MFA Rules not processed for user `49f280ab086b4610855c4ffa378517d9`. Rule list: `[]` (Enabled: `True`). {{(pid=81399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 03:32:44.131833 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-00f9e3d1-8782-4328-a0d1-2ccae9f5c57c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:32:44.187993 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 1268/2529] 10.222.0.18 () {58 vars in 958 bytes} [Tue Sep 13 03:32:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 13 03:32:44.198081 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68b5fcd9-0c54-4885-ab48-3daa5d42c52e admin admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:32:44.199475 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-68b5fcd9-0c54-4885-ab48-3daa5d42c52e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:32:44.205760 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:32:44.255255 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68b5fcd9-0c54-4885-ab48-3daa5d42c52e admin admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:32:44.258862 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e11b0a05-0864-42ae-9217-6f15bb4111df service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdffbc851086441ebd01022d1e4c2609', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b9d94f6a68f7470d9494f112175620dc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:32:44.259481 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e11b0a05-0864-42ae-9217-6f15bb4111df service neutron] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:32:44.260342 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e11b0a05-0864-42ae-9217-6f15bb4111df service neutron] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:32:44.260609 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e11b0a05-0864-42ae-9217-6f15bb4111df service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:32:44.270010 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e11b0a05-0864-42ae-9217-6f15bb4111df service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:32:44.271365 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e11b0a05-0864-42ae-9217-6f15bb4111df service neutron] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:32:44.281430 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 1262/2530] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Sep 13 03:32:44 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 03:32:44.545411 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00f9e3d1-8782-4328-a0d1-2ccae9f5c57c None None] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:32:44.547781 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-00f9e3d1-8782-4328-a0d1-2ccae9f5c57c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:32:44.565265 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00f9e3d1-8782-4328-a0d1-2ccae9f5c57c None None] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:32:44.572083 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0ccf101-2805-45b4-b9f1-6cebd12f0a8b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76c5181fc7be45279ac00017bb4640b3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:32:44.572671 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ccf101-2805-45b4-b9f1-6cebd12f0a8b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:32:44.572950 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ccf101-2805-45b4-b9f1-6cebd12f0a8b admin admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:32:44.573264 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ccf101-2805-45b4-b9f1-6cebd12f0a8b admin admin] PATH_INFO: `/v3/projects/385bcc0ea9454746a9d4b5c3bc8ac416` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:32:44.574657 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0ccf101-2805-45b4-b9f1-6cebd12f0a8b admin admin] RBAC: Authorizing `identity:delete_project(project_id=385bcc0ea9454746a9d4b5c3bc8ac416)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:32:44.576578 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0ccf101-2805-45b4-b9f1-6cebd12f0a8b admin admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:32:44.643298 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-d0ccf101-2805-45b4-b9f1-6cebd12f0a8b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '385bcc0ea9454746a9d4b5c3bc8ac416', 'request_id': 'req-d0ccf101-2805-45b4-b9f1-6cebd12f0a8b'} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:32:44.652656 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-d0ccf101-2805-45b4-b9f1-6cebd12f0a8b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '385bcc0ea9454746a9d4b5c3bc8ac416', 'request_id': 'req-d0ccf101-2805-45b4-b9f1-6cebd12f0a8b'} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:32:44.663875 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-d0ccf101-2805-45b4-b9f1-6cebd12f0a8b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '385bcc0ea9454746a9d4b5c3bc8ac416', 'request_id': 'req-d0ccf101-2805-45b4-b9f1-6cebd12f0a8b'} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:32:44.685049 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-d0ccf101-2805-45b4-b9f1-6cebd12f0a8b admin admin] The token cache is being invalidate because project 385bcc0ea9454746a9d4b5c3bc8ac416 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81399) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 03:32:44.685413 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-d0ccf101-2805-45b4-b9f1-6cebd12f0a8b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:32:44.686455 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-d0ccf101-2805-45b4-b9f1-6cebd12f0a8b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:32:44.688716 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 1269/2531] 10.222.0.18 () {58 vars in 1233 bytes} [Tue Sep 13 03:32:44 2022] DELETE /identity/v3/projects/385bcc0ea9454746a9d4b5c3bc8ac416 => generated 0 bytes in 144 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 13 03:32:44.695325 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e11b0a05-0864-42ae-9217-6f15bb4111df service neutron] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:32:44.706313 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e11b0a05-0864-42ae-9217-6f15bb4111df service neutron] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:32:44.781796 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c34636f2-d612-4555-9208-1a69726c5cf7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76c5181fc7be45279ac00017bb4640b3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:32:44.782425 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c34636f2-d612-4555-9208-1a69726c5cf7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:32:44.782732 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c34636f2-d612-4555-9208-1a69726c5cf7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:32:44.783001 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c34636f2-d612-4555-9208-1a69726c5cf7 admin admin] PATH_INFO: `/v3/users/a00d4283e9784101bdbc412decbb3384` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:32:44.787590 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c34636f2-d612-4555-9208-1a69726c5cf7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a00d4283e9784101bdbc412decbb3384)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:32:44.789891 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c34636f2-d612-4555-9208-1a69726c5cf7 admin admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:32:44.839681 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-c34636f2-d612-4555-9208-1a69726c5cf7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a00d4283e9784101bdbc412decbb3384'} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:32:44.847122 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-c34636f2-d612-4555-9208-1a69726c5cf7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a00d4283e9784101bdbc412decbb3384'} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:32:44.847754 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-c34636f2-d612-4555-9208-1a69726c5cf7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a00d4283e9784101bdbc412decbb3384'} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:32:44.861493 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-c34636f2-d612-4555-9208-1a69726c5cf7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a00d4283e9784101bdbc412decbb3384'} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:32:44.868500 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-c34636f2-d612-4555-9208-1a69726c5cf7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a00d4283e9784101bdbc412decbb3384'} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:32:44.874745 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 1263/2532] 10.222.0.18 () {58 vars in 1224 bytes} [Tue Sep 13 03:32:44 2022] DELETE /identity/v3/users/a00d4283e9784101bdbc412decbb3384 => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 03:32:45.144337 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0ccf101-2805-45b4-b9f1-6cebd12f0a8b admin admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:32:45.145837 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-d0ccf101-2805-45b4-b9f1-6cebd12f0a8b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:32:45.161673 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0ccf101-2805-45b4-b9f1-6cebd12f0a8b admin admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:32:45.219025 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6ac86ef-f7c7-400f-b0e8-36f516d5bb32 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76c5181fc7be45279ac00017bb4640b3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:32:45.219653 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ac86ef-f7c7-400f-b0e8-36f516d5bb32 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:32:45.219927 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ac86ef-f7c7-400f-b0e8-36f516d5bb32 admin admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:32:45.220199 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ac86ef-f7c7-400f-b0e8-36f516d5bb32 admin admin] PATH_INFO: `/v3/projects/f10d1a6d246f4e6eb137db7a3ec8f6f7` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:32:45.221317 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6ac86ef-f7c7-400f-b0e8-36f516d5bb32 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f10d1a6d246f4e6eb137db7a3ec8f6f7)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:32:45.223248 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6ac86ef-f7c7-400f-b0e8-36f516d5bb32 admin admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:32:45.234251 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:32:45.269416 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-a6ac86ef-f7c7-400f-b0e8-36f516d5bb32 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f10d1a6d246f4e6eb137db7a3ec8f6f7', 'request_id': 'req-a6ac86ef-f7c7-400f-b0e8-36f516d5bb32'} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:32:45.274878 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-a6ac86ef-f7c7-400f-b0e8-36f516d5bb32 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f10d1a6d246f4e6eb137db7a3ec8f6f7', 'request_id': 'req-a6ac86ef-f7c7-400f-b0e8-36f516d5bb32'} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:32:45.280920 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-a6ac86ef-f7c7-400f-b0e8-36f516d5bb32 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f10d1a6d246f4e6eb137db7a3ec8f6f7', 'request_id': 'req-a6ac86ef-f7c7-400f-b0e8-36f516d5bb32'} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:32:45.293792 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-a6ac86ef-f7c7-400f-b0e8-36f516d5bb32 admin admin] The token cache is being invalidate because project f10d1a6d246f4e6eb137db7a3ec8f6f7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81399) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 03:32:45.294085 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-a6ac86ef-f7c7-400f-b0e8-36f516d5bb32 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:32:45.294763 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-a6ac86ef-f7c7-400f-b0e8-36f516d5bb32 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:32:45.296236 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 1270/2533] 10.222.0.18 () {58 vars in 1233 bytes} [Tue Sep 13 03:32:45 2022] DELETE /identity/v3/projects/f10d1a6d246f4e6eb137db7a3ec8f6f7 => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 03:32:45.301544 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fda99d4-afd5-41e1-9389-71d05055b9da None None] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:32:45.302002 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fda99d4-afd5-41e1-9389-71d05055b9da None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:32:45.302288 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fda99d4-afd5-41e1-9389-71d05055b9da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:32:45.322210 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.auth.core [None req-7fda99d4-afd5-41e1-9389-71d05055b9da None None] MFA Rules not processed for user `49f280ab086b4610855c4ffa378517d9`. Rule list: `[]` (Enabled: `True`). {{(pid=81400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 03:32:45.323442 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-7fda99d4-afd5-41e1-9389-71d05055b9da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:32:45.343653 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:32:45.384367 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 1264/2534] 10.222.0.18 () {58 vars in 958 bytes} [Tue Sep 13 03:32:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 03:32:45.392332 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c9d2c12-c592-458f-bc0c-2388563ba335 None None] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:32:45.392620 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c9d2c12-c592-458f-bc0c-2388563ba335 None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:32:45.392895 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c9d2c12-c592-458f-bc0c-2388563ba335 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:32:45.412716 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.auth.core [None req-1c9d2c12-c592-458f-bc0c-2388563ba335 None None] MFA Rules not processed for user `49f280ab086b4610855c4ffa378517d9`. Rule list: `[]` (Enabled: `True`). {{(pid=81399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 03:32:45.413871 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-1c9d2c12-c592-458f-bc0c-2388563ba335 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:32:45.422098 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 1271/2535] 10.222.0.18 () {58 vars in 958 bytes} [Tue Sep 13 03:32:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 03:32:45.426203 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fda99d4-afd5-41e1-9389-71d05055b9da None None] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:32:45.435785 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fda99d4-afd5-41e1-9389-71d05055b9da None None] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:32:45.440319 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96c19a7f-c120-4e12-ab22-8c1ce0d07655 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76c5181fc7be45279ac00017bb4640b3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:32:45.440900 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c19a7f-c120-4e12-ab22-8c1ce0d07655 admin admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:32:45.441172 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c19a7f-c120-4e12-ab22-8c1ce0d07655 admin admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:32:45.441436 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c19a7f-c120-4e12-ab22-8c1ce0d07655 admin admin] PATH_INFO: `/v3/domains` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:32:45.442062 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96c19a7f-c120-4e12-ab22-8c1ce0d07655 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 03:32:45.442357 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96c19a7f-c120-4e12-ab22-8c1ce0d07655 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:32:45.444033 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96c19a7f-c120-4e12-ab22-8c1ce0d07655 admin admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:32:45.456470 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 1265/2536] 10.222.0.18 () {58 vars in 1153 bytes} [Tue Sep 13 03:32:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 03:32:45.460823 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c9d2c12-c592-458f-bc0c-2388563ba335 None None] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:32:45.469976 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c9d2c12-c592-458f-bc0c-2388563ba335 None None] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:32:45.474733 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a2111d4-de1b-4137-babe-8e51a1830f00 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76c5181fc7be45279ac00017bb4640b3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:32:45.475331 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a2111d4-de1b-4137-babe-8e51a1830f00 admin admin] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:32:45.475609 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a2111d4-de1b-4137-babe-8e51a1830f00 admin admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:32:45.475882 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a2111d4-de1b-4137-babe-8e51a1830f00 admin admin] PATH_INFO: `/v3/projects` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:32:45.476511 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a2111d4-de1b-4137-babe-8e51a1830f00 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:32:45.478415 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a2111d4-de1b-4137-babe-8e51a1830f00 admin admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:32:45.507973 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 1272/2537] 10.222.0.18 () {60 vars in 1153 bytes} [Tue Sep 13 03:32:45 2022] POST /identity/v3/projects => generated 387 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 13 03:32:45.512403 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96c19a7f-c120-4e12-ab22-8c1ce0d07655 admin admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:32:45.522241 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96c19a7f-c120-4e12-ab22-8c1ce0d07655 admin admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:32:45.594320 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02c3c6a5-c7d0-476d-babd-60fd422cadcb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76c5181fc7be45279ac00017bb4640b3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:32:45.595138 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02c3c6a5-c7d0-476d-babd-60fd422cadcb admin admin] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:32:45.595418 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02c3c6a5-c7d0-476d-babd-60fd422cadcb admin admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:32:45.595690 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02c3c6a5-c7d0-476d-babd-60fd422cadcb admin admin] PATH_INFO: `/v3/users` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:32:45.596319 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02c3c6a5-c7d0-476d-babd-60fd422cadcb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:32:45.598249 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02c3c6a5-c7d0-476d-babd-60fd422cadcb admin admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:32:45.623220 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 1266/2538] 10.222.0.18 () {60 vars in 1144 bytes} [Tue Sep 13 03:32:45 2022] POST /identity/v3/users => generated 294 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 03:32:45.627569 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a2111d4-de1b-4137-babe-8e51a1830f00 admin admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:32:45.637646 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a2111d4-de1b-4137-babe-8e51a1830f00 admin admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:32:45.642707 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38eefa43-53fb-4ddc-8a29-d62e5f338738 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76c5181fc7be45279ac00017bb4640b3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:32:45.643293 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38eefa43-53fb-4ddc-8a29-d62e5f338738 admin admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:32:45.643569 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38eefa43-53fb-4ddc-8a29-d62e5f338738 admin admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:32:45.643836 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38eefa43-53fb-4ddc-8a29-d62e5f338738 admin admin] PATH_INFO: `/v3/roles` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:32:45.644504 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38eefa43-53fb-4ddc-8a29-d62e5f338738 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:32:45.646078 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38eefa43-53fb-4ddc-8a29-d62e5f338738 admin admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:32:45.656422 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 1273/2539] 10.222.0.18 () {58 vars in 1122 bytes} [Tue Sep 13 03:32:45 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 03:32:45.660194 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02c3c6a5-c7d0-476d-babd-60fd422cadcb admin admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:32:45.670227 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02c3c6a5-c7d0-476d-babd-60fd422cadcb admin admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:32:45.674847 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77fde01a-c470-4ec7-a8f0-8577f37b9b65 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76c5181fc7be45279ac00017bb4640b3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:32:45.675434 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77fde01a-c470-4ec7-a8f0-8577f37b9b65 admin admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:32:45.675712 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77fde01a-c470-4ec7-a8f0-8577f37b9b65 admin admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:32:45.675989 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77fde01a-c470-4ec7-a8f0-8577f37b9b65 admin admin] PATH_INFO: `/v3/roles` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:32:45.676616 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77fde01a-c470-4ec7-a8f0-8577f37b9b65 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:32:45.678196 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77fde01a-c470-4ec7-a8f0-8577f37b9b65 admin admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:32:45.688610 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 1267/2540] 10.222.0.18 () {58 vars in 1122 bytes} [Tue Sep 13 03:32:45 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 13 03:32:45.692469 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38eefa43-53fb-4ddc-8a29-d62e5f338738 admin admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:32:45.702763 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38eefa43-53fb-4ddc-8a29-d62e5f338738 admin admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:32:45.707431 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e88d6753-4ad9-4856-8977-20d25584aaec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76c5181fc7be45279ac00017bb4640b3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:32:45.708066 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e88d6753-4ad9-4856-8977-20d25584aaec admin admin] REQUEST_METHOD: `PUT` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:32:45.708344 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e88d6753-4ad9-4856-8977-20d25584aaec admin admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:32:45.708617 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e88d6753-4ad9-4856-8977-20d25584aaec admin admin] PATH_INFO: `/v3/projects/21decde13aaa4c6aba32fa6a936d7b50/users/9fb1aa9194e7412596fbb9d813ff4312/roles/1eb7a5b5203945c29e6d22d653f47847` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:32:45.724168 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e88d6753-4ad9-4856-8977-20d25584aaec admin admin] RBAC: Authorizing `identity:create_grant(project_id=21decde13aaa4c6aba32fa6a936d7b50, user_id=9fb1aa9194e7412596fbb9d813ff4312, role_id=1eb7a5b5203945c29e6d22d653f47847)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:32:45.726908 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e88d6753-4ad9-4856-8977-20d25584aaec admin admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:32:45.739964 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 1274/2541] 10.222.0.18 () {60 vars in 1483 bytes} [Tue Sep 13 03:32:45 2022] PUT /identity/v3/projects/21decde13aaa4c6aba32fa6a936d7b50/users/9fb1aa9194e7412596fbb9d813ff4312/roles/1eb7a5b5203945c29e6d22d653f47847 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 03:32:45.750617 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7f0894b-9cfc-4380-a782-ea7806369737 None None] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:32:45.750903 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7f0894b-9cfc-4380-a782-ea7806369737 None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:32:45.751177 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7f0894b-9cfc-4380-a782-ea7806369737 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:32:45.780211 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.auth.core [None req-e7f0894b-9cfc-4380-a782-ea7806369737 None None] MFA Rules not processed for user `9fb1aa9194e7412596fbb9d813ff4312`. Rule list: `[]` (Enabled: `True`). {{(pid=81400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 03:32:45.781262 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-e7f0894b-9cfc-4380-a782-ea7806369737 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:32:45.847844 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 1268/2542] 10.222.0.18 () {58 vars in 958 bytes} [Tue Sep 13 03:32:45 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 03:32:45.854041 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e88d6753-4ad9-4856-8977-20d25584aaec admin admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:32:45.922107 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e88d6753-4ad9-4856-8977-20d25584aaec admin admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:32:45.928055 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3368d1c7-76eb-412e-9951-fa3ce32791f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76c5181fc7be45279ac00017bb4640b3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:32:45.928934 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3368d1c7-76eb-412e-9951-fa3ce32791f1 admin admin] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:32:45.929210 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3368d1c7-76eb-412e-9951-fa3ce32791f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:32:45.929473 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3368d1c7-76eb-412e-9951-fa3ce32791f1 admin admin] PATH_INFO: `/v3/projects` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:32:45.930102 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3368d1c7-76eb-412e-9951-fa3ce32791f1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:32:45.932220 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3368d1c7-76eb-412e-9951-fa3ce32791f1 admin admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:32:45.961064 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 1275/2543] 10.222.0.18 () {60 vars in 1153 bytes} [Tue Sep 13 03:32:45 2022] POST /identity/v3/projects => generated 387 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 03:32:45.965121 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7f0894b-9cfc-4380-a782-ea7806369737 None None] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:32:45.975949 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7f0894b-9cfc-4380-a782-ea7806369737 None None] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:32:45.994821 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:32:46.034131 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ca3fdbd-598e-48cc-b352-5400cdb4bb53 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76c5181fc7be45279ac00017bb4640b3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:32:46.034686 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca3fdbd-598e-48cc-b352-5400cdb4bb53 admin admin] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:32:46.034962 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca3fdbd-598e-48cc-b352-5400cdb4bb53 admin admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:32:46.035225 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca3fdbd-598e-48cc-b352-5400cdb4bb53 admin admin] PATH_INFO: `/v3/users` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:32:46.035852 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ca3fdbd-598e-48cc-b352-5400cdb4bb53 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:32:46.038081 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ca3fdbd-598e-48cc-b352-5400cdb4bb53 admin admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:32:46.059331 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 1269/2544] 10.222.0.18 () {60 vars in 1144 bytes} [Tue Sep 13 03:32:45 2022] POST /identity/v3/users => generated 300 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 03:32:46.063333 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3368d1c7-76eb-412e-9951-fa3ce32791f1 admin admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:32:46.073785 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3368d1c7-76eb-412e-9951-fa3ce32791f1 admin admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:32:46.078436 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88270dea-fa61-4733-82eb-e6f9b27cace3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76c5181fc7be45279ac00017bb4640b3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:32:46.079074 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88270dea-fa61-4733-82eb-e6f9b27cace3 admin admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:32:46.079351 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88270dea-fa61-4733-82eb-e6f9b27cace3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:32:46.079618 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88270dea-fa61-4733-82eb-e6f9b27cace3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:32:46.080257 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88270dea-fa61-4733-82eb-e6f9b27cace3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:32:46.081833 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88270dea-fa61-4733-82eb-e6f9b27cace3 admin admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:32:46.093248 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 1276/2545] 10.222.0.18 () {58 vars in 1122 bytes} [Tue Sep 13 03:32:46 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 03:32:46.097410 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ca3fdbd-598e-48cc-b352-5400cdb4bb53 admin admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:32:46.108396 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ca3fdbd-598e-48cc-b352-5400cdb4bb53 admin admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:32:46.112915 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69816672-ab1f-464e-9a41-b657273ff96a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76c5181fc7be45279ac00017bb4640b3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:32:46.113547 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69816672-ab1f-464e-9a41-b657273ff96a admin admin] REQUEST_METHOD: `PUT` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:32:46.113822 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69816672-ab1f-464e-9a41-b657273ff96a admin admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:32:46.114868 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69816672-ab1f-464e-9a41-b657273ff96a admin admin] PATH_INFO: `/v3/projects/fcfa08c42a4c4fe9b1aabd94c3b4b91f/users/b28a6f47425040958a87b252181edc83/roles/425d4b481c7846e0bc07e7e805d5b15f` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:32:46.130777 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69816672-ab1f-464e-9a41-b657273ff96a admin admin] RBAC: Authorizing `identity:create_grant(project_id=fcfa08c42a4c4fe9b1aabd94c3b4b91f, user_id=b28a6f47425040958a87b252181edc83, role_id=425d4b481c7846e0bc07e7e805d5b15f)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:32:46.133474 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69816672-ab1f-464e-9a41-b657273ff96a admin admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:32:46.146693 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 1270/2546] 10.222.0.18 () {60 vars in 1483 bytes} [Tue Sep 13 03:32:46 2022] PUT /identity/v3/projects/fcfa08c42a4c4fe9b1aabd94c3b4b91f/users/b28a6f47425040958a87b252181edc83/roles/425d4b481c7846e0bc07e7e805d5b15f => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 03:32:46.151052 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88270dea-fa61-4733-82eb-e6f9b27cace3 admin admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:32:46.161147 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88270dea-fa61-4733-82eb-e6f9b27cace3 admin admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:32:46.169872 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:32:46.221536 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02228a2d-7754-4c80-83d8-52ad5a6a9c39 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76c5181fc7be45279ac00017bb4640b3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:32:46.222153 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02228a2d-7754-4c80-83d8-52ad5a6a9c39 admin admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:32:46.222427 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02228a2d-7754-4c80-83d8-52ad5a6a9c39 admin admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:32:46.222727 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02228a2d-7754-4c80-83d8-52ad5a6a9c39 admin admin] PATH_INFO: `/v3/roles` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:32:46.223361 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02228a2d-7754-4c80-83d8-52ad5a6a9c39 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:32:46.224940 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02228a2d-7754-4c80-83d8-52ad5a6a9c39 admin admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:32:46.239387 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 1277/2547] 10.222.0.18 () {58 vars in 1122 bytes} [Tue Sep 13 03:32:46 2022] GET /identity/v3/roles => generated 1420 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 03:32:46.243589 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69816672-ab1f-464e-9a41-b657273ff96a admin admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:32:46.253905 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69816672-ab1f-464e-9a41-b657273ff96a admin admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:32:46.258558 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df6cfe5e-7ac7-4ce2-b442-9942b352b6c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76c5181fc7be45279ac00017bb4640b3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:32:46.259333 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df6cfe5e-7ac7-4ce2-b442-9942b352b6c0 admin admin] REQUEST_METHOD: `PUT` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:32:46.259641 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df6cfe5e-7ac7-4ce2-b442-9942b352b6c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:32:46.259909 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df6cfe5e-7ac7-4ce2-b442-9942b352b6c0 admin admin] PATH_INFO: `/v3/projects/fcfa08c42a4c4fe9b1aabd94c3b4b91f/users/b28a6f47425040958a87b252181edc83/roles/425d4b481c7846e0bc07e7e805d5b15f` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:32:46.261806 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df6cfe5e-7ac7-4ce2-b442-9942b352b6c0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fcfa08c42a4c4fe9b1aabd94c3b4b91f, user_id=b28a6f47425040958a87b252181edc83, role_id=425d4b481c7846e0bc07e7e805d5b15f)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:32:46.264929 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df6cfe5e-7ac7-4ce2-b442-9942b352b6c0 admin admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:32:46.274667 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 1271/2548] 10.222.0.18 () {60 vars in 1483 bytes} [Tue Sep 13 03:32:46 2022] PUT /identity/v3/projects/fcfa08c42a4c4fe9b1aabd94c3b4b91f/users/b28a6f47425040958a87b252181edc83/roles/425d4b481c7846e0bc07e7e805d5b15f => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 13 03:32:46.286123 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f08fb514-7fff-441c-a35b-fa3417452b40 None None] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:32:46.286406 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f08fb514-7fff-441c-a35b-fa3417452b40 None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:32:46.286861 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f08fb514-7fff-441c-a35b-fa3417452b40 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:32:46.317334 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.auth.core [None req-f08fb514-7fff-441c-a35b-fa3417452b40 None None] MFA Rules not processed for user `b28a6f47425040958a87b252181edc83`. Rule list: `[]` (Enabled: `True`). {{(pid=81399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 03:32:46.318538 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-f08fb514-7fff-441c-a35b-fa3417452b40 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:32:46.390477 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 1278/2549] 10.222.0.18 () {58 vars in 958 bytes} [Tue Sep 13 03:32:46 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 03:32:46.402175 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df6cfe5e-7ac7-4ce2-b442-9942b352b6c0 admin admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:32:46.404034 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-df6cfe5e-7ac7-4ce2-b442-9942b352b6c0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:32:46.465136 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df6cfe5e-7ac7-4ce2-b442-9942b352b6c0 admin admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:32:46.468770 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ded106b-c37c-4827-9ef7-f4cfd7036b43 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '924ee3f608d34a18b0ddc4695f102a56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b9d94f6a68f7470d9494f112175620dc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:32:46.469371 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ded106b-c37c-4827-9ef7-f4cfd7036b43 service cinder] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:32:46.469641 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ded106b-c37c-4827-9ef7-f4cfd7036b43 service cinder] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:32:46.469906 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ded106b-c37c-4827-9ef7-f4cfd7036b43 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:32:46.478730 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ded106b-c37c-4827-9ef7-f4cfd7036b43 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:32:46.480151 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ded106b-c37c-4827-9ef7-f4cfd7036b43 service cinder] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:32:46.489313 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 1272/2550] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Sep 13 03:32:46 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 03:32:46.554773 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f08fb514-7fff-441c-a35b-fa3417452b40 None None] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:32:46.616064 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f08fb514-7fff-441c-a35b-fa3417452b40 None None] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:32:46.620695 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca471921-34cd-4cde-8692-8d6a999409b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76c5181fc7be45279ac00017bb4640b3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:32:46.621314 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca471921-34cd-4cde-8692-8d6a999409b0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:32:46.621591 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca471921-34cd-4cde-8692-8d6a999409b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:32:46.621862 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca471921-34cd-4cde-8692-8d6a999409b0 admin admin] PATH_INFO: `/v3/users/9fb1aa9194e7412596fbb9d813ff4312` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:32:46.622996 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca471921-34cd-4cde-8692-8d6a999409b0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9fb1aa9194e7412596fbb9d813ff4312)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:32:46.625209 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca471921-34cd-4cde-8692-8d6a999409b0 admin admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:32:46.665375 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:32:46.675798 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-ca471921-34cd-4cde-8692-8d6a999409b0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9fb1aa9194e7412596fbb9d813ff4312'} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:32:46.683391 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-ca471921-34cd-4cde-8692-8d6a999409b0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9fb1aa9194e7412596fbb9d813ff4312'} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:32:46.695575 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-ca471921-34cd-4cde-8692-8d6a999409b0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9fb1aa9194e7412596fbb9d813ff4312'} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:32:46.696320 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-ca471921-34cd-4cde-8692-8d6a999409b0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9fb1aa9194e7412596fbb9d813ff4312'} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:32:46.705450 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-ca471921-34cd-4cde-8692-8d6a999409b0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9fb1aa9194e7412596fbb9d813ff4312'} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:32:46.712246 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 1279/2551] 10.222.0.18 () {58 vars in 1224 bytes} [Tue Sep 13 03:32:46 2022] DELETE /identity/v3/users/9fb1aa9194e7412596fbb9d813ff4312 => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 03:32:46.717048 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db2359cc-b949-44f1-a03b-6c7218e49b29 None None] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:32:46.717398 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db2359cc-b949-44f1-a03b-6c7218e49b29 None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:32:46.717738 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db2359cc-b949-44f1-a03b-6c7218e49b29 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:32:46.736850 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.auth.core [None req-db2359cc-b949-44f1-a03b-6c7218e49b29 None None] MFA Rules not processed for user `49f280ab086b4610855c4ffa378517d9`. Rule list: `[]` (Enabled: `True`). {{(pid=81400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 03:32:46.738041 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-db2359cc-b949-44f1-a03b-6c7218e49b29 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:32:46.804560 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 1273/2552] 10.222.0.18 () {58 vars in 958 bytes} [Tue Sep 13 03:32:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 03:32:46.814511 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca471921-34cd-4cde-8692-8d6a999409b0 admin admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:32:46.815928 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-ca471921-34cd-4cde-8692-8d6a999409b0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:32:46.875602 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca471921-34cd-4cde-8692-8d6a999409b0 admin admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:32:46.879303 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abbdd1cd-0072-42da-badd-082f790b4a5b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdffbc851086441ebd01022d1e4c2609', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b9d94f6a68f7470d9494f112175620dc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:32:46.879944 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abbdd1cd-0072-42da-badd-082f790b4a5b service neutron] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:32:46.880220 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abbdd1cd-0072-42da-badd-082f790b4a5b service neutron] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:32:46.880489 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abbdd1cd-0072-42da-badd-082f790b4a5b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:32:46.890520 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abbdd1cd-0072-42da-badd-082f790b4a5b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:32:46.892009 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abbdd1cd-0072-42da-badd-082f790b4a5b service neutron] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:32:46.902203 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 1280/2553] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Sep 13 03:32:46 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 03:32:47.257554 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db2359cc-b949-44f1-a03b-6c7218e49b29 None None] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:32:47.259159 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-db2359cc-b949-44f1-a03b-6c7218e49b29 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:32:47.272255 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db2359cc-b949-44f1-a03b-6c7218e49b29 None None] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:32:47.277158 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94997bcb-3ec1-435b-a0a0-e689004a5909 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76c5181fc7be45279ac00017bb4640b3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:32:47.277907 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94997bcb-3ec1-435b-a0a0-e689004a5909 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:32:47.278249 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94997bcb-3ec1-435b-a0a0-e689004a5909 admin admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:32:47.278629 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94997bcb-3ec1-435b-a0a0-e689004a5909 admin admin] PATH_INFO: `/v3/projects/21decde13aaa4c6aba32fa6a936d7b50` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:32:47.279759 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94997bcb-3ec1-435b-a0a0-e689004a5909 admin admin] RBAC: Authorizing `identity:delete_project(project_id=21decde13aaa4c6aba32fa6a936d7b50)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:32:47.281678 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94997bcb-3ec1-435b-a0a0-e689004a5909 admin admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:32:47.295429 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:32:47.331316 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-94997bcb-3ec1-435b-a0a0-e689004a5909 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '21decde13aaa4c6aba32fa6a936d7b50', 'request_id': 'req-94997bcb-3ec1-435b-a0a0-e689004a5909'} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:32:47.336747 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-94997bcb-3ec1-435b-a0a0-e689004a5909 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '21decde13aaa4c6aba32fa6a936d7b50', 'request_id': 'req-94997bcb-3ec1-435b-a0a0-e689004a5909'} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:32:47.342812 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-94997bcb-3ec1-435b-a0a0-e689004a5909 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '21decde13aaa4c6aba32fa6a936d7b50', 'request_id': 'req-94997bcb-3ec1-435b-a0a0-e689004a5909'} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:32:47.355121 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-94997bcb-3ec1-435b-a0a0-e689004a5909 admin admin] The token cache is being invalidate because project 21decde13aaa4c6aba32fa6a936d7b50 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81400) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 03:32:47.355404 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-94997bcb-3ec1-435b-a0a0-e689004a5909 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:32:47.356019 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-94997bcb-3ec1-435b-a0a0-e689004a5909 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:32:47.357491 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 1274/2554] 10.222.0.18 () {58 vars in 1233 bytes} [Tue Sep 13 03:32:47 2022] DELETE /identity/v3/projects/21decde13aaa4c6aba32fa6a936d7b50 => generated 0 bytes in 101 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 03:32:47.361571 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abbdd1cd-0072-42da-badd-082f790b4a5b service neutron] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:32:47.372073 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abbdd1cd-0072-42da-badd-082f790b4a5b service neutron] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:32:47.428527 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eb87518-aeff-4c70-aea2-51195795b9f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76c5181fc7be45279ac00017bb4640b3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:32:47.429172 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eb87518-aeff-4c70-aea2-51195795b9f4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:32:47.429444 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eb87518-aeff-4c70-aea2-51195795b9f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:32:47.429733 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eb87518-aeff-4c70-aea2-51195795b9f4 admin admin] PATH_INFO: `/v3/users/b28a6f47425040958a87b252181edc83` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:32:47.430862 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eb87518-aeff-4c70-aea2-51195795b9f4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b28a6f47425040958a87b252181edc83)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:32:47.433427 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eb87518-aeff-4c70-aea2-51195795b9f4 admin admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:32:47.476053 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-7eb87518-aeff-4c70-aea2-51195795b9f4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b28a6f47425040958a87b252181edc83'} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:32:47.483330 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-7eb87518-aeff-4c70-aea2-51195795b9f4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b28a6f47425040958a87b252181edc83'} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:32:47.485142 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=85,keystone:DELETE=13,keystone:INSERT=2 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:32:47.497232 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-7eb87518-aeff-4c70-aea2-51195795b9f4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b28a6f47425040958a87b252181edc83'} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:32:47.497816 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-7eb87518-aeff-4c70-aea2-51195795b9f4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b28a6f47425040958a87b252181edc83'} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:32:47.504909 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-7eb87518-aeff-4c70-aea2-51195795b9f4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b28a6f47425040958a87b252181edc83'} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:32:47.510637 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 1281/2555] 10.222.0.18 () {58 vars in 1224 bytes} [Tue Sep 13 03:32:47 2022] DELETE /identity/v3/users/b28a6f47425040958a87b252181edc83 => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 03:32:47.777513 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94997bcb-3ec1-435b-a0a0-e689004a5909 admin admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:32:47.779083 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-94997bcb-3ec1-435b-a0a0-e689004a5909 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:32:47.794268 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94997bcb-3ec1-435b-a0a0-e689004a5909 admin admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:32:47.854856 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32f94eb2-29de-4394-83a6-4d5b2e8e4bda admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76c5181fc7be45279ac00017bb4640b3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:32:47.855612 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f94eb2-29de-4394-83a6-4d5b2e8e4bda admin admin] REQUEST_METHOD: `DELETE` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:32:47.856014 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f94eb2-29de-4394-83a6-4d5b2e8e4bda admin admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:32:47.856355 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f94eb2-29de-4394-83a6-4d5b2e8e4bda admin admin] PATH_INFO: `/v3/projects/fcfa08c42a4c4fe9b1aabd94c3b4b91f` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:32:47.857546 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32f94eb2-29de-4394-83a6-4d5b2e8e4bda admin admin] RBAC: Authorizing `identity:delete_project(project_id=fcfa08c42a4c4fe9b1aabd94c3b4b91f)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:32:47.859599 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32f94eb2-29de-4394-83a6-4d5b2e8e4bda admin admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:32:47.908707 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-32f94eb2-29de-4394-83a6-4d5b2e8e4bda admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fcfa08c42a4c4fe9b1aabd94c3b4b91f', 'request_id': 'req-32f94eb2-29de-4394-83a6-4d5b2e8e4bda'} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:32:47.914585 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-32f94eb2-29de-4394-83a6-4d5b2e8e4bda admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fcfa08c42a4c4fe9b1aabd94c3b4b91f', 'request_id': 'req-32f94eb2-29de-4394-83a6-4d5b2e8e4bda'} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:32:47.920962 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-32f94eb2-29de-4394-83a6-4d5b2e8e4bda admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fcfa08c42a4c4fe9b1aabd94c3b4b91f', 'request_id': 'req-32f94eb2-29de-4394-83a6-4d5b2e8e4bda'} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:32:47.933616 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-32f94eb2-29de-4394-83a6-4d5b2e8e4bda admin admin] The token cache is being invalidate because project fcfa08c42a4c4fe9b1aabd94c3b4b91f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81400) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 03:32:47.933978 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-32f94eb2-29de-4394-83a6-4d5b2e8e4bda admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:32:47.934600 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-32f94eb2-29de-4394-83a6-4d5b2e8e4bda admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:32:47.936182 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 1275/2556] 10.222.0.18 () {58 vars in 1233 bytes} [Tue Sep 13 03:32:47 2022] DELETE /identity/v3/projects/fcfa08c42a4c4fe9b1aabd94c3b4b91f => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 03:32:47.941001 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22f0ba62-06c9-4627-9856-4ca80b073370 None None] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:32:47.941336 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22f0ba62-06c9-4627-9856-4ca80b073370 None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:32:47.941606 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22f0ba62-06c9-4627-9856-4ca80b073370 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:32:47.961760 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.auth.core [None req-22f0ba62-06c9-4627-9856-4ca80b073370 None None] MFA Rules not processed for user `49f280ab086b4610855c4ffa378517d9`. Rule list: `[]` (Enabled: `True`). {{(pid=81399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 03:32:47.963562 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-22f0ba62-06c9-4627-9856-4ca80b073370 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:32:48.022491 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 1282/2557] 10.222.0.18 () {58 vars in 958 bytes} [Tue Sep 13 03:32:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 03:32:48.030855 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84324e1f-4ab7-4230-8a20-9261fa23b8fa None None] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:32:48.031207 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84324e1f-4ab7-4230-8a20-9261fa23b8fa None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:32:48.031545 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84324e1f-4ab7-4230-8a20-9261fa23b8fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:32:48.051243 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.auth.core [None req-84324e1f-4ab7-4230-8a20-9261fa23b8fa None None] MFA Rules not processed for user `49f280ab086b4610855c4ffa378517d9`. Rule list: `[]` (Enabled: `True`). {{(pid=81400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 03:32:48.052410 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-84324e1f-4ab7-4230-8a20-9261fa23b8fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:32:48.060496 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 1276/2558] 10.222.0.18 () {58 vars in 958 bytes} [Tue Sep 13 03:32:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 03:32:48.064835 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22f0ba62-06c9-4627-9856-4ca80b073370 None None] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:32:48.075361 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22f0ba62-06c9-4627-9856-4ca80b073370 None None] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:32:48.080051 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9c0e736-fa44-487c-bc29-e444f12382dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76c5181fc7be45279ac00017bb4640b3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:32:48.080647 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9c0e736-fa44-487c-bc29-e444f12382dd admin admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:32:48.080951 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9c0e736-fa44-487c-bc29-e444f12382dd admin admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:32:48.081218 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9c0e736-fa44-487c-bc29-e444f12382dd admin admin] PATH_INFO: `/v3/domains` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:32:48.081852 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9c0e736-fa44-487c-bc29-e444f12382dd admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 03:32:48.082145 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9c0e736-fa44-487c-bc29-e444f12382dd admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:32:48.083784 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9c0e736-fa44-487c-bc29-e444f12382dd admin admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:32:48.096044 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 1283/2559] 10.222.0.18 () {58 vars in 1153 bytes} [Tue Sep 13 03:32:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 03:32:48.100309 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84324e1f-4ab7-4230-8a20-9261fa23b8fa None None] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:32:48.109400 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84324e1f-4ab7-4230-8a20-9261fa23b8fa None None] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:32:48.113871 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2071b1f-4e9f-4186-8e2c-20e80e7fd298 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76c5181fc7be45279ac00017bb4640b3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:32:48.114765 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2071b1f-4e9f-4186-8e2c-20e80e7fd298 admin admin] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:32:48.115041 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2071b1f-4e9f-4186-8e2c-20e80e7fd298 admin admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:32:48.115306 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2071b1f-4e9f-4186-8e2c-20e80e7fd298 admin admin] PATH_INFO: `/v3/projects` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:32:48.115925 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2071b1f-4e9f-4186-8e2c-20e80e7fd298 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:32:48.117842 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2071b1f-4e9f-4186-8e2c-20e80e7fd298 admin admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:32:48.144471 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 1277/2560] 10.222.0.18 () {60 vars in 1153 bytes} [Tue Sep 13 03:32:48 2022] POST /identity/v3/projects => generated 401 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 03:32:48.148472 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9c0e736-fa44-487c-bc29-e444f12382dd admin admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:32:48.159001 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9c0e736-fa44-487c-bc29-e444f12382dd admin admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:32:48.224937 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44f65ace-0854-4e5f-bbbb-f188c7f52470 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76c5181fc7be45279ac00017bb4640b3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:32:48.225542 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f65ace-0854-4e5f-bbbb-f188c7f52470 admin admin] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:32:48.225811 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f65ace-0854-4e5f-bbbb-f188c7f52470 admin admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:32:48.226075 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f65ace-0854-4e5f-bbbb-f188c7f52470 admin admin] PATH_INFO: `/v3/users` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:32:48.226750 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44f65ace-0854-4e5f-bbbb-f188c7f52470 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:32:48.228644 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44f65ace-0854-4e5f-bbbb-f188c7f52470 admin admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:32:48.251401 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 1284/2561] 10.222.0.18 () {60 vars in 1144 bytes} [Tue Sep 13 03:32:48 2022] POST /identity/v3/users => generated 301 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 13 03:32:48.255667 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2071b1f-4e9f-4186-8e2c-20e80e7fd298 admin admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:32:48.261912 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=83,keystone:DELETE=14,keystone:INSERT=3 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:32:48.268577 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2071b1f-4e9f-4186-8e2c-20e80e7fd298 admin admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:32:48.274409 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4f248b3-4279-43fb-9e9e-0b14e8def9c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76c5181fc7be45279ac00017bb4640b3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:32:48.275806 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4f248b3-4279-43fb-9e9e-0b14e8def9c7 admin admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:32:48.276148 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4f248b3-4279-43fb-9e9e-0b14e8def9c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:32:48.276482 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4f248b3-4279-43fb-9e9e-0b14e8def9c7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:32:48.277186 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4f248b3-4279-43fb-9e9e-0b14e8def9c7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:32:48.278879 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4f248b3-4279-43fb-9e9e-0b14e8def9c7 admin admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:32:48.289305 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 1278/2562] 10.222.0.18 () {58 vars in 1122 bytes} [Tue Sep 13 03:32:48 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 03:32:48.293506 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44f65ace-0854-4e5f-bbbb-f188c7f52470 admin admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:32:48.303604 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44f65ace-0854-4e5f-bbbb-f188c7f52470 admin admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:32:48.309231 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6adb32a7-7dfb-4c8c-915b-68d47857ac4e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76c5181fc7be45279ac00017bb4640b3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:32:48.309839 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6adb32a7-7dfb-4c8c-915b-68d47857ac4e admin admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:32:48.310141 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6adb32a7-7dfb-4c8c-915b-68d47857ac4e admin admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:32:48.310410 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6adb32a7-7dfb-4c8c-915b-68d47857ac4e admin admin] PATH_INFO: `/v3/roles` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:32:48.311076 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6adb32a7-7dfb-4c8c-915b-68d47857ac4e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:32:48.312659 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6adb32a7-7dfb-4c8c-915b-68d47857ac4e admin admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:32:48.324402 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 1285/2563] 10.222.0.18 () {58 vars in 1122 bytes} [Tue Sep 13 03:32:48 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 03:32:48.328619 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4f248b3-4279-43fb-9e9e-0b14e8def9c7 admin admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:32:48.332560 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6adb32a7-7dfb-4c8c-915b-68d47857ac4e admin admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:32:48.334739 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-6adb32a7-7dfb-4c8c-915b-68d47857ac4e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:32:48.338993 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4f248b3-4279-43fb-9e9e-0b14e8def9c7 admin admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:32:48.344433 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26c944bf-0623-433a-bc59-2e808b7d7f57 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76c5181fc7be45279ac00017bb4640b3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:32:48.345092 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c944bf-0623-433a-bc59-2e808b7d7f57 admin admin] REQUEST_METHOD: `PUT` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:32:48.345375 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c944bf-0623-433a-bc59-2e808b7d7f57 admin admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:32:48.345648 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c944bf-0623-433a-bc59-2e808b7d7f57 admin admin] PATH_INFO: `/v3/projects/fc52403c02bd4d24afdee02860fc46ee/users/d5d36294cf714e8f894ead1f728723c8/roles/1eb7a5b5203945c29e6d22d653f47847` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:32:48.361528 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26c944bf-0623-433a-bc59-2e808b7d7f57 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fc52403c02bd4d24afdee02860fc46ee, user_id=d5d36294cf714e8f894ead1f728723c8, role_id=1eb7a5b5203945c29e6d22d653f47847)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:32:48.364183 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26c944bf-0623-433a-bc59-2e808b7d7f57 admin admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:32:48.382711 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 1279/2564] 10.222.0.18 () {60 vars in 1483 bytes} [Tue Sep 13 03:32:48 2022] PUT /identity/v3/projects/fc52403c02bd4d24afdee02860fc46ee/users/d5d36294cf714e8f894ead1f728723c8/roles/1eb7a5b5203945c29e6d22d653f47847 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 03:32:48.386061 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=95,keystone:DELETE=2,keystone:INSERT=3 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:32:48.398047 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e4e0fe-29df-42fd-a39d-5202580f7ff3 None None] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:32:48.398422 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e4e0fe-29df-42fd-a39d-5202580f7ff3 None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:32:48.398779 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e4e0fe-29df-42fd-a39d-5202580f7ff3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:32:48.425622 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6adb32a7-7dfb-4c8c-915b-68d47857ac4e admin admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:32:48.434211 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.auth.core [None req-c9e4e0fe-29df-42fd-a39d-5202580f7ff3 None None] MFA Rules not processed for user `d5d36294cf714e8f894ead1f728723c8`. Rule list: `[]` (Enabled: `True`). {{(pid=81400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 03:32:48.435428 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-c9e4e0fe-29df-42fd-a39d-5202580f7ff3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:32:48.488780 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62d3e56e-090f-46e8-82fc-f6620810d26a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7377f5bee4a49959182e1bd34bc97a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b9d94f6a68f7470d9494f112175620dc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:32:48.489392 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d3e56e-090f-46e8-82fc-f6620810d26a service nova] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:32:48.489670 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d3e56e-090f-46e8-82fc-f6620810d26a service nova] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:32:48.489941 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d3e56e-090f-46e8-82fc-f6620810d26a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:32:48.491735 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-62d3e56e-090f-46e8-82fc-f6620810d26a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:32:48.500961 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 1280/2565] 10.222.0.18 () {58 vars in 958 bytes} [Tue Sep 13 03:32:48 2022] POST /identity/v3/auth/tokens => generated 3278 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 03:32:48.504207 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62d3e56e-090f-46e8-82fc-f6620810d26a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:32:48.505573 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9e4e0fe-29df-42fd-a39d-5202580f7ff3 None None] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:32:48.505784 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62d3e56e-090f-46e8-82fc-f6620810d26a service nova] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:32:48.517073 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 1286/2566] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Sep 13 03:32:48 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 185 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 13 03:32:48.570349 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9e4e0fe-29df-42fd-a39d-5202580f7ff3 None None] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:32:48.575520 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dc2e079-7a5a-49ee-8db0-eb5e7e23a39a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76c5181fc7be45279ac00017bb4640b3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:32:48.576122 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc2e079-7a5a-49ee-8db0-eb5e7e23a39a admin admin] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:32:48.576396 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc2e079-7a5a-49ee-8db0-eb5e7e23a39a admin admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:32:48.576661 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc2e079-7a5a-49ee-8db0-eb5e7e23a39a admin admin] PATH_INFO: `/v3/projects` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:32:48.577291 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dc2e079-7a5a-49ee-8db0-eb5e7e23a39a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:32:48.579233 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dc2e079-7a5a-49ee-8db0-eb5e7e23a39a admin admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:32:48.609032 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 1281/2567] 10.222.0.18 () {60 vars in 1153 bytes} [Tue Sep 13 03:32:48 2022] POST /identity/v3/projects => generated 401 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 13 03:32:48.610109 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62d3e56e-090f-46e8-82fc-f6620810d26a service nova] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:32:48.622022 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62d3e56e-090f-46e8-82fc-f6620810d26a service nova] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:32:48.678150 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec93d801-290f-4a33-a0b0-3434c0737058 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76c5181fc7be45279ac00017bb4640b3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:32:48.678789 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec93d801-290f-4a33-a0b0-3434c0737058 admin admin] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:32:48.679062 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec93d801-290f-4a33-a0b0-3434c0737058 admin admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:32:48.679328 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec93d801-290f-4a33-a0b0-3434c0737058 admin admin] PATH_INFO: `/v3/users` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:32:48.679937 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec93d801-290f-4a33-a0b0-3434c0737058 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:32:48.682475 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec93d801-290f-4a33-a0b0-3434c0737058 admin admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:32:48.702993 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 1287/2568] 10.222.0.18 () {60 vars in 1144 bytes} [Tue Sep 13 03:32:48 2022] POST /identity/v3/users => generated 307 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 03:32:48.706964 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dc2e079-7a5a-49ee-8db0-eb5e7e23a39a admin admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:32:48.717376 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dc2e079-7a5a-49ee-8db0-eb5e7e23a39a admin admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:32:48.722803 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d70a531-b780-427a-a1af-4f97d0f1cc7c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76c5181fc7be45279ac00017bb4640b3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:32:48.723566 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d70a531-b780-427a-a1af-4f97d0f1cc7c admin admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:32:48.723848 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d70a531-b780-427a-a1af-4f97d0f1cc7c admin admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:32:48.724121 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d70a531-b780-427a-a1af-4f97d0f1cc7c admin admin] PATH_INFO: `/v3/roles` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:32:48.724765 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d70a531-b780-427a-a1af-4f97d0f1cc7c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:32:48.726465 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d70a531-b780-427a-a1af-4f97d0f1cc7c admin admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:32:48.740018 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 1282/2569] 10.222.0.18 () {58 vars in 1122 bytes} [Tue Sep 13 03:32:48 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 03:32:48.743738 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec93d801-290f-4a33-a0b0-3434c0737058 admin admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:32:48.755010 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec93d801-290f-4a33-a0b0-3434c0737058 admin admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:32:48.759547 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc67261a-e783-497a-9778-c93a597fb60b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76c5181fc7be45279ac00017bb4640b3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:32:48.760224 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc67261a-e783-497a-9778-c93a597fb60b admin admin] REQUEST_METHOD: `PUT` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:32:48.761235 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc67261a-e783-497a-9778-c93a597fb60b admin admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:32:48.761509 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc67261a-e783-497a-9778-c93a597fb60b admin admin] PATH_INFO: `/v3/projects/53293930b4d84b9d921becaaf81e17cc/users/34bbe61401c94d4c88d0fd3468ea615e/roles/425d4b481c7846e0bc07e7e805d5b15f` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:32:48.777318 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc67261a-e783-497a-9778-c93a597fb60b admin admin] RBAC: Authorizing `identity:create_grant(project_id=53293930b4d84b9d921becaaf81e17cc, user_id=34bbe61401c94d4c88d0fd3468ea615e, role_id=425d4b481c7846e0bc07e7e805d5b15f)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:32:48.780317 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc67261a-e783-497a-9778-c93a597fb60b admin admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:32:48.795103 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 1288/2570] 10.222.0.18 () {60 vars in 1483 bytes} [Tue Sep 13 03:32:48 2022] PUT /identity/v3/projects/53293930b4d84b9d921becaaf81e17cc/users/34bbe61401c94d4c88d0fd3468ea615e/roles/425d4b481c7846e0bc07e7e805d5b15f => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 03:32:48.799171 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d70a531-b780-427a-a1af-4f97d0f1cc7c admin admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:32:48.810080 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d70a531-b780-427a-a1af-4f97d0f1cc7c admin admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:32:48.838724 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:32:48.870162 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-666c8895-8c2a-48b2-b722-c7d6dcac220f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76c5181fc7be45279ac00017bb4640b3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:32:48.870712 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-666c8895-8c2a-48b2-b722-c7d6dcac220f admin admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:32:48.870987 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-666c8895-8c2a-48b2-b722-c7d6dcac220f admin admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:32:48.871253 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-666c8895-8c2a-48b2-b722-c7d6dcac220f admin admin] PATH_INFO: `/v3/roles` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:32:48.871914 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-666c8895-8c2a-48b2-b722-c7d6dcac220f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:32:48.873505 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-666c8895-8c2a-48b2-b722-c7d6dcac220f admin admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:32:48.883722 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 1283/2571] 10.222.0.18 () {58 vars in 1122 bytes} [Tue Sep 13 03:32:48 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 03:32:48.887869 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc67261a-e783-497a-9778-c93a597fb60b admin admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:32:48.897991 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc67261a-e783-497a-9778-c93a597fb60b admin admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:32:48.902564 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0587ec2-7b8a-413e-a0d9-00ee86634b3f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76c5181fc7be45279ac00017bb4640b3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:32:48.903225 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0587ec2-7b8a-413e-a0d9-00ee86634b3f admin admin] REQUEST_METHOD: `PUT` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:32:48.903499 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0587ec2-7b8a-413e-a0d9-00ee86634b3f admin admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:32:48.903767 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0587ec2-7b8a-413e-a0d9-00ee86634b3f admin admin] PATH_INFO: `/v3/projects/53293930b4d84b9d921becaaf81e17cc/users/34bbe61401c94d4c88d0fd3468ea615e/roles/425d4b481c7846e0bc07e7e805d5b15f` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:32:48.905621 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0587ec2-7b8a-413e-a0d9-00ee86634b3f admin admin] RBAC: Authorizing `identity:create_grant(project_id=53293930b4d84b9d921becaaf81e17cc, user_id=34bbe61401c94d4c88d0fd3468ea615e, role_id=425d4b481c7846e0bc07e7e805d5b15f)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:32:48.908259 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0587ec2-7b8a-413e-a0d9-00ee86634b3f admin admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:32:48.918973 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 1289/2572] 10.222.0.18 () {60 vars in 1483 bytes} [Tue Sep 13 03:32:48 2022] PUT /identity/v3/projects/53293930b4d84b9d921becaaf81e17cc/users/34bbe61401c94d4c88d0fd3468ea615e/roles/425d4b481c7846e0bc07e7e805d5b15f => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 03:32:48.929817 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99de6150-a65b-4445-9a8f-30381991212c None None] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:32:48.930110 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99de6150-a65b-4445-9a8f-30381991212c None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:32:48.930428 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99de6150-a65b-4445-9a8f-30381991212c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:32:48.958634 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.auth.core [None req-99de6150-a65b-4445-9a8f-30381991212c None None] MFA Rules not processed for user `34bbe61401c94d4c88d0fd3468ea615e`. Rule list: `[]` (Enabled: `True`). {{(pid=81400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 03:32:48.959867 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-99de6150-a65b-4445-9a8f-30381991212c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:32:49.029471 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 1284/2573] 10.222.0.18 () {58 vars in 958 bytes} [Tue Sep 13 03:32:48 2022] POST /identity/v3/auth/tokens => generated 3345 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 03:32:49.040697 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0587ec2-7b8a-413e-a0d9-00ee86634b3f admin admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:32:49.042464 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-c0587ec2-7b8a-413e-a0d9-00ee86634b3f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:32:49.061730 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:32:49.106680 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0587ec2-7b8a-413e-a0d9-00ee86634b3f admin admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:32:49.110277 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1be5e9fa-40d2-44a4-aa75-e549a288d8e4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '924ee3f608d34a18b0ddc4695f102a56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b9d94f6a68f7470d9494f112175620dc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:32:49.110906 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be5e9fa-40d2-44a4-aa75-e549a288d8e4 service cinder] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:32:49.111182 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be5e9fa-40d2-44a4-aa75-e549a288d8e4 service cinder] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:32:49.111447 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be5e9fa-40d2-44a4-aa75-e549a288d8e4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:32:49.120988 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1be5e9fa-40d2-44a4-aa75-e549a288d8e4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:32:49.122401 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1be5e9fa-40d2-44a4-aa75-e549a288d8e4 service cinder] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:32:49.132740 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 1290/2574] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Sep 13 03:32:49 2022] GET /identity/v3/auth/tokens => generated 3345 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 03:32:49.210164 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99de6150-a65b-4445-9a8f-30381991212c None None] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:32:49.219751 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99de6150-a65b-4445-9a8f-30381991212c None None] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:32:49.223369 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0f3e815-1cc5-44c2-9c94-694c95287503 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '924ee3f608d34a18b0ddc4695f102a56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b9d94f6a68f7470d9494f112175620dc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:32:49.223962 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0f3e815-1cc5-44c2-9c94-694c95287503 service cinder] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:32:49.224240 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0f3e815-1cc5-44c2-9c94-694c95287503 service cinder] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:32:49.224513 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0f3e815-1cc5-44c2-9c94-694c95287503 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:32:49.280077 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0f3e815-1cc5-44c2-9c94-694c95287503 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:32:49.281514 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0f3e815-1cc5-44c2-9c94-694c95287503 service cinder] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:32:49.291018 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 1285/2575] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Sep 13 03:32:49 2022] GET /identity/v3/auth/tokens => generated 3278 bytes in 81 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 03:32:51.683840 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1be5e9fa-40d2-44a4-aa75-e549a288d8e4 service cinder] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:32:51.685291 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-1be5e9fa-40d2-44a4-aa75-e549a288d8e4 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:32:51.740730 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1be5e9fa-40d2-44a4-aa75-e549a288d8e4 service cinder] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:32:51.744310 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0327cb9a-7870-4123-9664-b1eb50e9bde9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdffbc851086441ebd01022d1e4c2609', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b9d94f6a68f7470d9494f112175620dc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:32:51.744900 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0327cb9a-7870-4123-9664-b1eb50e9bde9 service neutron] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:32:51.745173 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0327cb9a-7870-4123-9664-b1eb50e9bde9 service neutron] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:32:51.745438 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0327cb9a-7870-4123-9664-b1eb50e9bde9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:32:51.747156 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-0327cb9a-7870-4123-9664-b1eb50e9bde9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:32:51.807634 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0327cb9a-7870-4123-9664-b1eb50e9bde9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:32:51.809065 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0327cb9a-7870-4123-9664-b1eb50e9bde9 service neutron] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:32:51.818250 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 1291/2576] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Sep 13 03:32:51 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 135 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 03:32:53.347801 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0f3e815-1cc5-44c2-9c94-694c95287503 service cinder] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:32:53.405279 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:32:53.419765 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0f3e815-1cc5-44c2-9c94-694c95287503 service cinder] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:32:53.425011 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8fbfc7b-e166-46fe-97d9-16328ddb1690 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76c5181fc7be45279ac00017bb4640b3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:32:53.425872 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8fbfc7b-e166-46fe-97d9-16328ddb1690 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:32:53.427571 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8fbfc7b-e166-46fe-97d9-16328ddb1690 admin admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:32:53.427702 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8fbfc7b-e166-46fe-97d9-16328ddb1690 admin admin] PATH_INFO: `/v3/users/d5d36294cf714e8f894ead1f728723c8` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:32:53.429183 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8fbfc7b-e166-46fe-97d9-16328ddb1690 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d5d36294cf714e8f894ead1f728723c8)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:32:53.431912 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8fbfc7b-e166-46fe-97d9-16328ddb1690 admin admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:32:53.483191 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-d8fbfc7b-e166-46fe-97d9-16328ddb1690 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd5d36294cf714e8f894ead1f728723c8'} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:32:53.493690 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-d8fbfc7b-e166-46fe-97d9-16328ddb1690 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd5d36294cf714e8f894ead1f728723c8'} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:32:53.494343 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-d8fbfc7b-e166-46fe-97d9-16328ddb1690 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd5d36294cf714e8f894ead1f728723c8'} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:32:53.508696 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-d8fbfc7b-e166-46fe-97d9-16328ddb1690 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd5d36294cf714e8f894ead1f728723c8'} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:32:53.516224 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-d8fbfc7b-e166-46fe-97d9-16328ddb1690 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd5d36294cf714e8f894ead1f728723c8'} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:32:53.523684 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 1286/2577] 10.222.0.18 () {58 vars in 1224 bytes} [Tue Sep 13 03:32:53 2022] DELETE /identity/v3/users/d5d36294cf714e8f894ead1f728723c8 => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 13 03:32:53.528790 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0581054-15bf-4634-921e-8b2fd9040305 None None] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:32:53.529292 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0581054-15bf-4634-921e-8b2fd9040305 None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:32:53.529409 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0581054-15bf-4634-921e-8b2fd9040305 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:32:53.550975 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.auth.core [None req-c0581054-15bf-4634-921e-8b2fd9040305 None None] MFA Rules not processed for user `49f280ab086b4610855c4ffa378517d9`. Rule list: `[]` (Enabled: `True`). {{(pid=81399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 03:32:53.552419 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-c0581054-15bf-4634-921e-8b2fd9040305 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:32:53.635456 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 1292/2578] 10.222.0.18 () {58 vars in 958 bytes} [Tue Sep 13 03:32:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 03:32:53.642382 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8fbfc7b-e166-46fe-97d9-16328ddb1690 admin admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:32:53.643899 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-d8fbfc7b-e166-46fe-97d9-16328ddb1690 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:32:53.727881 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8fbfc7b-e166-46fe-97d9-16328ddb1690 admin admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:32:53.732797 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27a83433-d962-48eb-af06-6270cb130b76 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdffbc851086441ebd01022d1e4c2609', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b9d94f6a68f7470d9494f112175620dc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:32:53.733500 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27a83433-d962-48eb-af06-6270cb130b76 service neutron] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:32:53.733853 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27a83433-d962-48eb-af06-6270cb130b76 service neutron] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:32:53.734194 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27a83433-d962-48eb-af06-6270cb130b76 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:32:53.744552 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27a83433-d962-48eb-af06-6270cb130b76 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:32:53.746087 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27a83433-d962-48eb-af06-6270cb130b76 service neutron] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:32:53.756270 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 1287/2579] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Sep 13 03:32:53 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 03:32:53.768263 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0581054-15bf-4634-921e-8b2fd9040305 None None] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:32:53.769790 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-c0581054-15bf-4634-921e-8b2fd9040305 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:32:53.783188 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:32:53.871810 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0581054-15bf-4634-921e-8b2fd9040305 None None] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:32:53.878327 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9cc8293-0dfe-4b12-9a1d-2565a7a13c7c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7377f5bee4a49959182e1bd34bc97a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b9d94f6a68f7470d9494f112175620dc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:32:53.878990 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9cc8293-0dfe-4b12-9a1d-2565a7a13c7c service nova] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:32:53.879272 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9cc8293-0dfe-4b12-9a1d-2565a7a13c7c service nova] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:32:53.879545 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9cc8293-0dfe-4b12-9a1d-2565a7a13c7c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:32:53.881347 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-b9cc8293-0dfe-4b12-9a1d-2565a7a13c7c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:32:53.962725 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9cc8293-0dfe-4b12-9a1d-2565a7a13c7c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:32:53.964942 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9cc8293-0dfe-4b12-9a1d-2565a7a13c7c service nova] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:32:53.986351 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 1293/2580] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Sep 13 03:32:53 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 218 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 03:32:54.198843 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27a83433-d962-48eb-af06-6270cb130b76 service neutron] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:32:54.201577 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-27a83433-d962-48eb-af06-6270cb130b76 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:32:54.232675 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27a83433-d962-48eb-af06-6270cb130b76 service neutron] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:32:54.241743 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-426cd3c0-5d0a-4ea8-a4b5-6290d87da07c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76c5181fc7be45279ac00017bb4640b3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:32:54.242384 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-426cd3c0-5d0a-4ea8-a4b5-6290d87da07c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:32:54.242711 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-426cd3c0-5d0a-4ea8-a4b5-6290d87da07c admin admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:32:54.242981 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-426cd3c0-5d0a-4ea8-a4b5-6290d87da07c admin admin] PATH_INFO: `/v3/projects/fc52403c02bd4d24afdee02860fc46ee` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:32:54.245686 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-426cd3c0-5d0a-4ea8-a4b5-6290d87da07c admin admin] RBAC: Authorizing `identity:delete_project(project_id=fc52403c02bd4d24afdee02860fc46ee)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:32:54.248109 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-426cd3c0-5d0a-4ea8-a4b5-6290d87da07c admin admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:32:54.324336 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-426cd3c0-5d0a-4ea8-a4b5-6290d87da07c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fc52403c02bd4d24afdee02860fc46ee', 'request_id': 'req-426cd3c0-5d0a-4ea8-a4b5-6290d87da07c'} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:32:54.330326 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-426cd3c0-5d0a-4ea8-a4b5-6290d87da07c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fc52403c02bd4d24afdee02860fc46ee', 'request_id': 'req-426cd3c0-5d0a-4ea8-a4b5-6290d87da07c'} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:32:54.337774 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-426cd3c0-5d0a-4ea8-a4b5-6290d87da07c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fc52403c02bd4d24afdee02860fc46ee', 'request_id': 'req-426cd3c0-5d0a-4ea8-a4b5-6290d87da07c'} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:32:54.343072 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:32:54.361694 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-426cd3c0-5d0a-4ea8-a4b5-6290d87da07c admin admin] The token cache is being invalidate because project fc52403c02bd4d24afdee02860fc46ee was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81400) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 03:32:54.362285 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-426cd3c0-5d0a-4ea8-a4b5-6290d87da07c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:32:54.362978 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-426cd3c0-5d0a-4ea8-a4b5-6290d87da07c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:32:54.367673 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 1288/2581] 10.222.0.18 () {58 vars in 1233 bytes} [Tue Sep 13 03:32:54 2022] DELETE /identity/v3/projects/fc52403c02bd4d24afdee02860fc46ee => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 03:32:54.368908 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9cc8293-0dfe-4b12-9a1d-2565a7a13c7c service nova] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:32:54.380406 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9cc8293-0dfe-4b12-9a1d-2565a7a13c7c service nova] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:32:54.444418 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd356e9e-6125-4ec1-acd9-584e0eccaa3f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76c5181fc7be45279ac00017bb4640b3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:32:54.445058 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd356e9e-6125-4ec1-acd9-584e0eccaa3f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:32:54.445329 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd356e9e-6125-4ec1-acd9-584e0eccaa3f admin admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:32:54.445593 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd356e9e-6125-4ec1-acd9-584e0eccaa3f admin admin] PATH_INFO: `/v3/users/34bbe61401c94d4c88d0fd3468ea615e` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:32:54.446817 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd356e9e-6125-4ec1-acd9-584e0eccaa3f admin admin] RBAC: Authorizing `identity:delete_user(user_id=34bbe61401c94d4c88d0fd3468ea615e)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:32:54.448782 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd356e9e-6125-4ec1-acd9-584e0eccaa3f admin admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:32:54.494801 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:32:54.502301 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-cd356e9e-6125-4ec1-acd9-584e0eccaa3f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '34bbe61401c94d4c88d0fd3468ea615e'} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:32:54.509980 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-cd356e9e-6125-4ec1-acd9-584e0eccaa3f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '34bbe61401c94d4c88d0fd3468ea615e'} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:32:54.525665 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-cd356e9e-6125-4ec1-acd9-584e0eccaa3f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '34bbe61401c94d4c88d0fd3468ea615e'} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:32:54.526965 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-cd356e9e-6125-4ec1-acd9-584e0eccaa3f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '34bbe61401c94d4c88d0fd3468ea615e'} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:32:54.533826 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-cd356e9e-6125-4ec1-acd9-584e0eccaa3f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '34bbe61401c94d4c88d0fd3468ea615e'} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:32:54.539975 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 1294/2582] 10.222.0.18 () {58 vars in 1224 bytes} [Tue Sep 13 03:32:54 2022] DELETE /identity/v3/users/34bbe61401c94d4c88d0fd3468ea615e => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 03:32:54.829683 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-426cd3c0-5d0a-4ea8-a4b5-6290d87da07c admin admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:32:54.831561 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-426cd3c0-5d0a-4ea8-a4b5-6290d87da07c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:32:54.845959 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-426cd3c0-5d0a-4ea8-a4b5-6290d87da07c admin admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:32:54.906268 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8b6bf86-5253-47c2-a7e5-c68f45d448c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76c5181fc7be45279ac00017bb4640b3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:32:54.906967 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b6bf86-5253-47c2-a7e5-c68f45d448c6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:32:54.907242 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b6bf86-5253-47c2-a7e5-c68f45d448c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:32:54.907513 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b6bf86-5253-47c2-a7e5-c68f45d448c6 admin admin] PATH_INFO: `/v3/projects/53293930b4d84b9d921becaaf81e17cc` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:32:54.908632 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8b6bf86-5253-47c2-a7e5-c68f45d448c6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=53293930b4d84b9d921becaaf81e17cc)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:32:54.910940 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8b6bf86-5253-47c2-a7e5-c68f45d448c6 admin admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:32:54.957414 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-e8b6bf86-5253-47c2-a7e5-c68f45d448c6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '53293930b4d84b9d921becaaf81e17cc', 'request_id': 'req-e8b6bf86-5253-47c2-a7e5-c68f45d448c6'} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:32:54.963997 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-e8b6bf86-5253-47c2-a7e5-c68f45d448c6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '53293930b4d84b9d921becaaf81e17cc', 'request_id': 'req-e8b6bf86-5253-47c2-a7e5-c68f45d448c6'} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:32:54.970136 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-e8b6bf86-5253-47c2-a7e5-c68f45d448c6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '53293930b4d84b9d921becaaf81e17cc', 'request_id': 'req-e8b6bf86-5253-47c2-a7e5-c68f45d448c6'} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:32:54.983434 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-e8b6bf86-5253-47c2-a7e5-c68f45d448c6 admin admin] The token cache is being invalidate because project 53293930b4d84b9d921becaaf81e17cc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81400) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 03:32:54.983721 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-e8b6bf86-5253-47c2-a7e5-c68f45d448c6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:32:54.984340 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-e8b6bf86-5253-47c2-a7e5-c68f45d448c6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:32:54.985805 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 1289/2583] 10.222.0.18 () {58 vars in 1233 bytes} [Tue Sep 13 03:32:54 2022] DELETE /identity/v3/projects/53293930b4d84b9d921becaaf81e17cc => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 03:32:54.991755 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc0798fa-aaaa-42ea-8e1c-5142fa654390 None None] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:32:54.992041 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc0798fa-aaaa-42ea-8e1c-5142fa654390 None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:32:54.992309 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc0798fa-aaaa-42ea-8e1c-5142fa654390 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:32:55.013061 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.auth.core [None req-fc0798fa-aaaa-42ea-8e1c-5142fa654390 None None] MFA Rules not processed for user `49f280ab086b4610855c4ffa378517d9`. Rule list: `[]` (Enabled: `True`). {{(pid=81399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 03:32:55.014360 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-fc0798fa-aaaa-42ea-8e1c-5142fa654390 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:32:55.073485 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 1295/2584] 10.222.0.18 () {58 vars in 958 bytes} [Tue Sep 13 03:32:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 03:32:55.081454 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c8daaba-58b1-4e96-a061-7f2b8ffeff88 None None] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:32:55.081797 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c8daaba-58b1-4e96-a061-7f2b8ffeff88 None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:32:55.082075 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c8daaba-58b1-4e96-a061-7f2b8ffeff88 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:32:55.103196 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.auth.core [None req-9c8daaba-58b1-4e96-a061-7f2b8ffeff88 None None] MFA Rules not processed for user `49f280ab086b4610855c4ffa378517d9`. Rule list: `[]` (Enabled: `True`). {{(pid=81400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 03:32:55.105247 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-9c8daaba-58b1-4e96-a061-7f2b8ffeff88 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:32:55.113238 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 1290/2585] 10.222.0.18 () {58 vars in 958 bytes} [Tue Sep 13 03:32:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 03:32:55.117436 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc0798fa-aaaa-42ea-8e1c-5142fa654390 None None] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:32:55.126934 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc0798fa-aaaa-42ea-8e1c-5142fa654390 None None] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:32:55.131478 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-468fa310-7572-447d-b265-f0b82adae289 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76c5181fc7be45279ac00017bb4640b3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:32:55.132064 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-468fa310-7572-447d-b265-f0b82adae289 admin admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:32:55.132341 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-468fa310-7572-447d-b265-f0b82adae289 admin admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:32:55.132609 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-468fa310-7572-447d-b265-f0b82adae289 admin admin] PATH_INFO: `/v3/domains` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:32:55.133238 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-468fa310-7572-447d-b265-f0b82adae289 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 03:32:55.133535 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-468fa310-7572-447d-b265-f0b82adae289 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:32:55.135151 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-468fa310-7572-447d-b265-f0b82adae289 admin admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:32:55.146391 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 1296/2586] 10.222.0.18 () {58 vars in 1153 bytes} [Tue Sep 13 03:32:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 03:32:55.150443 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c8daaba-58b1-4e96-a061-7f2b8ffeff88 None None] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:32:55.160090 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c8daaba-58b1-4e96-a061-7f2b8ffeff88 None None] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:32:55.165005 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00fbb773-1293-4107-baf2-4836c7f4c124 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76c5181fc7be45279ac00017bb4640b3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:32:55.165591 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00fbb773-1293-4107-baf2-4836c7f4c124 admin admin] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:32:55.165868 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00fbb773-1293-4107-baf2-4836c7f4c124 admin admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:32:55.166137 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00fbb773-1293-4107-baf2-4836c7f4c124 admin admin] PATH_INFO: `/v3/projects` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:32:55.166944 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00fbb773-1293-4107-baf2-4836c7f4c124 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:32:55.169489 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00fbb773-1293-4107-baf2-4836c7f4c124 admin admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:32:55.195716 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 1291/2587] 10.222.0.18 () {60 vars in 1153 bytes} [Tue Sep 13 03:32:55 2022] POST /identity/v3/projects => generated 371 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 13 03:32:55.199709 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-468fa310-7572-447d-b265-f0b82adae289 admin admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:32:55.209526 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-468fa310-7572-447d-b265-f0b82adae289 admin admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:32:55.264240 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47d98c0d-b6e4-40c1-aee2-cf294fb92157 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76c5181fc7be45279ac00017bb4640b3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:32:55.264770 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47d98c0d-b6e4-40c1-aee2-cf294fb92157 admin admin] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:32:55.265045 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47d98c0d-b6e4-40c1-aee2-cf294fb92157 admin admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:32:55.265342 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47d98c0d-b6e4-40c1-aee2-cf294fb92157 admin admin] PATH_INFO: `/v3/users` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:32:55.265963 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47d98c0d-b6e4-40c1-aee2-cf294fb92157 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:32:55.267930 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47d98c0d-b6e4-40c1-aee2-cf294fb92157 admin admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:32:55.288703 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 1297/2588] 10.222.0.18 () {60 vars in 1144 bytes} [Tue Sep 13 03:32:55 2022] POST /identity/v3/users => generated 286 bytes in 89 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 03:32:55.293837 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00fbb773-1293-4107-baf2-4836c7f4c124 admin admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:32:55.303416 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00fbb773-1293-4107-baf2-4836c7f4c124 admin admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:32:55.310696 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ce192ba-5474-4a33-bbb4-372c416aabd1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76c5181fc7be45279ac00017bb4640b3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:32:55.311286 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce192ba-5474-4a33-bbb4-372c416aabd1 admin admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:32:55.311576 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce192ba-5474-4a33-bbb4-372c416aabd1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:32:55.311824 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce192ba-5474-4a33-bbb4-372c416aabd1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:32:55.312452 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ce192ba-5474-4a33-bbb4-372c416aabd1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:32:55.314033 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ce192ba-5474-4a33-bbb4-372c416aabd1 admin admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:32:55.324699 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 1292/2589] 10.222.0.18 () {58 vars in 1122 bytes} [Tue Sep 13 03:32:55 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 03:32:55.328568 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47d98c0d-b6e4-40c1-aee2-cf294fb92157 admin admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:32:55.338305 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47d98c0d-b6e4-40c1-aee2-cf294fb92157 admin admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:32:55.343396 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1f7803c-0cbd-4ae3-8b90-699faf070b3d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76c5181fc7be45279ac00017bb4640b3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:32:55.343988 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f7803c-0cbd-4ae3-8b90-699faf070b3d admin admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:32:55.344266 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f7803c-0cbd-4ae3-8b90-699faf070b3d admin admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:32:55.344535 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f7803c-0cbd-4ae3-8b90-699faf070b3d admin admin] PATH_INFO: `/v3/roles` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:32:55.345161 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1f7803c-0cbd-4ae3-8b90-699faf070b3d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:32:55.346766 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1f7803c-0cbd-4ae3-8b90-699faf070b3d admin admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:32:55.357800 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 1298/2590] 10.222.0.18 () {58 vars in 1122 bytes} [Tue Sep 13 03:32:55 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 03:32:55.362478 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ce192ba-5474-4a33-bbb4-372c416aabd1 admin admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:32:55.373216 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ce192ba-5474-4a33-bbb4-372c416aabd1 admin admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:32:55.377925 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a414075a-f58d-415a-83a7-ea566d357eb6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76c5181fc7be45279ac00017bb4640b3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:32:55.378579 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a414075a-f58d-415a-83a7-ea566d357eb6 admin admin] REQUEST_METHOD: `PUT` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:32:55.378874 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a414075a-f58d-415a-83a7-ea566d357eb6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:32:55.379145 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a414075a-f58d-415a-83a7-ea566d357eb6 admin admin] PATH_INFO: `/v3/projects/2a29b9a08d2c4529a33325b812b32440/users/043054f3ad764b01aefdcc6062aa45af/roles/1eb7a5b5203945c29e6d22d653f47847` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:32:55.394058 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a414075a-f58d-415a-83a7-ea566d357eb6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2a29b9a08d2c4529a33325b812b32440, user_id=043054f3ad764b01aefdcc6062aa45af, role_id=1eb7a5b5203945c29e6d22d653f47847)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:32:55.396823 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a414075a-f58d-415a-83a7-ea566d357eb6 admin admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:32:55.411517 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 1293/2591] 10.222.0.18 () {60 vars in 1483 bytes} [Tue Sep 13 03:32:55 2022] PUT /identity/v3/projects/2a29b9a08d2c4529a33325b812b32440/users/043054f3ad764b01aefdcc6062aa45af/roles/1eb7a5b5203945c29e6d22d653f47847 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 03:32:55.423576 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67ce3b5c-ffe9-4f04-a4b3-91b887a8eb34 None None] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:32:55.423759 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67ce3b5c-ffe9-4f04-a4b3-91b887a8eb34 None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:32:55.424026 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67ce3b5c-ffe9-4f04-a4b3-91b887a8eb34 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:32:55.452343 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.auth.core [None req-67ce3b5c-ffe9-4f04-a4b3-91b887a8eb34 None None] MFA Rules not processed for user `043054f3ad764b01aefdcc6062aa45af`. Rule list: `[]` (Enabled: `True`). {{(pid=81399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 03:32:55.453550 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-67ce3b5c-ffe9-4f04-a4b3-91b887a8eb34 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:32:55.462697 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:DELETE=5,keystone:SELECT=91,keystone:INSERT=4 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:32:55.525257 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 1299/2592] 10.222.0.18 () {58 vars in 958 bytes} [Tue Sep 13 03:32:55 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 03:32:55.538061 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a414075a-f58d-415a-83a7-ea566d357eb6 admin admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:32:55.539858 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-a414075a-f58d-415a-83a7-ea566d357eb6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:32:55.578991 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:32:55.615018 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a414075a-f58d-415a-83a7-ea566d357eb6 admin admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:32:55.618938 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b86f4412-ecdc-4c8a-9cae-794a9f104fb0 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b5fdc2ff46a14638b409a13e73ba5500', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b9d94f6a68f7470d9494f112175620dc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:32:55.619539 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b86f4412-ecdc-4c8a-9cae-794a9f104fb0 service glance] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:32:55.619840 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b86f4412-ecdc-4c8a-9cae-794a9f104fb0 service glance] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:32:55.620157 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b86f4412-ecdc-4c8a-9cae-794a9f104fb0 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:32:55.628676 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b86f4412-ecdc-4c8a-9cae-794a9f104fb0 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:32:55.630269 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b86f4412-ecdc-4c8a-9cae-794a9f104fb0 service glance] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:32:55.639990 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 1294/2593] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Sep 13 03:32:55 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 03:33:05.522199 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=22 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:33:05.635563 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=16 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:33:18.233327 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67ce3b5c-ffe9-4f04-a4b3-91b887a8eb34 None None] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:33:18.234747 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-67ce3b5c-ffe9-4f04-a4b3-91b887a8eb34 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:33:18.291696 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67ce3b5c-ffe9-4f04-a4b3-91b887a8eb34 None None] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:33:18.295282 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c80ecacd-516c-4b26-b626-439e7f5f1e2d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdffbc851086441ebd01022d1e4c2609', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b9d94f6a68f7470d9494f112175620dc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:33:18.295960 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c80ecacd-516c-4b26-b626-439e7f5f1e2d service neutron] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:33:18.296244 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c80ecacd-516c-4b26-b626-439e7f5f1e2d service neutron] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:33:18.296516 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c80ecacd-516c-4b26-b626-439e7f5f1e2d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:33:18.298225 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-c80ecacd-516c-4b26-b626-439e7f5f1e2d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:33:18.362046 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c80ecacd-516c-4b26-b626-439e7f5f1e2d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:33:18.363718 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c80ecacd-516c-4b26-b626-439e7f5f1e2d service neutron] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:33:18.373118 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 1300/2594] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Sep 13 03:33:18 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 140 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 03:33:21.640393 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b86f4412-ecdc-4c8a-9cae-794a9f104fb0 service glance] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:33:21.642649 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-b86f4412-ecdc-4c8a-9cae-794a9f104fb0 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:33:21.658061 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b86f4412-ecdc-4c8a-9cae-794a9f104fb0 service glance] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:33:21.663568 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25d5b025-8e1e-460a-8105-9b161d38081b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76c5181fc7be45279ac00017bb4640b3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:33:21.664337 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25d5b025-8e1e-460a-8105-9b161d38081b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:33:21.664714 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25d5b025-8e1e-460a-8105-9b161d38081b admin admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:33:21.665084 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25d5b025-8e1e-460a-8105-9b161d38081b admin admin] PATH_INFO: `/v3/users/4f76a7c6fe9347b0bea5f701bccb6150` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:33:21.666348 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25d5b025-8e1e-460a-8105-9b161d38081b admin admin] RBAC: Authorizing `identity:delete_user(user_id=4f76a7c6fe9347b0bea5f701bccb6150)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:33:21.674678 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25d5b025-8e1e-460a-8105-9b161d38081b admin admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:33:21.729888 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-25d5b025-8e1e-460a-8105-9b161d38081b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4f76a7c6fe9347b0bea5f701bccb6150'} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:33:21.737792 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-25d5b025-8e1e-460a-8105-9b161d38081b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4f76a7c6fe9347b0bea5f701bccb6150'} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:33:21.738753 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-25d5b025-8e1e-460a-8105-9b161d38081b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4f76a7c6fe9347b0bea5f701bccb6150'} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:33:21.752114 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-25d5b025-8e1e-460a-8105-9b161d38081b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4f76a7c6fe9347b0bea5f701bccb6150'} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:33:21.769964 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-25d5b025-8e1e-460a-8105-9b161d38081b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4f76a7c6fe9347b0bea5f701bccb6150'} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:33:21.769964 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 1295/2595] 10.222.0.18 () {58 vars in 1224 bytes} [Tue Sep 13 03:33:21 2022] DELETE /identity/v3/users/4f76a7c6fe9347b0bea5f701bccb6150 => generated 0 bytes in 128 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 03:33:21.936634 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c80ecacd-516c-4b26-b626-439e7f5f1e2d service neutron] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:33:21.950518 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c80ecacd-516c-4b26-b626-439e7f5f1e2d service neutron] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:33:22.006313 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff2f1a57-4dc0-4ede-9685-8045562f09cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76c5181fc7be45279ac00017bb4640b3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:33:22.006912 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff2f1a57-4dc0-4ede-9685-8045562f09cc admin admin] REQUEST_METHOD: `DELETE` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:33:22.007204 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff2f1a57-4dc0-4ede-9685-8045562f09cc admin admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:33:22.007481 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff2f1a57-4dc0-4ede-9685-8045562f09cc admin admin] PATH_INFO: `/v3/projects/676c9aac6fa348b89a2d5cef106f1a74` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:33:22.008603 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff2f1a57-4dc0-4ede-9685-8045562f09cc admin admin] RBAC: Authorizing `identity:delete_project(project_id=676c9aac6fa348b89a2d5cef106f1a74)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:33:22.010541 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff2f1a57-4dc0-4ede-9685-8045562f09cc admin admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:33:22.056576 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-ff2f1a57-4dc0-4ede-9685-8045562f09cc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '676c9aac6fa348b89a2d5cef106f1a74', 'request_id': 'req-ff2f1a57-4dc0-4ede-9685-8045562f09cc'} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:33:22.058643 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:33:22.063386 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-ff2f1a57-4dc0-4ede-9685-8045562f09cc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '676c9aac6fa348b89a2d5cef106f1a74', 'request_id': 'req-ff2f1a57-4dc0-4ede-9685-8045562f09cc'} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:33:22.071130 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-ff2f1a57-4dc0-4ede-9685-8045562f09cc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '676c9aac6fa348b89a2d5cef106f1a74', 'request_id': 'req-ff2f1a57-4dc0-4ede-9685-8045562f09cc'} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:33:22.083648 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-ff2f1a57-4dc0-4ede-9685-8045562f09cc admin admin] The token cache is being invalidate because project 676c9aac6fa348b89a2d5cef106f1a74 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81399) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 03:33:22.083938 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-ff2f1a57-4dc0-4ede-9685-8045562f09cc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:33:22.084554 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-ff2f1a57-4dc0-4ede-9685-8045562f09cc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:33:22.086029 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 1301/2596] 10.222.0.18 () {58 vars in 1233 bytes} [Tue Sep 13 03:33:21 2022] DELETE /identity/v3/projects/676c9aac6fa348b89a2d5cef106f1a74 => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 03:33:22.090960 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d77dc1-f551-4224-8fa1-afaa849d699b None None] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:33:22.091321 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d77dc1-f551-4224-8fa1-afaa849d699b None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:33:22.091625 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d77dc1-f551-4224-8fa1-afaa849d699b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:33:22.111818 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.auth.core [None req-f0d77dc1-f551-4224-8fa1-afaa849d699b None None] MFA Rules not processed for user `49f280ab086b4610855c4ffa378517d9`. Rule list: `[]` (Enabled: `True`). {{(pid=81400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 03:33:22.113010 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-f0d77dc1-f551-4224-8fa1-afaa849d699b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:33:22.171278 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 1296/2597] 10.222.0.18 () {58 vars in 958 bytes} [Tue Sep 13 03:33:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 03:33:22.179214 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a4df1d5-3b65-4161-b366-c2b7f1820524 None None] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:33:22.179514 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a4df1d5-3b65-4161-b366-c2b7f1820524 None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:33:22.179788 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a4df1d5-3b65-4161-b366-c2b7f1820524 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:33:22.199169 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.auth.core [None req-3a4df1d5-3b65-4161-b366-c2b7f1820524 None None] MFA Rules not processed for user `49f280ab086b4610855c4ffa378517d9`. Rule list: `[]` (Enabled: `True`). {{(pid=81399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 03:33:22.200363 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-3a4df1d5-3b65-4161-b366-c2b7f1820524 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:33:22.208759 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 1302/2598] 10.222.0.18 () {58 vars in 958 bytes} [Tue Sep 13 03:33:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 03:33:22.212717 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0d77dc1-f551-4224-8fa1-afaa849d699b None None] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:33:22.222914 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0d77dc1-f551-4224-8fa1-afaa849d699b None None] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:33:22.227418 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25cfd8e4-cd43-4c10-8b16-fec9cd7cba88 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76c5181fc7be45279ac00017bb4640b3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:33:22.228001 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25cfd8e4-cd43-4c10-8b16-fec9cd7cba88 admin admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:33:22.228277 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25cfd8e4-cd43-4c10-8b16-fec9cd7cba88 admin admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:33:22.228552 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25cfd8e4-cd43-4c10-8b16-fec9cd7cba88 admin admin] PATH_INFO: `/v3/domains` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:33:22.229184 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25cfd8e4-cd43-4c10-8b16-fec9cd7cba88 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 03:33:22.229483 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25cfd8e4-cd43-4c10-8b16-fec9cd7cba88 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:33:22.231123 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25cfd8e4-cd43-4c10-8b16-fec9cd7cba88 admin admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:33:22.242777 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 1297/2599] 10.222.0.18 () {58 vars in 1153 bytes} [Tue Sep 13 03:33:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 03:33:22.246722 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a4df1d5-3b65-4161-b366-c2b7f1820524 None None] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:33:22.255699 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a4df1d5-3b65-4161-b366-c2b7f1820524 None None] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:33:22.260518 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9305a57-791f-4a53-8944-d5b2d476c996 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76c5181fc7be45279ac00017bb4640b3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:33:22.261044 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9305a57-791f-4a53-8944-d5b2d476c996 admin admin] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:33:22.261319 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9305a57-791f-4a53-8944-d5b2d476c996 admin admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:33:22.261584 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9305a57-791f-4a53-8944-d5b2d476c996 admin admin] PATH_INFO: `/v3/projects` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:33:22.262236 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9305a57-791f-4a53-8944-d5b2d476c996 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:33:22.264172 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9305a57-791f-4a53-8944-d5b2d476c996 admin admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:33:22.291852 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 1303/2600] 10.222.0.18 () {60 vars in 1153 bytes} [Tue Sep 13 03:33:22 2022] POST /identity/v3/projects => generated 385 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 13 03:33:22.295770 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25cfd8e4-cd43-4c10-8b16-fec9cd7cba88 admin admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:33:22.305538 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25cfd8e4-cd43-4c10-8b16-fec9cd7cba88 admin admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:33:22.361254 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9fdb7cd-4be0-4bc2-ad83-94ab16d8bd13 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76c5181fc7be45279ac00017bb4640b3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:33:22.361879 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9fdb7cd-4be0-4bc2-ad83-94ab16d8bd13 admin admin] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:33:22.362160 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9fdb7cd-4be0-4bc2-ad83-94ab16d8bd13 admin admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:33:22.362434 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9fdb7cd-4be0-4bc2-ad83-94ab16d8bd13 admin admin] PATH_INFO: `/v3/users` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:33:22.363089 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9fdb7cd-4be0-4bc2-ad83-94ab16d8bd13 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:33:22.365017 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9fdb7cd-4be0-4bc2-ad83-94ab16d8bd13 admin admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:33:22.372837 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:33:22.387315 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 1298/2601] 10.222.0.18 () {60 vars in 1144 bytes} [Tue Sep 13 03:33:22 2022] POST /identity/v3/users => generated 293 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 03:33:22.391808 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9305a57-791f-4a53-8944-d5b2d476c996 admin admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:33:22.401809 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9305a57-791f-4a53-8944-d5b2d476c996 admin admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:33:22.406680 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9bf445f-a1fd-454e-ab7a-3fc4c22ee718 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76c5181fc7be45279ac00017bb4640b3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:33:22.407917 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9bf445f-a1fd-454e-ab7a-3fc4c22ee718 admin admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:33:22.408230 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9bf445f-a1fd-454e-ab7a-3fc4c22ee718 admin admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:33:22.408500 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9bf445f-a1fd-454e-ab7a-3fc4c22ee718 admin admin] PATH_INFO: `/v3/roles` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:33:22.409142 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9bf445f-a1fd-454e-ab7a-3fc4c22ee718 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:33:22.410819 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9bf445f-a1fd-454e-ab7a-3fc4c22ee718 admin admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:33:22.421117 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 1304/2602] 10.222.0.18 () {58 vars in 1122 bytes} [Tue Sep 13 03:33:22 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 03:33:22.424862 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9fdb7cd-4be0-4bc2-ad83-94ab16d8bd13 admin admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:33:22.434907 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9fdb7cd-4be0-4bc2-ad83-94ab16d8bd13 admin admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:33:22.439816 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f9d63dd-e84c-4dac-8f57-08bb3b7d9bdd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76c5181fc7be45279ac00017bb4640b3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:33:22.440409 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f9d63dd-e84c-4dac-8f57-08bb3b7d9bdd admin admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:33:22.440687 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f9d63dd-e84c-4dac-8f57-08bb3b7d9bdd admin admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:33:22.440978 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f9d63dd-e84c-4dac-8f57-08bb3b7d9bdd admin admin] PATH_INFO: `/v3/roles` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:33:22.441628 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f9d63dd-e84c-4dac-8f57-08bb3b7d9bdd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:33:22.443404 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f9d63dd-e84c-4dac-8f57-08bb3b7d9bdd admin admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:33:22.455260 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 1299/2603] 10.222.0.18 () {58 vars in 1122 bytes} [Tue Sep 13 03:33:22 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 03:33:22.459355 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9bf445f-a1fd-454e-ab7a-3fc4c22ee718 admin admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:33:22.469196 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9bf445f-a1fd-454e-ab7a-3fc4c22ee718 admin admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:33:22.475090 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8007d3d5-0a4a-4406-bc09-8caee4ad152c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76c5181fc7be45279ac00017bb4640b3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:33:22.475742 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8007d3d5-0a4a-4406-bc09-8caee4ad152c admin admin] REQUEST_METHOD: `PUT` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:33:22.476025 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8007d3d5-0a4a-4406-bc09-8caee4ad152c admin admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:33:22.476302 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8007d3d5-0a4a-4406-bc09-8caee4ad152c admin admin] PATH_INFO: `/v3/projects/a8768c348a7f46bd85164072acc772b0/users/44eef37fd6fc4dcc923b066da840594a/roles/1eb7a5b5203945c29e6d22d653f47847` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:33:22.491867 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8007d3d5-0a4a-4406-bc09-8caee4ad152c admin admin] RBAC: Authorizing `identity:create_grant(project_id=a8768c348a7f46bd85164072acc772b0, user_id=44eef37fd6fc4dcc923b066da840594a, role_id=1eb7a5b5203945c29e6d22d653f47847)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:33:22.494486 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8007d3d5-0a4a-4406-bc09-8caee4ad152c admin admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:33:22.506582 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 1305/2604] 10.222.0.18 () {60 vars in 1483 bytes} [Tue Sep 13 03:33:22 2022] PUT /identity/v3/projects/a8768c348a7f46bd85164072acc772b0/users/44eef37fd6fc4dcc923b066da840594a/roles/1eb7a5b5203945c29e6d22d653f47847 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 03:33:22.518253 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c994a4-7ee3-49a9-a94e-5f6fb62b2fd3 None None] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:33:22.518578 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c994a4-7ee3-49a9-a94e-5f6fb62b2fd3 None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:33:22.518871 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c994a4-7ee3-49a9-a94e-5f6fb62b2fd3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:33:22.547567 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.auth.core [None req-b0c994a4-7ee3-49a9-a94e-5f6fb62b2fd3 None None] MFA Rules not processed for user `44eef37fd6fc4dcc923b066da840594a`. Rule list: `[]` (Enabled: `True`). {{(pid=81400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 03:33:22.548768 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-b0c994a4-7ee3-49a9-a94e-5f6fb62b2fd3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:33:22.615269 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 1300/2605] 10.222.0.18 () {58 vars in 958 bytes} [Tue Sep 13 03:33:22 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 13 03:33:22.627495 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8007d3d5-0a4a-4406-bc09-8caee4ad152c admin admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:33:22.629209 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-8007d3d5-0a4a-4406-bc09-8caee4ad152c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:33:22.704495 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8007d3d5-0a4a-4406-bc09-8caee4ad152c admin admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:33:22.710227 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28c3e205-9500-4712-ba1a-57f8eb11eac5 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7377f5bee4a49959182e1bd34bc97a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b9d94f6a68f7470d9494f112175620dc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:33:22.710858 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c3e205-9500-4712-ba1a-57f8eb11eac5 service nova] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:33:22.711145 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c3e205-9500-4712-ba1a-57f8eb11eac5 service nova] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:33:22.711410 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c3e205-9500-4712-ba1a-57f8eb11eac5 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:33:22.719867 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28c3e205-9500-4712-ba1a-57f8eb11eac5 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:33:22.721553 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28c3e205-9500-4712-ba1a-57f8eb11eac5 service nova] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:33:22.731950 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 1306/2606] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Sep 13 03:33:22 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 13 03:33:32.612477 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:INSERT=3,keystone:SELECT=39 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:33:32.727154 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=67,keystone:DELETE=2,keystone:INSERT=3 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:33:37.093330 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0c994a4-7ee3-49a9-a94e-5f6fb62b2fd3 None None] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:33:37.094697 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-b0c994a4-7ee3-49a9-a94e-5f6fb62b2fd3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:33:37.154128 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0c994a4-7ee3-49a9-a94e-5f6fb62b2fd3 None None] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:33:37.159143 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f597ea2-7e12-469b-8a53-d5cc4423a9b9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdffbc851086441ebd01022d1e4c2609', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b9d94f6a68f7470d9494f112175620dc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:33:37.159710 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f597ea2-7e12-469b-8a53-d5cc4423a9b9 service neutron] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:33:37.159946 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f597ea2-7e12-469b-8a53-d5cc4423a9b9 service neutron] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:33:37.160220 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f597ea2-7e12-469b-8a53-d5cc4423a9b9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:33:37.162428 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-8f597ea2-7e12-469b-8a53-d5cc4423a9b9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:33:37.232820 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f597ea2-7e12-469b-8a53-d5cc4423a9b9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:33:37.234245 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f597ea2-7e12-469b-8a53-d5cc4423a9b9 service neutron] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:33:37.244431 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 1301/2607] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Sep 13 03:33:37 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 152 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 03:33:37.410323 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28c3e205-9500-4712-ba1a-57f8eb11eac5 service nova] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:33:37.420810 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28c3e205-9500-4712-ba1a-57f8eb11eac5 service nova] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:33:37.425939 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38618415-2c5b-4771-9c95-fbebea4905db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76c5181fc7be45279ac00017bb4640b3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:33:37.426556 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38618415-2c5b-4771-9c95-fbebea4905db admin admin] REQUEST_METHOD: `DELETE` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:33:37.426864 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38618415-2c5b-4771-9c95-fbebea4905db admin admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:33:37.427136 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38618415-2c5b-4771-9c95-fbebea4905db admin admin] PATH_INFO: `/v3/users/44eef37fd6fc4dcc923b066da840594a` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:33:37.428251 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38618415-2c5b-4771-9c95-fbebea4905db admin admin] RBAC: Authorizing `identity:delete_user(user_id=44eef37fd6fc4dcc923b066da840594a)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:33:37.430153 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38618415-2c5b-4771-9c95-fbebea4905db admin admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:33:37.476795 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-38618415-2c5b-4771-9c95-fbebea4905db admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '44eef37fd6fc4dcc923b066da840594a'} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:33:37.484512 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-38618415-2c5b-4771-9c95-fbebea4905db admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '44eef37fd6fc4dcc923b066da840594a'} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:33:37.498645 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-38618415-2c5b-4771-9c95-fbebea4905db admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '44eef37fd6fc4dcc923b066da840594a'} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:33:37.499393 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-38618415-2c5b-4771-9c95-fbebea4905db admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '44eef37fd6fc4dcc923b066da840594a'} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:33:37.508063 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-38618415-2c5b-4771-9c95-fbebea4905db admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '44eef37fd6fc4dcc923b066da840594a'} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:33:37.514078 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 1307/2608] 10.222.0.18 () {58 vars in 1224 bytes} [Tue Sep 13 03:33:37 2022] DELETE /identity/v3/users/44eef37fd6fc4dcc923b066da840594a => generated 0 bytes in 105 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 13 03:33:37.518259 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e24cddc-77e7-47ff-9404-7ff4ded1accc None None] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:33:37.518809 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e24cddc-77e7-47ff-9404-7ff4ded1accc None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:33:37.519161 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e24cddc-77e7-47ff-9404-7ff4ded1accc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:33:37.539708 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.auth.core [None req-6e24cddc-77e7-47ff-9404-7ff4ded1accc None None] MFA Rules not processed for user `49f280ab086b4610855c4ffa378517d9`. Rule list: `[]` (Enabled: `True`). {{(pid=81400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 03:33:37.541010 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-6e24cddc-77e7-47ff-9404-7ff4ded1accc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:33:37.601950 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 1302/2609] 10.222.0.18 () {58 vars in 958 bytes} [Tue Sep 13 03:33:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 13 03:33:37.611501 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38618415-2c5b-4771-9c95-fbebea4905db admin admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:33:37.613624 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-38618415-2c5b-4771-9c95-fbebea4905db admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:33:37.670584 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38618415-2c5b-4771-9c95-fbebea4905db admin admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:33:37.674142 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90795e01-aed0-4e38-b73f-45173ce92f92 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdffbc851086441ebd01022d1e4c2609', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b9d94f6a68f7470d9494f112175620dc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:33:37.674779 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90795e01-aed0-4e38-b73f-45173ce92f92 service neutron] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:33:37.675659 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90795e01-aed0-4e38-b73f-45173ce92f92 service neutron] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:33:37.675933 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90795e01-aed0-4e38-b73f-45173ce92f92 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:33:37.685087 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90795e01-aed0-4e38-b73f-45173ce92f92 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:33:37.686511 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90795e01-aed0-4e38-b73f-45173ce92f92 service neutron] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:33:37.696245 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 1308/2610] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Sep 13 03:33:37 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 03:33:37.973148 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e24cddc-77e7-47ff-9404-7ff4ded1accc None None] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:33:37.974736 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-6e24cddc-77e7-47ff-9404-7ff4ded1accc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:33:37.989294 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e24cddc-77e7-47ff-9404-7ff4ded1accc None None] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:33:37.994186 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8946ed0d-d2b8-4e5c-89c3-84bd818845ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76c5181fc7be45279ac00017bb4640b3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:33:37.994941 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8946ed0d-d2b8-4e5c-89c3-84bd818845ee admin admin] REQUEST_METHOD: `DELETE` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:33:37.995316 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8946ed0d-d2b8-4e5c-89c3-84bd818845ee admin admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:33:37.996181 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8946ed0d-d2b8-4e5c-89c3-84bd818845ee admin admin] PATH_INFO: `/v3/projects/a8768c348a7f46bd85164072acc772b0` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:33:37.997629 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8946ed0d-d2b8-4e5c-89c3-84bd818845ee admin admin] RBAC: Authorizing `identity:delete_project(project_id=a8768c348a7f46bd85164072acc772b0)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:33:38.005590 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8946ed0d-d2b8-4e5c-89c3-84bd818845ee admin admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:33:38.057606 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:33:38.061759 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-8946ed0d-d2b8-4e5c-89c3-84bd818845ee admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a8768c348a7f46bd85164072acc772b0', 'request_id': 'req-8946ed0d-d2b8-4e5c-89c3-84bd818845ee'} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:33:38.068664 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-8946ed0d-d2b8-4e5c-89c3-84bd818845ee admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a8768c348a7f46bd85164072acc772b0', 'request_id': 'req-8946ed0d-d2b8-4e5c-89c3-84bd818845ee'} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:33:38.075603 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-8946ed0d-d2b8-4e5c-89c3-84bd818845ee admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a8768c348a7f46bd85164072acc772b0', 'request_id': 'req-8946ed0d-d2b8-4e5c-89c3-84bd818845ee'} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:33:38.088434 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-8946ed0d-d2b8-4e5c-89c3-84bd818845ee admin admin] The token cache is being invalidate because project a8768c348a7f46bd85164072acc772b0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81400) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 03:33:38.088721 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-8946ed0d-d2b8-4e5c-89c3-84bd818845ee admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:33:38.089343 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-8946ed0d-d2b8-4e5c-89c3-84bd818845ee admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:33:38.090884 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 1303/2611] 10.222.0.18 () {58 vars in 1233 bytes} [Tue Sep 13 03:33:37 2022] DELETE /identity/v3/projects/a8768c348a7f46bd85164072acc772b0 => generated 0 bytes in 119 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 03:33:38.096006 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc3c32b-d425-4b23-a0e4-38d299e63e9f None None] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:33:38.096297 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc3c32b-d425-4b23-a0e4-38d299e63e9f None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:33:38.096573 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc3c32b-d425-4b23-a0e4-38d299e63e9f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:33:38.117183 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.auth.core [None req-9bc3c32b-d425-4b23-a0e4-38d299e63e9f None None] MFA Rules not processed for user `49f280ab086b4610855c4ffa378517d9`. Rule list: `[]` (Enabled: `True`). {{(pid=81399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 03:33:38.118402 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-9bc3c32b-d425-4b23-a0e4-38d299e63e9f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:33:38.181500 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 1309/2612] 10.222.0.18 () {58 vars in 958 bytes} [Tue Sep 13 03:33:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 03:33:38.189524 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de6d08ec-7909-4364-9b36-3fbfede59b0f None None] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:33:38.190094 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de6d08ec-7909-4364-9b36-3fbfede59b0f None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:33:38.190218 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de6d08ec-7909-4364-9b36-3fbfede59b0f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:33:38.212991 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.auth.core [None req-de6d08ec-7909-4364-9b36-3fbfede59b0f None None] MFA Rules not processed for user `49f280ab086b4610855c4ffa378517d9`. Rule list: `[]` (Enabled: `True`). {{(pid=81400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 03:33:38.214390 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-de6d08ec-7909-4364-9b36-3fbfede59b0f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:33:38.224417 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 1304/2613] 10.222.0.18 () {58 vars in 958 bytes} [Tue Sep 13 03:33:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 03:33:38.228418 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bc3c32b-d425-4b23-a0e4-38d299e63e9f None None] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:33:38.238857 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bc3c32b-d425-4b23-a0e4-38d299e63e9f None None] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:33:38.243716 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d41a722-4070-43da-a3d3-1b0b30dd528d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76c5181fc7be45279ac00017bb4640b3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:33:38.244327 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d41a722-4070-43da-a3d3-1b0b30dd528d admin admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:33:38.244598 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d41a722-4070-43da-a3d3-1b0b30dd528d admin admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:33:38.244861 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d41a722-4070-43da-a3d3-1b0b30dd528d admin admin] PATH_INFO: `/v3/domains` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:33:38.245510 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d41a722-4070-43da-a3d3-1b0b30dd528d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 03:33:38.245809 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d41a722-4070-43da-a3d3-1b0b30dd528d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:33:38.247508 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d41a722-4070-43da-a3d3-1b0b30dd528d admin admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:33:38.263433 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 1310/2614] 10.222.0.18 () {58 vars in 1153 bytes} [Tue Sep 13 03:33:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 03:33:38.268253 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de6d08ec-7909-4364-9b36-3fbfede59b0f None None] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:33:38.279077 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de6d08ec-7909-4364-9b36-3fbfede59b0f None None] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:33:38.284557 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd6bb011-b309-490a-b729-e2b1f3c3646e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76c5181fc7be45279ac00017bb4640b3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:33:38.285258 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd6bb011-b309-490a-b729-e2b1f3c3646e admin admin] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:33:38.286852 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd6bb011-b309-490a-b729-e2b1f3c3646e admin admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:33:38.286852 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd6bb011-b309-490a-b729-e2b1f3c3646e admin admin] PATH_INFO: `/v3/projects` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:33:38.286852 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd6bb011-b309-490a-b729-e2b1f3c3646e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:33:38.288712 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd6bb011-b309-490a-b729-e2b1f3c3646e admin admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:33:38.327268 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 1305/2615] 10.222.0.18 () {60 vars in 1153 bytes} [Tue Sep 13 03:33:38 2022] POST /identity/v3/projects => generated 405 bytes in 59 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 13 03:33:38.332267 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d41a722-4070-43da-a3d3-1b0b30dd528d admin admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:33:38.337422 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:33:38.343938 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d41a722-4070-43da-a3d3-1b0b30dd528d admin admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:33:38.420435 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e17cc63-046b-4c96-9602-b6dc6332a56d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76c5181fc7be45279ac00017bb4640b3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:33:38.421921 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e17cc63-046b-4c96-9602-b6dc6332a56d admin admin] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:33:38.422401 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e17cc63-046b-4c96-9602-b6dc6332a56d admin admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:33:38.422937 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e17cc63-046b-4c96-9602-b6dc6332a56d admin admin] PATH_INFO: `/v3/users` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:33:38.424093 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e17cc63-046b-4c96-9602-b6dc6332a56d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:33:38.426117 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e17cc63-046b-4c96-9602-b6dc6332a56d admin admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:33:38.451580 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 1311/2616] 10.222.0.18 () {60 vars in 1144 bytes} [Tue Sep 13 03:33:38 2022] POST /identity/v3/users => generated 303 bytes in 120 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 03:33:38.455875 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd6bb011-b309-490a-b729-e2b1f3c3646e admin admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:33:38.468332 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd6bb011-b309-490a-b729-e2b1f3c3646e admin admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:33:38.473418 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d575bdc7-655d-4eb8-b582-973810e85ac2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76c5181fc7be45279ac00017bb4640b3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:33:38.474040 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d575bdc7-655d-4eb8-b582-973810e85ac2 admin admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:33:38.474479 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d575bdc7-655d-4eb8-b582-973810e85ac2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:33:38.474791 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d575bdc7-655d-4eb8-b582-973810e85ac2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:33:38.475447 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d575bdc7-655d-4eb8-b582-973810e85ac2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:33:38.477077 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d575bdc7-655d-4eb8-b582-973810e85ac2 admin admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:33:38.488759 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 1306/2617] 10.222.0.18 () {58 vars in 1122 bytes} [Tue Sep 13 03:33:38 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 03:33:38.492916 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e17cc63-046b-4c96-9602-b6dc6332a56d admin admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:33:38.505077 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e17cc63-046b-4c96-9602-b6dc6332a56d admin admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:33:38.511173 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61691aff-c192-4178-8950-511df93499d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76c5181fc7be45279ac00017bb4640b3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:33:38.512092 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61691aff-c192-4178-8950-511df93499d7 admin admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:33:38.512354 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61691aff-c192-4178-8950-511df93499d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:33:38.512640 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61691aff-c192-4178-8950-511df93499d7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:33:38.513571 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61691aff-c192-4178-8950-511df93499d7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:33:38.515388 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61691aff-c192-4178-8950-511df93499d7 admin admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:33:38.536102 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 1312/2618] 10.222.0.18 () {58 vars in 1122 bytes} [Tue Sep 13 03:33:38 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 03:33:38.540550 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d575bdc7-655d-4eb8-b582-973810e85ac2 admin admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:33:38.551863 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d575bdc7-655d-4eb8-b582-973810e85ac2 admin admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:33:38.556972 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c2f83e1-04dc-4d18-8097-980e82de34b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76c5181fc7be45279ac00017bb4640b3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:33:38.557779 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c2f83e1-04dc-4d18-8097-980e82de34b5 admin admin] REQUEST_METHOD: `PUT` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:33:38.558073 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c2f83e1-04dc-4d18-8097-980e82de34b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:33:38.558344 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c2f83e1-04dc-4d18-8097-980e82de34b5 admin admin] PATH_INFO: `/v3/projects/2693d127c85544419c09b59212308f0e/users/5628b55b44f14a248290d28e5bd130a9/roles/1eb7a5b5203945c29e6d22d653f47847` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:33:38.575237 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c2f83e1-04dc-4d18-8097-980e82de34b5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2693d127c85544419c09b59212308f0e, user_id=5628b55b44f14a248290d28e5bd130a9, role_id=1eb7a5b5203945c29e6d22d653f47847)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:33:38.577937 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c2f83e1-04dc-4d18-8097-980e82de34b5 admin admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:33:38.592827 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 1307/2619] 10.222.0.18 () {60 vars in 1483 bytes} [Tue Sep 13 03:33:38 2022] PUT /identity/v3/projects/2693d127c85544419c09b59212308f0e/users/5628b55b44f14a248290d28e5bd130a9/roles/1eb7a5b5203945c29e6d22d653f47847 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 03:33:38.597964 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca025f4-d857-40e4-8fa8-289df6186873 None None] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:33:38.598248 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca025f4-d857-40e4-8fa8-289df6186873 None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:33:38.598536 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca025f4-d857-40e4-8fa8-289df6186873 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:33:38.618199 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.auth.core [None req-7ca025f4-d857-40e4-8fa8-289df6186873 None None] MFA Rules not processed for user `49f280ab086b4610855c4ffa378517d9`. Rule list: `[]` (Enabled: `True`). {{(pid=81399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 03:33:38.619647 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-7ca025f4-d857-40e4-8fa8-289df6186873 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:33:38.685288 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 1313/2620] 10.222.0.18 () {58 vars in 958 bytes} [Tue Sep 13 03:33:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 03:33:38.695712 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c2f83e1-04dc-4d18-8097-980e82de34b5 admin admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:33:38.697381 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-7c2f83e1-04dc-4d18-8097-980e82de34b5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:33:38.755611 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c2f83e1-04dc-4d18-8097-980e82de34b5 admin admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:33:38.759267 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3da5bc4-0547-4211-8257-2f2d973ea8c8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdffbc851086441ebd01022d1e4c2609', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b9d94f6a68f7470d9494f112175620dc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:33:38.759859 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3da5bc4-0547-4211-8257-2f2d973ea8c8 service neutron] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:33:38.760141 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3da5bc4-0547-4211-8257-2f2d973ea8c8 service neutron] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:33:38.760430 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3da5bc4-0547-4211-8257-2f2d973ea8c8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:33:38.769782 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3da5bc4-0547-4211-8257-2f2d973ea8c8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:33:38.771263 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3da5bc4-0547-4211-8257-2f2d973ea8c8 service neutron] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:33:38.780749 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 1308/2621] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Sep 13 03:33:38 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 03:33:39.901161 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20cf1366-47dd-428f-9095-cf1e45083c48 None None] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:33:39.901703 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20cf1366-47dd-428f-9095-cf1e45083c48 None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:33:39.901849 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20cf1366-47dd-428f-9095-cf1e45083c48 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:33:39.941445 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.auth.core [None req-20cf1366-47dd-428f-9095-cf1e45083c48 None None] MFA Rules not processed for user `5628b55b44f14a248290d28e5bd130a9`. Rule list: `[]` (Enabled: `True`). {{(pid=81399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 03:33:39.942816 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-20cf1366-47dd-428f-9095-cf1e45083c48 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:33:40.019219 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 1314/2622] 10.222.0.18 () {58 vars in 958 bytes} [Tue Sep 13 03:33:39 2022] POST /identity/v3/auth/tokens => generated 3282 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 03:33:40.023506 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3da5bc4-0547-4211-8257-2f2d973ea8c8 service neutron] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:33:40.036366 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3da5bc4-0547-4211-8257-2f2d973ea8c8 service neutron] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:33:40.041959 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78b2cbb8-ce4a-4f64-9486-0b242af5d455 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76c5181fc7be45279ac00017bb4640b3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:33:40.042137 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b2cbb8-ce4a-4f64-9486-0b242af5d455 admin admin] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:33:40.042356 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b2cbb8-ce4a-4f64-9486-0b242af5d455 admin admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:33:40.043211 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b2cbb8-ce4a-4f64-9486-0b242af5d455 admin admin] PATH_INFO: `/v3/projects` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:33:40.044237 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78b2cbb8-ce4a-4f64-9486-0b242af5d455 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:33:40.045822 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78b2cbb8-ce4a-4f64-9486-0b242af5d455 admin admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:33:40.082682 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 1309/2623] 10.222.0.18 () {60 vars in 1153 bytes} [Tue Sep 13 03:33:40 2022] POST /identity/v3/projects => generated 405 bytes in 57 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 03:33:40.084532 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20cf1366-47dd-428f-9095-cf1e45083c48 None None] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:33:40.097341 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20cf1366-47dd-428f-9095-cf1e45083c48 None None] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:33:40.105334 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:33:40.156469 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d16673e-934a-4cde-9bd3-af91f6586510 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76c5181fc7be45279ac00017bb4640b3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:33:40.157078 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d16673e-934a-4cde-9bd3-af91f6586510 admin admin] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:33:40.157391 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d16673e-934a-4cde-9bd3-af91f6586510 admin admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:33:40.157685 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d16673e-934a-4cde-9bd3-af91f6586510 admin admin] PATH_INFO: `/v3/users` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:33:40.158597 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d16673e-934a-4cde-9bd3-af91f6586510 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:33:40.160918 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d16673e-934a-4cde-9bd3-af91f6586510 admin admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:33:40.182352 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 1315/2624] 10.222.0.18 () {60 vars in 1144 bytes} [Tue Sep 13 03:33:40 2022] POST /identity/v3/users => generated 309 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 03:33:40.186323 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78b2cbb8-ce4a-4f64-9486-0b242af5d455 admin admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:33:40.197687 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78b2cbb8-ce4a-4f64-9486-0b242af5d455 admin admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:33:40.202440 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4201520-efd4-4eaf-b9b5-1e25f89cee61 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76c5181fc7be45279ac00017bb4640b3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:33:40.203156 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4201520-efd4-4eaf-b9b5-1e25f89cee61 admin admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:33:40.203441 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4201520-efd4-4eaf-b9b5-1e25f89cee61 admin admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:33:40.203713 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4201520-efd4-4eaf-b9b5-1e25f89cee61 admin admin] PATH_INFO: `/v3/roles` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:33:40.204362 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4201520-efd4-4eaf-b9b5-1e25f89cee61 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:33:40.206717 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4201520-efd4-4eaf-b9b5-1e25f89cee61 admin admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:33:40.230334 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 1310/2625] 10.222.0.18 () {58 vars in 1122 bytes} [Tue Sep 13 03:33:40 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 03:33:40.234117 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d16673e-934a-4cde-9bd3-af91f6586510 admin admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:33:40.245872 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d16673e-934a-4cde-9bd3-af91f6586510 admin admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:33:40.251281 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fde0e00-f625-4862-9434-66a735cc0a26 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76c5181fc7be45279ac00017bb4640b3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:33:40.251955 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fde0e00-f625-4862-9434-66a735cc0a26 admin admin] REQUEST_METHOD: `PUT` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:33:40.252231 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fde0e00-f625-4862-9434-66a735cc0a26 admin admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:33:40.252502 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fde0e00-f625-4862-9434-66a735cc0a26 admin admin] PATH_INFO: `/v3/projects/a661c7d6b54349bc871a99ace196339e/users/df12da3d70114e28be25e2de2bd85404/roles/425d4b481c7846e0bc07e7e805d5b15f` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:33:40.271030 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fde0e00-f625-4862-9434-66a735cc0a26 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a661c7d6b54349bc871a99ace196339e, user_id=df12da3d70114e28be25e2de2bd85404, role_id=425d4b481c7846e0bc07e7e805d5b15f)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:33:40.273717 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fde0e00-f625-4862-9434-66a735cc0a26 admin admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:33:40.294288 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 1316/2626] 10.222.0.18 () {60 vars in 1483 bytes} [Tue Sep 13 03:33:40 2022] PUT /identity/v3/projects/a661c7d6b54349bc871a99ace196339e/users/df12da3d70114e28be25e2de2bd85404/roles/425d4b481c7846e0bc07e7e805d5b15f => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 03:33:40.298527 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4201520-efd4-4eaf-b9b5-1e25f89cee61 admin admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:33:40.310842 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4201520-efd4-4eaf-b9b5-1e25f89cee61 admin admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:33:40.359962 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:DELETE=4,keystone:SELECT=92,keystone:INSERT=4 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:33:40.388422 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6983688a-3393-425d-b8bc-4d0ba0c3d5c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76c5181fc7be45279ac00017bb4640b3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:33:40.389798 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6983688a-3393-425d-b8bc-4d0ba0c3d5c6 admin admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:33:40.390300 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6983688a-3393-425d-b8bc-4d0ba0c3d5c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:33:40.390879 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6983688a-3393-425d-b8bc-4d0ba0c3d5c6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:33:40.391759 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6983688a-3393-425d-b8bc-4d0ba0c3d5c6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:33:40.395592 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6983688a-3393-425d-b8bc-4d0ba0c3d5c6 admin admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:33:40.408463 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 1311/2627] 10.222.0.18 () {58 vars in 1122 bytes} [Tue Sep 13 03:33:40 2022] GET /identity/v3/roles => generated 1420 bytes in 110 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 03:33:40.414071 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fde0e00-f625-4862-9434-66a735cc0a26 admin admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:33:40.432640 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fde0e00-f625-4862-9434-66a735cc0a26 admin admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:33:40.439859 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e289952-78b7-4160-ab1d-0265fb33caa5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76c5181fc7be45279ac00017bb4640b3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:33:40.440540 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e289952-78b7-4160-ab1d-0265fb33caa5 admin admin] REQUEST_METHOD: `PUT` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:33:40.440845 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e289952-78b7-4160-ab1d-0265fb33caa5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:33:40.441143 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e289952-78b7-4160-ab1d-0265fb33caa5 admin admin] PATH_INFO: `/v3/projects/a661c7d6b54349bc871a99ace196339e/users/df12da3d70114e28be25e2de2bd85404/roles/425d4b481c7846e0bc07e7e805d5b15f` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:33:40.443413 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e289952-78b7-4160-ab1d-0265fb33caa5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a661c7d6b54349bc871a99ace196339e, user_id=df12da3d70114e28be25e2de2bd85404, role_id=425d4b481c7846e0bc07e7e805d5b15f)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:33:40.446068 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e289952-78b7-4160-ab1d-0265fb33caa5 admin admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:33:40.459320 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 1317/2628] 10.222.0.18 () {60 vars in 1483 bytes} [Tue Sep 13 03:33:40 2022] PUT /identity/v3/projects/a661c7d6b54349bc871a99ace196339e/users/df12da3d70114e28be25e2de2bd85404/roles/425d4b481c7846e0bc07e7e805d5b15f => generated 0 bytes in 46 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 03:33:40.536938 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6983688a-3393-425d-b8bc-4d0ba0c3d5c6 admin admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:33:40.538841 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-6983688a-3393-425d-b8bc-4d0ba0c3d5c6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:33:40.560708 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6983688a-3393-425d-b8bc-4d0ba0c3d5c6 admin admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:33:40.656273 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8664d9ed-30ce-4c36-86dc-02f31d7e334d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76c5181fc7be45279ac00017bb4640b3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:33:40.656680 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8664d9ed-30ce-4c36-86dc-02f31d7e334d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:33:40.657049 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8664d9ed-30ce-4c36-86dc-02f31d7e334d admin admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:33:40.657448 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8664d9ed-30ce-4c36-86dc-02f31d7e334d admin admin] PATH_INFO: `/v3/users/043054f3ad764b01aefdcc6062aa45af` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:33:40.659686 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8664d9ed-30ce-4c36-86dc-02f31d7e334d admin admin] RBAC: Authorizing `identity:delete_user(user_id=043054f3ad764b01aefdcc6062aa45af)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:33:40.662421 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8664d9ed-30ce-4c36-86dc-02f31d7e334d admin admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:33:40.747456 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-8664d9ed-30ce-4c36-86dc-02f31d7e334d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '043054f3ad764b01aefdcc6062aa45af'} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:33:40.758986 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-8664d9ed-30ce-4c36-86dc-02f31d7e334d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '043054f3ad764b01aefdcc6062aa45af'} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:33:40.759721 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-8664d9ed-30ce-4c36-86dc-02f31d7e334d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '043054f3ad764b01aefdcc6062aa45af'} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:33:40.776092 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-8664d9ed-30ce-4c36-86dc-02f31d7e334d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '043054f3ad764b01aefdcc6062aa45af'} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:33:40.786238 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-8664d9ed-30ce-4c36-86dc-02f31d7e334d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '043054f3ad764b01aefdcc6062aa45af'} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:33:40.795097 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 1312/2629] 10.222.0.18 () {58 vars in 1224 bytes} [Tue Sep 13 03:33:40 2022] DELETE /identity/v3/users/043054f3ad764b01aefdcc6062aa45af => generated 0 bytes in 259 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 03:33:40.799747 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-645954e6-fd77-4274-ab8e-125865476b8a None None] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:33:40.800058 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-645954e6-fd77-4274-ab8e-125865476b8a None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:33:40.800261 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-645954e6-fd77-4274-ab8e-125865476b8a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:33:40.826663 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.auth.core [None req-645954e6-fd77-4274-ab8e-125865476b8a None None] MFA Rules not processed for user `49f280ab086b4610855c4ffa378517d9`. Rule list: `[]` (Enabled: `True`). {{(pid=81399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 03:33:40.828068 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-645954e6-fd77-4274-ab8e-125865476b8a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:33:40.862924 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8664d9ed-30ce-4c36-86dc-02f31d7e334d admin admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:33:40.864373 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-8664d9ed-30ce-4c36-86dc-02f31d7e334d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:33:40.929706 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 1318/2630] 10.222.0.18 () {58 vars in 958 bytes} [Tue Sep 13 03:33:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 13 03:33:40.939620 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-645954e6-fd77-4274-ab8e-125865476b8a None None] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:33:40.941683 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-645954e6-fd77-4274-ab8e-125865476b8a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:33:40.980567 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:33:40.988829 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8664d9ed-30ce-4c36-86dc-02f31d7e334d admin admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:33:40.996356 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb8286a5-af3f-4456-b3f2-34c57e913b64 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76c5181fc7be45279ac00017bb4640b3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:33:40.997566 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb8286a5-af3f-4456-b3f2-34c57e913b64 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:33:40.998009 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb8286a5-af3f-4456-b3f2-34c57e913b64 admin admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:33:40.998652 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb8286a5-af3f-4456-b3f2-34c57e913b64 admin admin] PATH_INFO: `/v3/users/83bd9ab163ba4229bcfd810345430564` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:33:40.999597 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb8286a5-af3f-4456-b3f2-34c57e913b64 admin admin] RBAC: Authorizing `identity:delete_user(user_id=83bd9ab163ba4229bcfd810345430564)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:33:41.001601 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb8286a5-af3f-4456-b3f2-34c57e913b64 admin admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:33:41.020964 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-645954e6-fd77-4274-ab8e-125865476b8a None None] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:33:41.025858 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b50c16de-296a-470d-958c-872e89d5a9ba service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdffbc851086441ebd01022d1e4c2609', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b9d94f6a68f7470d9494f112175620dc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:33:41.026474 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b50c16de-296a-470d-958c-872e89d5a9ba service neutron] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:33:41.026790 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b50c16de-296a-470d-958c-872e89d5a9ba service neutron] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:33:41.027143 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b50c16de-296a-470d-958c-872e89d5a9ba service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:33:41.038234 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b50c16de-296a-470d-958c-872e89d5a9ba service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:33:41.039803 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b50c16de-296a-470d-958c-872e89d5a9ba service neutron] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:33:41.053281 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 1319/2631] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Sep 13 03:33:40 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 03:33:41.053477 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-bb8286a5-af3f-4456-b3f2-34c57e913b64 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '83bd9ab163ba4229bcfd810345430564'} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:33:41.062792 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-bb8286a5-af3f-4456-b3f2-34c57e913b64 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '83bd9ab163ba4229bcfd810345430564'} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:33:41.063598 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-bb8286a5-af3f-4456-b3f2-34c57e913b64 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '83bd9ab163ba4229bcfd810345430564'} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:33:41.080185 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-bb8286a5-af3f-4456-b3f2-34c57e913b64 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '83bd9ab163ba4229bcfd810345430564'} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:33:41.088582 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-bb8286a5-af3f-4456-b3f2-34c57e913b64 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '83bd9ab163ba4229bcfd810345430564'} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:33:41.102034 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 1313/2632] 10.222.0.18 () {58 vars in 1224 bytes} [Tue Sep 13 03:33:40 2022] DELETE /identity/v3/users/83bd9ab163ba4229bcfd810345430564 => generated 0 bytes in 237 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 13 03:33:41.278858 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-191e6f14-e1e0-4f8a-bb03-9ff6528e6bbd None None] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:33:41.279216 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-191e6f14-e1e0-4f8a-bb03-9ff6528e6bbd None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:33:41.280081 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-191e6f14-e1e0-4f8a-bb03-9ff6528e6bbd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:33:41.298580 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:33:41.354253 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.auth.core [None req-191e6f14-e1e0-4f8a-bb03-9ff6528e6bbd None None] MFA Rules not processed for user `df12da3d70114e28be25e2de2bd85404`. Rule list: `[]` (Enabled: `True`). {{(pid=81399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 03:33:41.358839 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-191e6f14-e1e0-4f8a-bb03-9ff6528e6bbd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:33:41.396127 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb8286a5-af3f-4456-b3f2-34c57e913b64 admin admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:33:41.398151 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-bb8286a5-af3f-4456-b3f2-34c57e913b64 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:33:41.501131 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 1320/2633] 10.222.0.18 () {58 vars in 958 bytes} [Tue Sep 13 03:33:41 2022] POST /identity/v3/auth/tokens => generated 3349 bytes in 223 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 03:33:41.502293 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-191e6f14-e1e0-4f8a-bb03-9ff6528e6bbd None None] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:33:41.504336 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-191e6f14-e1e0-4f8a-bb03-9ff6528e6bbd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:33:41.505555 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb8286a5-af3f-4456-b3f2-34c57e913b64 admin admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:33:41.516822 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b2dd559-caff-4b63-b780-975a12d016f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76c5181fc7be45279ac00017bb4640b3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:33:41.517008 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b2dd559-caff-4b63-b780-975a12d016f6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:33:41.517290 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b2dd559-caff-4b63-b780-975a12d016f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:33:41.517559 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b2dd559-caff-4b63-b780-975a12d016f6 admin admin] PATH_INFO: `/v3/projects/9b7c5a1fd8a24526b7859fdb030ccb95` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:33:41.519920 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b2dd559-caff-4b63-b780-975a12d016f6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9b7c5a1fd8a24526b7859fdb030ccb95)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:33:41.522017 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b2dd559-caff-4b63-b780-975a12d016f6 admin admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:33:41.529538 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-191e6f14-e1e0-4f8a-bb03-9ff6528e6bbd None None] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:33:41.536754 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-245faea5-d97c-4509-a758-a616a6d08c94 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76c5181fc7be45279ac00017bb4640b3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:33:41.538099 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-245faea5-d97c-4509-a758-a616a6d08c94 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:33:41.538472 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-245faea5-d97c-4509-a758-a616a6d08c94 admin admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:33:41.539156 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-245faea5-d97c-4509-a758-a616a6d08c94 admin admin] PATH_INFO: `/v3/projects/2a29b9a08d2c4529a33325b812b32440` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:33:41.541277 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-245faea5-d97c-4509-a758-a616a6d08c94 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2a29b9a08d2c4529a33325b812b32440)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:33:41.543831 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-245faea5-d97c-4509-a758-a616a6d08c94 admin admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:33:41.586495 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-8b2dd559-caff-4b63-b780-975a12d016f6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9b7c5a1fd8a24526b7859fdb030ccb95', 'request_id': 'req-8b2dd559-caff-4b63-b780-975a12d016f6'} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:33:41.596049 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-8b2dd559-caff-4b63-b780-975a12d016f6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9b7c5a1fd8a24526b7859fdb030ccb95', 'request_id': 'req-8b2dd559-caff-4b63-b780-975a12d016f6'} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:33:41.605680 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-8b2dd559-caff-4b63-b780-975a12d016f6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9b7c5a1fd8a24526b7859fdb030ccb95', 'request_id': 'req-8b2dd559-caff-4b63-b780-975a12d016f6'} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:33:41.619817 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-8b2dd559-caff-4b63-b780-975a12d016f6 admin admin] The token cache is being invalidate because project 9b7c5a1fd8a24526b7859fdb030ccb95 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81400) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 03:33:41.620706 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-8b2dd559-caff-4b63-b780-975a12d016f6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:33:41.621410 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-8b2dd559-caff-4b63-b780-975a12d016f6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:33:41.623056 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 1314/2634] 10.222.0.18 () {58 vars in 1233 bytes} [Tue Sep 13 03:33:41 2022] DELETE /identity/v3/projects/9b7c5a1fd8a24526b7859fdb030ccb95 => generated 0 bytes in 227 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 03:33:41.629997 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e840ccc1-5a3a-450e-8707-1dc74a32b1c9 None None] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:33:41.630279 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e840ccc1-5a3a-450e-8707-1dc74a32b1c9 None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:33:41.630581 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e840ccc1-5a3a-450e-8707-1dc74a32b1c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:33:41.632733 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-245faea5-d97c-4509-a758-a616a6d08c94 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2a29b9a08d2c4529a33325b812b32440', 'request_id': 'req-245faea5-d97c-4509-a758-a616a6d08c94'} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:33:41.639183 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-245faea5-d97c-4509-a758-a616a6d08c94 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2a29b9a08d2c4529a33325b812b32440', 'request_id': 'req-245faea5-d97c-4509-a758-a616a6d08c94'} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:33:41.640458 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:33:41.648861 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-245faea5-d97c-4509-a758-a616a6d08c94 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2a29b9a08d2c4529a33325b812b32440', 'request_id': 'req-245faea5-d97c-4509-a758-a616a6d08c94'} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:33:41.656049 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.auth.core [None req-e840ccc1-5a3a-450e-8707-1dc74a32b1c9 None None] MFA Rules not processed for user `49f280ab086b4610855c4ffa378517d9`. Rule list: `[]` (Enabled: `True`). {{(pid=81400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 03:33:41.657249 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-e840ccc1-5a3a-450e-8707-1dc74a32b1c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:33:41.669331 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-245faea5-d97c-4509-a758-a616a6d08c94 admin admin] The token cache is being invalidate because project 2a29b9a08d2c4529a33325b812b32440 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81399) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 03:33:41.669628 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-245faea5-d97c-4509-a758-a616a6d08c94 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:33:41.670268 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-245faea5-d97c-4509-a758-a616a6d08c94 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:33:41.672011 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 1321/2635] 10.222.0.18 () {58 vars in 1233 bytes} [Tue Sep 13 03:33:41 2022] DELETE /identity/v3/projects/2a29b9a08d2c4529a33325b812b32440 => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 03:33:41.733147 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 1315/2636] 10.222.0.18 () {58 vars in 958 bytes} [Tue Sep 13 03:33:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 03:33:41.741793 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f871d2-eeba-4353-936b-9433e0ec0fa5 None None] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:33:41.742077 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f871d2-eeba-4353-936b-9433e0ec0fa5 None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:33:41.742366 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f871d2-eeba-4353-936b-9433e0ec0fa5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:33:41.766550 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.auth.core [None req-c2f871d2-eeba-4353-936b-9433e0ec0fa5 None None] MFA Rules not processed for user `49f280ab086b4610855c4ffa378517d9`. Rule list: `[]` (Enabled: `True`). {{(pid=81399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 03:33:41.767928 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-c2f871d2-eeba-4353-936b-9433e0ec0fa5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:33:41.777132 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 1322/2637] 10.222.0.18 () {58 vars in 958 bytes} [Tue Sep 13 03:33:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 03:33:41.782249 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e840ccc1-5a3a-450e-8707-1dc74a32b1c9 None None] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:33:41.793688 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e840ccc1-5a3a-450e-8707-1dc74a32b1c9 None None] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:33:41.798748 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4f64a65-3c8d-460c-90ff-0dc7aeb870e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76c5181fc7be45279ac00017bb4640b3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:33:41.799451 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f64a65-3c8d-460c-90ff-0dc7aeb870e3 admin admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:33:41.800381 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f64a65-3c8d-460c-90ff-0dc7aeb870e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:33:41.800712 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f64a65-3c8d-460c-90ff-0dc7aeb870e3 admin admin] PATH_INFO: `/v3/domains` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:33:41.801644 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4f64a65-3c8d-460c-90ff-0dc7aeb870e3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 03:33:41.802019 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4f64a65-3c8d-460c-90ff-0dc7aeb870e3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:33:41.804082 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4f64a65-3c8d-460c-90ff-0dc7aeb870e3 admin admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:33:41.822115 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 1316/2638] 10.222.0.18 () {58 vars in 1153 bytes} [Tue Sep 13 03:33:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 03:33:41.823297 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2f871d2-eeba-4353-936b-9433e0ec0fa5 None None] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:33:41.833163 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2f871d2-eeba-4353-936b-9433e0ec0fa5 None None] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:33:41.838151 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e1e18a0-23a8-4ade-9462-c92e81823953 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76c5181fc7be45279ac00017bb4640b3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:33:41.838808 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e1e18a0-23a8-4ade-9462-c92e81823953 admin admin] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:33:41.839085 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e1e18a0-23a8-4ade-9462-c92e81823953 admin admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:33:41.839350 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e1e18a0-23a8-4ade-9462-c92e81823953 admin admin] PATH_INFO: `/v3/projects` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:33:41.839998 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e1e18a0-23a8-4ade-9462-c92e81823953 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:33:41.841923 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e1e18a0-23a8-4ade-9462-c92e81823953 admin admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:33:41.881532 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 1323/2639] 10.222.0.18 () {60 vars in 1153 bytes} [Tue Sep 13 03:33:41 2022] POST /identity/v3/projects => generated 409 bytes in 55 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 13 03:33:41.882970 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4f64a65-3c8d-460c-90ff-0dc7aeb870e3 admin admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:33:41.894589 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4f64a65-3c8d-460c-90ff-0dc7aeb870e3 admin admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:33:41.958067 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ff69ae9-0741-43bd-a373-d8c0e91dee28 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76c5181fc7be45279ac00017bb4640b3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:33:41.958720 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff69ae9-0741-43bd-a373-d8c0e91dee28 admin admin] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:33:41.959008 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff69ae9-0741-43bd-a373-d8c0e91dee28 admin admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:33:41.959280 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff69ae9-0741-43bd-a373-d8c0e91dee28 admin admin] PATH_INFO: `/v3/users` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:33:41.959951 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ff69ae9-0741-43bd-a373-d8c0e91dee28 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:33:41.961927 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ff69ae9-0741-43bd-a373-d8c0e91dee28 admin admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:33:41.983346 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 1317/2640] 10.222.0.18 () {60 vars in 1144 bytes} [Tue Sep 13 03:33:41 2022] POST /identity/v3/users => generated 305 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 03:33:41.987956 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e1e18a0-23a8-4ade-9462-c92e81823953 admin admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:33:41.999026 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e1e18a0-23a8-4ade-9462-c92e81823953 admin admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:33:42.004196 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cc2388c-33ba-4a5b-b1ff-e9ecd7d25612 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76c5181fc7be45279ac00017bb4640b3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:33:42.004783 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc2388c-33ba-4a5b-b1ff-e9ecd7d25612 admin admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:33:42.005054 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc2388c-33ba-4a5b-b1ff-e9ecd7d25612 admin admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:33:42.005317 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc2388c-33ba-4a5b-b1ff-e9ecd7d25612 admin admin] PATH_INFO: `/v3/roles` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:33:42.005947 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cc2388c-33ba-4a5b-b1ff-e9ecd7d25612 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:33:42.007555 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cc2388c-33ba-4a5b-b1ff-e9ecd7d25612 admin admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:33:42.017676 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 1324/2641] 10.222.0.18 () {58 vars in 1122 bytes} [Tue Sep 13 03:33:41 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 03:33:42.023349 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ff69ae9-0741-43bd-a373-d8c0e91dee28 admin admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:33:42.033436 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ff69ae9-0741-43bd-a373-d8c0e91dee28 admin admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:33:42.039544 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-344ce24f-c427-4196-a3c4-c81709ebb976 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76c5181fc7be45279ac00017bb4640b3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:33:42.040267 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-344ce24f-c427-4196-a3c4-c81709ebb976 admin admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:33:42.040640 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-344ce24f-c427-4196-a3c4-c81709ebb976 admin admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:33:42.041245 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-344ce24f-c427-4196-a3c4-c81709ebb976 admin admin] PATH_INFO: `/v3/roles` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:33:42.042145 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-344ce24f-c427-4196-a3c4-c81709ebb976 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:33:42.043865 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-344ce24f-c427-4196-a3c4-c81709ebb976 admin admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:33:42.059901 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 1318/2642] 10.222.0.18 () {58 vars in 1122 bytes} [Tue Sep 13 03:33:42 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 03:33:42.061921 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cc2388c-33ba-4a5b-b1ff-e9ecd7d25612 admin admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:33:42.075762 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cc2388c-33ba-4a5b-b1ff-e9ecd7d25612 admin admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:33:42.080665 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3640896a-abfe-49c3-bb8a-13ebc0681f9d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76c5181fc7be45279ac00017bb4640b3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:33:42.081399 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3640896a-abfe-49c3-bb8a-13ebc0681f9d admin admin] REQUEST_METHOD: `PUT` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:33:42.081776 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3640896a-abfe-49c3-bb8a-13ebc0681f9d admin admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:33:42.082109 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3640896a-abfe-49c3-bb8a-13ebc0681f9d admin admin] PATH_INFO: `/v3/projects/a0f237a0241c41fda216cee0dbd07ed0/users/fcb952fb95524cb1af9528f13e700df1/roles/1eb7a5b5203945c29e6d22d653f47847` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:33:42.098303 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3640896a-abfe-49c3-bb8a-13ebc0681f9d admin admin] RBAC: Authorizing `identity:create_grant(project_id=a0f237a0241c41fda216cee0dbd07ed0, user_id=fcb952fb95524cb1af9528f13e700df1, role_id=1eb7a5b5203945c29e6d22d653f47847)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:33:42.101005 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3640896a-abfe-49c3-bb8a-13ebc0681f9d admin admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:33:42.122693 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 1325/2643] 10.222.0.18 () {60 vars in 1483 bytes} [Tue Sep 13 03:33:42 2022] PUT /identity/v3/projects/a0f237a0241c41fda216cee0dbd07ed0/users/fcb952fb95524cb1af9528f13e700df1/roles/1eb7a5b5203945c29e6d22d653f47847 => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 03:33:42.128936 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a51a1c-ccc4-4270-b32c-8e01420641c5 None None] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:33:42.129275 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a51a1c-ccc4-4270-b32c-8e01420641c5 None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:33:42.129566 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a51a1c-ccc4-4270-b32c-8e01420641c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:33:42.157644 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.auth.core [None req-34a51a1c-ccc4-4270-b32c-8e01420641c5 None None] MFA Rules not processed for user `fcb952fb95524cb1af9528f13e700df1`. Rule list: `[]` (Enabled: `True`). {{(pid=81400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 03:33:42.158887 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-34a51a1c-ccc4-4270-b32c-8e01420641c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:33:42.221048 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:33:42.229085 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 1319/2644] 10.222.0.18 () {58 vars in 958 bytes} [Tue Sep 13 03:33:42 2022] POST /identity/v3/auth/tokens => generated 3286 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 13 03:33:42.230258 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3640896a-abfe-49c3-bb8a-13ebc0681f9d admin admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:33:42.243400 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:33:42.297077 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3640896a-abfe-49c3-bb8a-13ebc0681f9d admin admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:33:42.302661 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41664e74-bae9-4514-b01d-e9888cbf1abd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76c5181fc7be45279ac00017bb4640b3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:33:42.303269 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41664e74-bae9-4514-b01d-e9888cbf1abd admin admin] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:33:42.303548 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41664e74-bae9-4514-b01d-e9888cbf1abd admin admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:33:42.304071 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41664e74-bae9-4514-b01d-e9888cbf1abd admin admin] PATH_INFO: `/v3/projects` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:33:42.304443 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41664e74-bae9-4514-b01d-e9888cbf1abd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:33:42.306353 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41664e74-bae9-4514-b01d-e9888cbf1abd admin admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:33:42.335974 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 1326/2645] 10.222.0.18 () {60 vars in 1153 bytes} [Tue Sep 13 03:33:42 2022] POST /identity/v3/projects => generated 407 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 03:33:42.337258 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34a51a1c-ccc4-4270-b32c-8e01420641c5 None None] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:33:42.349155 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34a51a1c-ccc4-4270-b32c-8e01420641c5 None None] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:33:42.405323 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b8295b1-68ca-4b89-b49b-0cac2fff6065 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76c5181fc7be45279ac00017bb4640b3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:33:42.405967 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b8295b1-68ca-4b89-b49b-0cac2fff6065 admin admin] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:33:42.406195 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b8295b1-68ca-4b89-b49b-0cac2fff6065 admin admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:33:42.406466 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b8295b1-68ca-4b89-b49b-0cac2fff6065 admin admin] PATH_INFO: `/v3/users` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:33:42.407149 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b8295b1-68ca-4b89-b49b-0cac2fff6065 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:33:42.409062 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b8295b1-68ca-4b89-b49b-0cac2fff6065 admin admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:33:42.433564 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 1320/2646] 10.222.0.18 () {60 vars in 1144 bytes} [Tue Sep 13 03:33:42 2022] POST /identity/v3/users => generated 310 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 13 03:33:42.434785 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41664e74-bae9-4514-b01d-e9888cbf1abd admin admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:33:42.444935 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41664e74-bae9-4514-b01d-e9888cbf1abd admin admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:33:42.449581 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-198de452-5562-4f75-bfa3-46c722858219 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76c5181fc7be45279ac00017bb4640b3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:33:42.450175 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-198de452-5562-4f75-bfa3-46c722858219 admin admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:33:42.450451 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-198de452-5562-4f75-bfa3-46c722858219 admin admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:33:42.450759 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-198de452-5562-4f75-bfa3-46c722858219 admin admin] PATH_INFO: `/v3/roles` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:33:42.451393 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-198de452-5562-4f75-bfa3-46c722858219 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:33:42.453004 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-198de452-5562-4f75-bfa3-46c722858219 admin admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:33:42.464310 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 1327/2647] 10.222.0.18 () {58 vars in 1122 bytes} [Tue Sep 13 03:33:42 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 03:33:42.468613 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b8295b1-68ca-4b89-b49b-0cac2fff6065 admin admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:33:42.479551 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b8295b1-68ca-4b89-b49b-0cac2fff6065 admin admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:33:42.484489 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edb463ff-10f9-4d8a-82f4-fcff9fab4b3c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76c5181fc7be45279ac00017bb4640b3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:33:42.485230 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edb463ff-10f9-4d8a-82f4-fcff9fab4b3c admin admin] REQUEST_METHOD: `PUT` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:33:42.485510 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edb463ff-10f9-4d8a-82f4-fcff9fab4b3c admin admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:33:42.485782 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edb463ff-10f9-4d8a-82f4-fcff9fab4b3c admin admin] PATH_INFO: `/v3/projects/152f125031724eafaa495afbec83d9cb/users/cc15e637acd64efb864f2e0d3aa6e307/roles/425d4b481c7846e0bc07e7e805d5b15f` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:33:42.501942 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edb463ff-10f9-4d8a-82f4-fcff9fab4b3c admin admin] RBAC: Authorizing `identity:create_grant(project_id=152f125031724eafaa495afbec83d9cb, user_id=cc15e637acd64efb864f2e0d3aa6e307, role_id=425d4b481c7846e0bc07e7e805d5b15f)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:33:42.504728 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edb463ff-10f9-4d8a-82f4-fcff9fab4b3c admin admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:33:42.519557 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 1321/2648] 10.222.0.18 () {60 vars in 1483 bytes} [Tue Sep 13 03:33:42 2022] PUT /identity/v3/projects/152f125031724eafaa495afbec83d9cb/users/cc15e637acd64efb864f2e0d3aa6e307/roles/425d4b481c7846e0bc07e7e805d5b15f => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 03:33:42.523692 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-198de452-5562-4f75-bfa3-46c722858219 admin admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:33:42.533964 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-198de452-5562-4f75-bfa3-46c722858219 admin admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:33:42.594836 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-580ba2d8-d729-433d-91fb-5d3a7cd1aec5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76c5181fc7be45279ac00017bb4640b3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:33:42.595360 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-580ba2d8-d729-433d-91fb-5d3a7cd1aec5 admin admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:33:42.595640 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-580ba2d8-d729-433d-91fb-5d3a7cd1aec5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:33:42.595914 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-580ba2d8-d729-433d-91fb-5d3a7cd1aec5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:33:42.596556 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-580ba2d8-d729-433d-91fb-5d3a7cd1aec5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:33:42.598799 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-580ba2d8-d729-433d-91fb-5d3a7cd1aec5 admin admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:33:42.609938 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 1328/2649] 10.222.0.18 () {58 vars in 1122 bytes} [Tue Sep 13 03:33:42 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 03:33:42.614155 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edb463ff-10f9-4d8a-82f4-fcff9fab4b3c admin admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:33:42.624803 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edb463ff-10f9-4d8a-82f4-fcff9fab4b3c admin admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:33:42.630922 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a8d2fb1-54f4-45b1-ad97-cab20451a1c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76c5181fc7be45279ac00017bb4640b3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:33:42.631596 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a8d2fb1-54f4-45b1-ad97-cab20451a1c8 admin admin] REQUEST_METHOD: `PUT` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:33:42.631874 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a8d2fb1-54f4-45b1-ad97-cab20451a1c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:33:42.632188 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a8d2fb1-54f4-45b1-ad97-cab20451a1c8 admin admin] PATH_INFO: `/v3/projects/152f125031724eafaa495afbec83d9cb/users/cc15e637acd64efb864f2e0d3aa6e307/roles/425d4b481c7846e0bc07e7e805d5b15f` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:33:42.634134 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a8d2fb1-54f4-45b1-ad97-cab20451a1c8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=152f125031724eafaa495afbec83d9cb, user_id=cc15e637acd64efb864f2e0d3aa6e307, role_id=425d4b481c7846e0bc07e7e805d5b15f)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:33:42.636997 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a8d2fb1-54f4-45b1-ad97-cab20451a1c8 admin admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:33:42.648730 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 1322/2650] 10.222.0.18 () {60 vars in 1483 bytes} [Tue Sep 13 03:33:42 2022] PUT /identity/v3/projects/152f125031724eafaa495afbec83d9cb/users/cc15e637acd64efb864f2e0d3aa6e307/roles/425d4b481c7846e0bc07e7e805d5b15f => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 03:33:42.659003 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98b4aae9-9e97-452f-add3-f4433317de61 None None] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:33:42.659896 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98b4aae9-9e97-452f-add3-f4433317de61 None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:33:42.660172 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98b4aae9-9e97-452f-add3-f4433317de61 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:33:42.690172 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.auth.core [None req-98b4aae9-9e97-452f-add3-f4433317de61 None None] MFA Rules not processed for user `cc15e637acd64efb864f2e0d3aa6e307`. Rule list: `[]` (Enabled: `True`). {{(pid=81399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 03:33:42.691472 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-98b4aae9-9e97-452f-add3-f4433317de61 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:33:42.765962 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 1329/2651] 10.222.0.18 () {58 vars in 958 bytes} [Tue Sep 13 03:33:42 2022] POST /identity/v3/auth/tokens => generated 3351 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 03:33:42.778012 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a8d2fb1-54f4-45b1-ad97-cab20451a1c8 admin admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:33:42.780769 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-3a8d2fb1-54f4-45b1-ad97-cab20451a1c8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:33:42.843529 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a8d2fb1-54f4-45b1-ad97-cab20451a1c8 admin admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:33:42.848782 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98b4aae9-9e97-452f-add3-f4433317de61 None None] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:33:42.850223 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b854aabb-7ed2-491f-ada8-67b789f3ce3e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '924ee3f608d34a18b0ddc4695f102a56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b9d94f6a68f7470d9494f112175620dc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:33:42.850380 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-98b4aae9-9e97-452f-add3-f4433317de61 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:33:42.851191 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b854aabb-7ed2-491f-ada8-67b789f3ce3e service cinder] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:33:42.851632 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b854aabb-7ed2-491f-ada8-67b789f3ce3e service cinder] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:33:42.852131 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b854aabb-7ed2-491f-ada8-67b789f3ce3e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:33:42.864189 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:33:42.865526 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b854aabb-7ed2-491f-ada8-67b789f3ce3e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:33:42.867772 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b854aabb-7ed2-491f-ada8-67b789f3ce3e service cinder] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:33:42.880704 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 1323/2652] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Sep 13 03:33:42 2022] GET /identity/v3/auth/tokens => generated 3351 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 03:33:42.929876 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98b4aae9-9e97-452f-add3-f4433317de61 None None] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:33:42.934420 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76616429-f87d-4b86-b28b-9125c52ebac1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76c5181fc7be45279ac00017bb4640b3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:33:42.935253 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76616429-f87d-4b86-b28b-9125c52ebac1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:33:42.935531 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76616429-f87d-4b86-b28b-9125c52ebac1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:33:42.935796 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76616429-f87d-4b86-b28b-9125c52ebac1 admin admin] PATH_INFO: `/v3/users/5628b55b44f14a248290d28e5bd130a9` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:33:42.936915 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76616429-f87d-4b86-b28b-9125c52ebac1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5628b55b44f14a248290d28e5bd130a9)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:33:42.938850 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76616429-f87d-4b86-b28b-9125c52ebac1 admin admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:33:42.991023 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-76616429-f87d-4b86-b28b-9125c52ebac1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5628b55b44f14a248290d28e5bd130a9'} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:33:42.999161 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-76616429-f87d-4b86-b28b-9125c52ebac1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5628b55b44f14a248290d28e5bd130a9'} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:33:43.013062 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-76616429-f87d-4b86-b28b-9125c52ebac1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5628b55b44f14a248290d28e5bd130a9'} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:33:43.013754 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-76616429-f87d-4b86-b28b-9125c52ebac1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5628b55b44f14a248290d28e5bd130a9'} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:33:43.021357 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-76616429-f87d-4b86-b28b-9125c52ebac1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5628b55b44f14a248290d28e5bd130a9'} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:33:43.027512 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 1330/2653] 10.222.0.18 () {58 vars in 1224 bytes} [Tue Sep 13 03:33:42 2022] DELETE /identity/v3/users/5628b55b44f14a248290d28e5bd130a9 => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 13 03:33:43.043548 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b854aabb-7ed2-491f-ada8-67b789f3ce3e service cinder] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:33:43.052153 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b854aabb-7ed2-491f-ada8-67b789f3ce3e service cinder] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:33:43.106587 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01cd876f-b822-4a41-8871-8c80a0ef9405 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '924ee3f608d34a18b0ddc4695f102a56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b9d94f6a68f7470d9494f112175620dc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:33:43.107954 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01cd876f-b822-4a41-8871-8c80a0ef9405 service cinder] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:33:43.108253 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01cd876f-b822-4a41-8871-8c80a0ef9405 service cinder] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:33:43.108522 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01cd876f-b822-4a41-8871-8c80a0ef9405 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:33:43.110707 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-01cd876f-b822-4a41-8871-8c80a0ef9405 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:33:43.121759 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:33:43.177120 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01cd876f-b822-4a41-8871-8c80a0ef9405 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:33:43.178568 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01cd876f-b822-4a41-8871-8c80a0ef9405 service cinder] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:33:43.187834 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 1324/2654] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Sep 13 03:33:43 2022] GET /identity/v3/auth/tokens => generated 3286 bytes in 145 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 03:33:43.210266 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76616429-f87d-4b86-b28b-9125c52ebac1 admin admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:33:43.212116 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-76616429-f87d-4b86-b28b-9125c52ebac1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:33:43.281535 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76616429-f87d-4b86-b28b-9125c52ebac1 admin admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:33:43.286140 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a71ae56-0d9b-416f-b107-9a40e0251125 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76c5181fc7be45279ac00017bb4640b3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:33:43.287069 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a71ae56-0d9b-416f-b107-9a40e0251125 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:33:43.287346 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a71ae56-0d9b-416f-b107-9a40e0251125 admin admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:33:43.287611 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a71ae56-0d9b-416f-b107-9a40e0251125 admin admin] PATH_INFO: `/v3/projects/2693d127c85544419c09b59212308f0e` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:33:43.288718 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a71ae56-0d9b-416f-b107-9a40e0251125 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2693d127c85544419c09b59212308f0e)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:33:43.290624 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a71ae56-0d9b-416f-b107-9a40e0251125 admin admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:33:43.325836 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:33:43.345004 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-0a71ae56-0d9b-416f-b107-9a40e0251125 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2693d127c85544419c09b59212308f0e', 'request_id': 'req-0a71ae56-0d9b-416f-b107-9a40e0251125'} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:33:43.350882 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-0a71ae56-0d9b-416f-b107-9a40e0251125 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2693d127c85544419c09b59212308f0e', 'request_id': 'req-0a71ae56-0d9b-416f-b107-9a40e0251125'} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:33:43.356936 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-0a71ae56-0d9b-416f-b107-9a40e0251125 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2693d127c85544419c09b59212308f0e', 'request_id': 'req-0a71ae56-0d9b-416f-b107-9a40e0251125'} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:33:43.370717 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-0a71ae56-0d9b-416f-b107-9a40e0251125 admin admin] The token cache is being invalidate because project 2693d127c85544419c09b59212308f0e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81399) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 03:33:43.371009 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-0a71ae56-0d9b-416f-b107-9a40e0251125 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:33:43.371741 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-0a71ae56-0d9b-416f-b107-9a40e0251125 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:33:43.373220 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 1331/2655] 10.222.0.18 () {58 vars in 1233 bytes} [Tue Sep 13 03:33:43 2022] DELETE /identity/v3/projects/2693d127c85544419c09b59212308f0e => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 03:33:43.377459 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01cd876f-b822-4a41-8871-8c80a0ef9405 service cinder] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:33:43.389193 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01cd876f-b822-4a41-8871-8c80a0ef9405 service cinder] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:33:43.458226 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c03a425-2379-4e22-a033-575e9d2ce3e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76c5181fc7be45279ac00017bb4640b3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:33:43.459031 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c03a425-2379-4e22-a033-575e9d2ce3e4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:33:43.459379 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c03a425-2379-4e22-a033-575e9d2ce3e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:33:43.459724 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c03a425-2379-4e22-a033-575e9d2ce3e4 admin admin] PATH_INFO: `/v3/users/df12da3d70114e28be25e2de2bd85404` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:33:43.461047 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c03a425-2379-4e22-a033-575e9d2ce3e4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=df12da3d70114e28be25e2de2bd85404)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:33:43.463263 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c03a425-2379-4e22-a033-575e9d2ce3e4 admin admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:33:43.524379 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-0c03a425-2379-4e22-a033-575e9d2ce3e4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'df12da3d70114e28be25e2de2bd85404'} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:33:43.531312 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-0c03a425-2379-4e22-a033-575e9d2ce3e4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'df12da3d70114e28be25e2de2bd85404'} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:33:43.531991 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-0c03a425-2379-4e22-a033-575e9d2ce3e4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'df12da3d70114e28be25e2de2bd85404'} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:33:43.545089 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-0c03a425-2379-4e22-a033-575e9d2ce3e4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'df12da3d70114e28be25e2de2bd85404'} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:33:43.552519 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-0c03a425-2379-4e22-a033-575e9d2ce3e4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'df12da3d70114e28be25e2de2bd85404'} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:33:43.558263 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 1325/2656] 10.222.0.18 () {58 vars in 1224 bytes} [Tue Sep 13 03:33:43 2022] DELETE /identity/v3/users/df12da3d70114e28be25e2de2bd85404 => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 03:33:43.732922 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a71ae56-0d9b-416f-b107-9a40e0251125 admin admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:33:43.734408 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-0a71ae56-0d9b-416f-b107-9a40e0251125 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:33:43.748443 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a71ae56-0d9b-416f-b107-9a40e0251125 admin admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:33:43.804818 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3be20da0-9554-4d13-abd2-81a28805474e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76c5181fc7be45279ac00017bb4640b3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:33:43.805476 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3be20da0-9554-4d13-abd2-81a28805474e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:33:43.805754 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3be20da0-9554-4d13-abd2-81a28805474e admin admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:33:43.806023 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3be20da0-9554-4d13-abd2-81a28805474e admin admin] PATH_INFO: `/v3/projects/a661c7d6b54349bc871a99ace196339e` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:33:43.807153 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3be20da0-9554-4d13-abd2-81a28805474e admin admin] RBAC: Authorizing `identity:delete_project(project_id=a661c7d6b54349bc871a99ace196339e)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:33:43.809043 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3be20da0-9554-4d13-abd2-81a28805474e admin admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:33:43.854331 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-3be20da0-9554-4d13-abd2-81a28805474e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a661c7d6b54349bc871a99ace196339e', 'request_id': 'req-3be20da0-9554-4d13-abd2-81a28805474e'} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:33:43.860449 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-3be20da0-9554-4d13-abd2-81a28805474e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a661c7d6b54349bc871a99ace196339e', 'request_id': 'req-3be20da0-9554-4d13-abd2-81a28805474e'} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:33:43.867244 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-3be20da0-9554-4d13-abd2-81a28805474e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a661c7d6b54349bc871a99ace196339e', 'request_id': 'req-3be20da0-9554-4d13-abd2-81a28805474e'} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:33:43.879141 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-3be20da0-9554-4d13-abd2-81a28805474e admin admin] The token cache is being invalidate because project a661c7d6b54349bc871a99ace196339e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81399) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 03:33:43.879432 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-3be20da0-9554-4d13-abd2-81a28805474e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:33:43.880062 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-3be20da0-9554-4d13-abd2-81a28805474e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:33:43.881569 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 1332/2657] 10.222.0.18 () {58 vars in 1233 bytes} [Tue Sep 13 03:33:43 2022] DELETE /identity/v3/projects/a661c7d6b54349bc871a99ace196339e => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 03:33:43.886482 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf0d1016-a6b5-4b07-8689-772700535720 None None] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:33:43.886792 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf0d1016-a6b5-4b07-8689-772700535720 None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:33:43.887065 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf0d1016-a6b5-4b07-8689-772700535720 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:33:43.906552 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.auth.core [None req-cf0d1016-a6b5-4b07-8689-772700535720 None None] MFA Rules not processed for user `49f280ab086b4610855c4ffa378517d9`. Rule list: `[]` (Enabled: `True`). {{(pid=81400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 03:33:43.907745 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-cf0d1016-a6b5-4b07-8689-772700535720 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:33:43.931858 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:33:43.968538 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 1326/2658] 10.222.0.18 () {58 vars in 958 bytes} [Tue Sep 13 03:33:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 03:33:43.976721 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad0727af-66a2-467f-9dc0-33daab72aa6c None None] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:33:43.977002 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad0727af-66a2-467f-9dc0-33daab72aa6c None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:33:43.977271 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad0727af-66a2-467f-9dc0-33daab72aa6c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:33:43.996721 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.auth.core [None req-ad0727af-66a2-467f-9dc0-33daab72aa6c None None] MFA Rules not processed for user `49f280ab086b4610855c4ffa378517d9`. Rule list: `[]` (Enabled: `True`). {{(pid=81399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 03:33:43.997916 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-ad0727af-66a2-467f-9dc0-33daab72aa6c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:33:44.005930 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 1333/2659] 10.222.0.18 () {58 vars in 958 bytes} [Tue Sep 13 03:33:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 03:33:44.010120 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf0d1016-a6b5-4b07-8689-772700535720 None None] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:33:44.020322 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf0d1016-a6b5-4b07-8689-772700535720 None None] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:33:44.025075 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0d6cded-9ecf-4fda-941e-5df6f8c849ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76c5181fc7be45279ac00017bb4640b3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:33:44.025707 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d6cded-9ecf-4fda-941e-5df6f8c849ef admin admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:33:44.025987 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d6cded-9ecf-4fda-941e-5df6f8c849ef admin admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:33:44.026257 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d6cded-9ecf-4fda-941e-5df6f8c849ef admin admin] PATH_INFO: `/v3/domains` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:33:44.026926 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0d6cded-9ecf-4fda-941e-5df6f8c849ef admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 03:33:44.027252 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0d6cded-9ecf-4fda-941e-5df6f8c849ef admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:33:44.028844 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0d6cded-9ecf-4fda-941e-5df6f8c849ef admin admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:33:44.041084 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 1327/2660] 10.222.0.18 () {58 vars in 1153 bytes} [Tue Sep 13 03:33:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 03:33:44.045131 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad0727af-66a2-467f-9dc0-33daab72aa6c None None] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:33:44.054107 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad0727af-66a2-467f-9dc0-33daab72aa6c None None] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:33:44.058774 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dde4067a-64ce-45de-8f2b-0dddc458f2a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76c5181fc7be45279ac00017bb4640b3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:33:44.059387 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dde4067a-64ce-45de-8f2b-0dddc458f2a9 admin admin] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:33:44.059664 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dde4067a-64ce-45de-8f2b-0dddc458f2a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:33:44.059934 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dde4067a-64ce-45de-8f2b-0dddc458f2a9 admin admin] PATH_INFO: `/v3/projects` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:33:44.060595 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dde4067a-64ce-45de-8f2b-0dddc458f2a9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:33:44.062501 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dde4067a-64ce-45de-8f2b-0dddc458f2a9 admin admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:33:44.090574 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 1334/2661] 10.222.0.18 () {60 vars in 1153 bytes} [Tue Sep 13 03:33:44 2022] POST /identity/v3/projects => generated 379 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 13 03:33:44.094679 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0d6cded-9ecf-4fda-941e-5df6f8c849ef admin admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:33:44.105445 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0d6cded-9ecf-4fda-941e-5df6f8c849ef admin admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:33:44.160576 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ee6af43-2074-42dc-a7a3-25307bdeea51 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76c5181fc7be45279ac00017bb4640b3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:33:44.161384 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee6af43-2074-42dc-a7a3-25307bdeea51 admin admin] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:33:44.161758 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee6af43-2074-42dc-a7a3-25307bdeea51 admin admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:33:44.161957 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee6af43-2074-42dc-a7a3-25307bdeea51 admin admin] PATH_INFO: `/v3/users` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:33:44.162638 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ee6af43-2074-42dc-a7a3-25307bdeea51 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:33:44.164596 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ee6af43-2074-42dc-a7a3-25307bdeea51 admin admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:33:44.184997 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 1328/2662] 10.222.0.18 () {60 vars in 1144 bytes} [Tue Sep 13 03:33:44 2022] POST /identity/v3/users => generated 290 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 13 03:33:44.189077 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dde4067a-64ce-45de-8f2b-0dddc458f2a9 admin admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:33:44.199131 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dde4067a-64ce-45de-8f2b-0dddc458f2a9 admin admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:33:44.203644 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-013b3ac6-4628-4e23-8d2b-fbd76cd03ea5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76c5181fc7be45279ac00017bb4640b3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:33:44.204228 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-013b3ac6-4628-4e23-8d2b-fbd76cd03ea5 admin admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:33:44.204510 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-013b3ac6-4628-4e23-8d2b-fbd76cd03ea5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:33:44.204778 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-013b3ac6-4628-4e23-8d2b-fbd76cd03ea5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:33:44.205405 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-013b3ac6-4628-4e23-8d2b-fbd76cd03ea5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:33:44.207610 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-013b3ac6-4628-4e23-8d2b-fbd76cd03ea5 admin admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:33:44.217414 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 1335/2663] 10.222.0.18 () {58 vars in 1122 bytes} [Tue Sep 13 03:33:44 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 03:33:44.221419 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ee6af43-2074-42dc-a7a3-25307bdeea51 admin admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:33:44.231958 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ee6af43-2074-42dc-a7a3-25307bdeea51 admin admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:33:44.236528 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfbfd080-b7d8-4827-92d9-261434fa5bb7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76c5181fc7be45279ac00017bb4640b3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:33:44.237115 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfbfd080-b7d8-4827-92d9-261434fa5bb7 admin admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:33:44.237392 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfbfd080-b7d8-4827-92d9-261434fa5bb7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:33:44.237662 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfbfd080-b7d8-4827-92d9-261434fa5bb7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:33:44.238290 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfbfd080-b7d8-4827-92d9-261434fa5bb7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:33:44.239902 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfbfd080-b7d8-4827-92d9-261434fa5bb7 admin admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:33:44.251117 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 1329/2664] 10.222.0.18 () {58 vars in 1122 bytes} [Tue Sep 13 03:33:44 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 13 03:33:44.255788 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-013b3ac6-4628-4e23-8d2b-fbd76cd03ea5 admin admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:33:44.265710 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-013b3ac6-4628-4e23-8d2b-fbd76cd03ea5 admin admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:33:44.271130 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3535e18a-e3bb-4c86-b95d-5a94ae6e5970 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76c5181fc7be45279ac00017bb4640b3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:33:44.271771 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3535e18a-e3bb-4c86-b95d-5a94ae6e5970 admin admin] REQUEST_METHOD: `PUT` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:33:44.272034 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3535e18a-e3bb-4c86-b95d-5a94ae6e5970 admin admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:33:44.272300 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3535e18a-e3bb-4c86-b95d-5a94ae6e5970 admin admin] PATH_INFO: `/v3/projects/37a8009f1fa647b8be089db7d4c59fe4/users/400f426af7b64b1aadfb9efd1bffc948/roles/1eb7a5b5203945c29e6d22d653f47847` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:33:44.274513 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=85,keystone:DELETE=12,keystone:INSERT=3 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:33:44.289932 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3535e18a-e3bb-4c86-b95d-5a94ae6e5970 admin admin] RBAC: Authorizing `identity:create_grant(project_id=37a8009f1fa647b8be089db7d4c59fe4, user_id=400f426af7b64b1aadfb9efd1bffc948, role_id=1eb7a5b5203945c29e6d22d653f47847)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:33:44.292624 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3535e18a-e3bb-4c86-b95d-5a94ae6e5970 admin admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:33:44.304798 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 1336/2665] 10.222.0.18 () {60 vars in 1483 bytes} [Tue Sep 13 03:33:44 2022] PUT /identity/v3/projects/37a8009f1fa647b8be089db7d4c59fe4/users/400f426af7b64b1aadfb9efd1bffc948/roles/1eb7a5b5203945c29e6d22d653f47847 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 13 03:33:44.314856 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f6c1bcb-25ac-48cd-bd1c-e3852964cdb7 None None] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:33:44.315137 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f6c1bcb-25ac-48cd-bd1c-e3852964cdb7 None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:33:44.315954 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f6c1bcb-25ac-48cd-bd1c-e3852964cdb7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:33:44.343636 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.auth.core [None req-5f6c1bcb-25ac-48cd-bd1c-e3852964cdb7 None None] MFA Rules not processed for user `400f426af7b64b1aadfb9efd1bffc948`. Rule list: `[]` (Enabled: `True`). {{(pid=81400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 03:33:44.344841 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-5f6c1bcb-25ac-48cd-bd1c-e3852964cdb7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:33:44.407497 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 1330/2666] 10.222.0.18 () {58 vars in 958 bytes} [Tue Sep 13 03:33:44 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 03:33:44.412062 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3535e18a-e3bb-4c86-b95d-5a94ae6e5970 admin admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:33:44.471336 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3535e18a-e3bb-4c86-b95d-5a94ae6e5970 admin admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:33:44.475955 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc79d1ec-1c07-4145-98d7-6bc63764ab2c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76c5181fc7be45279ac00017bb4640b3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:33:44.476548 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc79d1ec-1c07-4145-98d7-6bc63764ab2c admin admin] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:33:44.476824 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc79d1ec-1c07-4145-98d7-6bc63764ab2c admin admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:33:44.477091 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc79d1ec-1c07-4145-98d7-6bc63764ab2c admin admin] PATH_INFO: `/v3/projects` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:33:44.477714 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc79d1ec-1c07-4145-98d7-6bc63764ab2c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:33:44.479631 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc79d1ec-1c07-4145-98d7-6bc63764ab2c admin admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:33:44.508121 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 1337/2667] 10.222.0.18 () {60 vars in 1153 bytes} [Tue Sep 13 03:33:44 2022] POST /identity/v3/projects => generated 381 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 03:33:44.511810 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f6c1bcb-25ac-48cd-bd1c-e3852964cdb7 None None] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:33:44.523202 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f6c1bcb-25ac-48cd-bd1c-e3852964cdb7 None None] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:33:44.544619 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:33:44.584918 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23414918-183a-4932-9079-d51907a40064 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76c5181fc7be45279ac00017bb4640b3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:33:44.585531 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23414918-183a-4932-9079-d51907a40064 admin admin] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:33:44.585809 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23414918-183a-4932-9079-d51907a40064 admin admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:33:44.586102 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23414918-183a-4932-9079-d51907a40064 admin admin] PATH_INFO: `/v3/users` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:33:44.587588 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23414918-183a-4932-9079-d51907a40064 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:33:44.589695 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23414918-183a-4932-9079-d51907a40064 admin admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:33:44.614647 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 1331/2668] 10.222.0.18 () {60 vars in 1144 bytes} [Tue Sep 13 03:33:44 2022] POST /identity/v3/users => generated 297 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 03:33:44.618407 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc79d1ec-1c07-4145-98d7-6bc63764ab2c admin admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:33:44.629583 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc79d1ec-1c07-4145-98d7-6bc63764ab2c admin admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:33:44.635014 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fde0137-103a-452e-8ebe-2cb81d09f6ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76c5181fc7be45279ac00017bb4640b3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:33:44.635624 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fde0137-103a-452e-8ebe-2cb81d09f6ec admin admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:33:44.635904 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fde0137-103a-452e-8ebe-2cb81d09f6ec admin admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:33:44.636175 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fde0137-103a-452e-8ebe-2cb81d09f6ec admin admin] PATH_INFO: `/v3/roles` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:33:44.636811 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fde0137-103a-452e-8ebe-2cb81d09f6ec admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:33:44.638424 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fde0137-103a-452e-8ebe-2cb81d09f6ec admin admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:33:44.650683 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 1338/2669] 10.222.0.18 () {58 vars in 1122 bytes} [Tue Sep 13 03:33:44 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 03:33:44.655053 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23414918-183a-4932-9079-d51907a40064 admin admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:33:44.665697 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23414918-183a-4932-9079-d51907a40064 admin admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:33:44.670365 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af29158a-04db-4cd9-95c5-902ea2294bef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76c5181fc7be45279ac00017bb4640b3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:33:44.671042 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af29158a-04db-4cd9-95c5-902ea2294bef admin admin] REQUEST_METHOD: `PUT` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:33:44.671323 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af29158a-04db-4cd9-95c5-902ea2294bef admin admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:33:44.671591 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af29158a-04db-4cd9-95c5-902ea2294bef admin admin] PATH_INFO: `/v3/projects/8ab195c79b6c4e6a951a3921bca2a456/users/2d684ed7b44f4236aacd46fdeda37f1e/roles/425d4b481c7846e0bc07e7e805d5b15f` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:33:44.686975 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af29158a-04db-4cd9-95c5-902ea2294bef admin admin] RBAC: Authorizing `identity:create_grant(project_id=8ab195c79b6c4e6a951a3921bca2a456, user_id=2d684ed7b44f4236aacd46fdeda37f1e, role_id=425d4b481c7846e0bc07e7e805d5b15f)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:33:44.690040 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af29158a-04db-4cd9-95c5-902ea2294bef admin admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:33:44.703213 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 1332/2670] 10.222.0.18 () {60 vars in 1483 bytes} [Tue Sep 13 03:33:44 2022] PUT /identity/v3/projects/8ab195c79b6c4e6a951a3921bca2a456/users/2d684ed7b44f4236aacd46fdeda37f1e/roles/425d4b481c7846e0bc07e7e805d5b15f => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 03:33:44.707209 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fde0137-103a-452e-8ebe-2cb81d09f6ec admin admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:33:44.717060 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fde0137-103a-452e-8ebe-2cb81d09f6ec admin admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:33:44.776656 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-286375bd-bdf4-4354-a016-8670e3c2b0e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76c5181fc7be45279ac00017bb4640b3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:33:44.777358 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-286375bd-bdf4-4354-a016-8670e3c2b0e7 admin admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:33:44.777645 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-286375bd-bdf4-4354-a016-8670e3c2b0e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:33:44.777919 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-286375bd-bdf4-4354-a016-8670e3c2b0e7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:33:44.778560 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-286375bd-bdf4-4354-a016-8670e3c2b0e7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:33:44.780268 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-286375bd-bdf4-4354-a016-8670e3c2b0e7 admin admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:33:44.790590 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 1339/2671] 10.222.0.18 () {58 vars in 1122 bytes} [Tue Sep 13 03:33:44 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 13 03:33:44.795256 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af29158a-04db-4cd9-95c5-902ea2294bef admin admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:33:44.807461 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af29158a-04db-4cd9-95c5-902ea2294bef admin admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:33:44.812974 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e036e079-429f-4c10-a838-0275bc53e3cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76c5181fc7be45279ac00017bb4640b3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:33:44.813862 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e036e079-429f-4c10-a838-0275bc53e3cb admin admin] REQUEST_METHOD: `PUT` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:33:44.814262 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e036e079-429f-4c10-a838-0275bc53e3cb admin admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:33:44.814723 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e036e079-429f-4c10-a838-0275bc53e3cb admin admin] PATH_INFO: `/v3/projects/8ab195c79b6c4e6a951a3921bca2a456/users/2d684ed7b44f4236aacd46fdeda37f1e/roles/425d4b481c7846e0bc07e7e805d5b15f` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:33:44.817458 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e036e079-429f-4c10-a838-0275bc53e3cb admin admin] RBAC: Authorizing `identity:create_grant(project_id=8ab195c79b6c4e6a951a3921bca2a456, user_id=2d684ed7b44f4236aacd46fdeda37f1e, role_id=425d4b481c7846e0bc07e7e805d5b15f)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:33:44.820432 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e036e079-429f-4c10-a838-0275bc53e3cb admin admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:33:44.831872 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 1333/2672] 10.222.0.18 () {60 vars in 1483 bytes} [Tue Sep 13 03:33:44 2022] PUT /identity/v3/projects/8ab195c79b6c4e6a951a3921bca2a456/users/2d684ed7b44f4236aacd46fdeda37f1e/roles/425d4b481c7846e0bc07e7e805d5b15f => generated 0 bytes in 38 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 03:33:44.842913 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d793189f-2810-48ee-bc0a-f7bed7759d94 None None] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:33:44.843206 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d793189f-2810-48ee-bc0a-f7bed7759d94 None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:33:44.843482 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d793189f-2810-48ee-bc0a-f7bed7759d94 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:33:44.872296 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.auth.core [None req-d793189f-2810-48ee-bc0a-f7bed7759d94 None None] MFA Rules not processed for user `2d684ed7b44f4236aacd46fdeda37f1e`. Rule list: `[]` (Enabled: `True`). {{(pid=81399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 03:33:44.873498 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-d793189f-2810-48ee-bc0a-f7bed7759d94 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:33:44.914236 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:33:44.942545 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 1340/2673] 10.222.0.18 () {58 vars in 958 bytes} [Tue Sep 13 03:33:44 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 03:33:44.953913 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e036e079-429f-4c10-a838-0275bc53e3cb admin admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:33:44.955648 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-e036e079-429f-4c10-a838-0275bc53e3cb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:33:45.011620 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e036e079-429f-4c10-a838-0275bc53e3cb admin admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:33:45.015254 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ac9e365-2214-4cdb-b108-d59c68874f7f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '924ee3f608d34a18b0ddc4695f102a56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b9d94f6a68f7470d9494f112175620dc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:33:45.015838 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac9e365-2214-4cdb-b108-d59c68874f7f service cinder] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:33:45.016108 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac9e365-2214-4cdb-b108-d59c68874f7f service cinder] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:33:45.016374 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac9e365-2214-4cdb-b108-d59c68874f7f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:33:45.025472 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ac9e365-2214-4cdb-b108-d59c68874f7f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:33:45.026912 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ac9e365-2214-4cdb-b108-d59c68874f7f service cinder] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:33:45.036882 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 1334/2674] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Sep 13 03:33:44 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 03:33:45.113415 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d793189f-2810-48ee-bc0a-f7bed7759d94 None None] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:33:45.123536 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d793189f-2810-48ee-bc0a-f7bed7759d94 None None] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:33:45.127238 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4cf5678-98e1-4480-ade9-bb20a1f1c317 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '924ee3f608d34a18b0ddc4695f102a56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b9d94f6a68f7470d9494f112175620dc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:33:45.127839 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4cf5678-98e1-4480-ade9-bb20a1f1c317 service cinder] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:33:45.128149 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4cf5678-98e1-4480-ade9-bb20a1f1c317 service cinder] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:33:45.128447 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4cf5678-98e1-4480-ade9-bb20a1f1c317 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:33:45.184914 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4cf5678-98e1-4480-ade9-bb20a1f1c317 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:33:45.186347 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4cf5678-98e1-4480-ade9-bb20a1f1c317 service cinder] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:33:45.195106 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 1341/2675] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Sep 13 03:33:45 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 03:33:55.031974 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=59,keystone:INSERT=5 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:33:55.191593 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=38 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:33:56.724482 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ac9e365-2214-4cdb-b108-d59c68874f7f service cinder] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:33:56.725935 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-9ac9e365-2214-4cdb-b108-d59c68874f7f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:33:56.793217 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ac9e365-2214-4cdb-b108-d59c68874f7f service cinder] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:33:56.798056 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb567594-9873-4855-8045-5cc5852e73e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76c5181fc7be45279ac00017bb4640b3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:33:56.798736 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb567594-9873-4855-8045-5cc5852e73e0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:33:56.799012 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb567594-9873-4855-8045-5cc5852e73e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:33:56.799283 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb567594-9873-4855-8045-5cc5852e73e0 admin admin] PATH_INFO: `/v3/users/fcb952fb95524cb1af9528f13e700df1` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:33:56.800465 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb567594-9873-4855-8045-5cc5852e73e0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=fcb952fb95524cb1af9528f13e700df1)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:33:56.802545 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb567594-9873-4855-8045-5cc5852e73e0 admin admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:33:56.849813 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-fb567594-9873-4855-8045-5cc5852e73e0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fcb952fb95524cb1af9528f13e700df1'} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:33:56.857670 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-fb567594-9873-4855-8045-5cc5852e73e0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fcb952fb95524cb1af9528f13e700df1'} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:33:56.858433 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-fb567594-9873-4855-8045-5cc5852e73e0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fcb952fb95524cb1af9528f13e700df1'} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:33:56.872359 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-fb567594-9873-4855-8045-5cc5852e73e0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fcb952fb95524cb1af9528f13e700df1'} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:33:56.880078 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-fb567594-9873-4855-8045-5cc5852e73e0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fcb952fb95524cb1af9528f13e700df1'} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:33:56.886086 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 1335/2676] 10.222.0.18 () {58 vars in 1224 bytes} [Tue Sep 13 03:33:56 2022] DELETE /identity/v3/users/fcb952fb95524cb1af9528f13e700df1 => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 03:33:56.890714 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2d603ef-c3d6-4b71-83bf-350d948157cd None None] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:33:56.891179 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2d603ef-c3d6-4b71-83bf-350d948157cd None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:33:56.891307 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2d603ef-c3d6-4b71-83bf-350d948157cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:33:56.912350 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.auth.core [None req-e2d603ef-c3d6-4b71-83bf-350d948157cd None None] MFA Rules not processed for user `49f280ab086b4610855c4ffa378517d9`. Rule list: `[]` (Enabled: `True`). {{(pid=81399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 03:33:56.913541 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-e2d603ef-c3d6-4b71-83bf-350d948157cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:33:56.977144 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 1342/2677] 10.222.0.18 () {58 vars in 958 bytes} [Tue Sep 13 03:33:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 13 03:33:56.993830 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb567594-9873-4855-8045-5cc5852e73e0 admin admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:33:56.993830 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-fb567594-9873-4855-8045-5cc5852e73e0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:33:57.047131 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb567594-9873-4855-8045-5cc5852e73e0 admin admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:33:57.050845 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-085ebfae-7f0f-47ec-961c-b3c8e63b902c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdffbc851086441ebd01022d1e4c2609', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b9d94f6a68f7470d9494f112175620dc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:33:57.051439 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-085ebfae-7f0f-47ec-961c-b3c8e63b902c service neutron] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:33:57.051712 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-085ebfae-7f0f-47ec-961c-b3c8e63b902c service neutron] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:33:57.051976 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-085ebfae-7f0f-47ec-961c-b3c8e63b902c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:33:57.060934 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-085ebfae-7f0f-47ec-961c-b3c8e63b902c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:33:57.062365 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-085ebfae-7f0f-47ec-961c-b3c8e63b902c service neutron] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:33:57.072791 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 1336/2678] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Sep 13 03:33:56 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 03:33:57.340624 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2d603ef-c3d6-4b71-83bf-350d948157cd None None] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:33:57.342046 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-e2d603ef-c3d6-4b71-83bf-350d948157cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:33:57.354242 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2d603ef-c3d6-4b71-83bf-350d948157cd None None] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:33:57.359029 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe736d48-6181-4075-befc-9384f2fe5f14 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76c5181fc7be45279ac00017bb4640b3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:33:57.359680 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe736d48-6181-4075-befc-9384f2fe5f14 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:33:57.359953 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe736d48-6181-4075-befc-9384f2fe5f14 admin admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:33:57.360224 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe736d48-6181-4075-befc-9384f2fe5f14 admin admin] PATH_INFO: `/v3/projects/a0f237a0241c41fda216cee0dbd07ed0` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:33:57.362153 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe736d48-6181-4075-befc-9384f2fe5f14 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a0f237a0241c41fda216cee0dbd07ed0)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:33:57.364831 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe736d48-6181-4075-befc-9384f2fe5f14 admin admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:33:57.411699 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-fe736d48-6181-4075-befc-9384f2fe5f14 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a0f237a0241c41fda216cee0dbd07ed0', 'request_id': 'req-fe736d48-6181-4075-befc-9384f2fe5f14'} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:33:57.417046 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-fe736d48-6181-4075-befc-9384f2fe5f14 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a0f237a0241c41fda216cee0dbd07ed0', 'request_id': 'req-fe736d48-6181-4075-befc-9384f2fe5f14'} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:33:57.422948 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-fe736d48-6181-4075-befc-9384f2fe5f14 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a0f237a0241c41fda216cee0dbd07ed0', 'request_id': 'req-fe736d48-6181-4075-befc-9384f2fe5f14'} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:33:57.435292 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-fe736d48-6181-4075-befc-9384f2fe5f14 admin admin] The token cache is being invalidate because project a0f237a0241c41fda216cee0dbd07ed0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81399) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 03:33:57.435578 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-fe736d48-6181-4075-befc-9384f2fe5f14 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:33:57.436208 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-fe736d48-6181-4075-befc-9384f2fe5f14 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:33:57.437656 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 1343/2679] 10.222.0.18 () {58 vars in 1233 bytes} [Tue Sep 13 03:33:57 2022] DELETE /identity/v3/projects/a0f237a0241c41fda216cee0dbd07ed0 => generated 0 bytes in 98 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 03:33:57.442253 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-085ebfae-7f0f-47ec-961c-b3c8e63b902c service neutron] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:33:57.451795 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-085ebfae-7f0f-47ec-961c-b3c8e63b902c service neutron] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:33:57.469357 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:33:57.509049 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13ce382c-eece-4a21-9f14-54efb63b27de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76c5181fc7be45279ac00017bb4640b3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:33:57.509665 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ce382c-eece-4a21-9f14-54efb63b27de admin admin] REQUEST_METHOD: `DELETE` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:33:57.509941 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ce382c-eece-4a21-9f14-54efb63b27de admin admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:33:57.510212 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ce382c-eece-4a21-9f14-54efb63b27de admin admin] PATH_INFO: `/v3/users/cc15e637acd64efb864f2e0d3aa6e307` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:33:57.511349 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13ce382c-eece-4a21-9f14-54efb63b27de admin admin] RBAC: Authorizing `identity:delete_user(user_id=cc15e637acd64efb864f2e0d3aa6e307)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:33:57.513254 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13ce382c-eece-4a21-9f14-54efb63b27de admin admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:33:57.558405 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-13ce382c-eece-4a21-9f14-54efb63b27de admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cc15e637acd64efb864f2e0d3aa6e307'} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:33:57.565564 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-13ce382c-eece-4a21-9f14-54efb63b27de admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cc15e637acd64efb864f2e0d3aa6e307'} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:33:57.566150 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-13ce382c-eece-4a21-9f14-54efb63b27de admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cc15e637acd64efb864f2e0d3aa6e307'} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:33:57.578478 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-13ce382c-eece-4a21-9f14-54efb63b27de admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cc15e637acd64efb864f2e0d3aa6e307'} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:33:57.585361 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-13ce382c-eece-4a21-9f14-54efb63b27de admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cc15e637acd64efb864f2e0d3aa6e307'} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:33:57.591084 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 1337/2680] 10.222.0.18 () {58 vars in 1224 bytes} [Tue Sep 13 03:33:57 2022] DELETE /identity/v3/users/cc15e637acd64efb864f2e0d3aa6e307 => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 03:33:57.858319 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe736d48-6181-4075-befc-9384f2fe5f14 admin admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:33:57.860229 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-fe736d48-6181-4075-befc-9384f2fe5f14 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:33:57.877184 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe736d48-6181-4075-befc-9384f2fe5f14 admin admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:33:57.934488 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aecc92f-7f18-4e43-bf16-4e9fe8ec3dc6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76c5181fc7be45279ac00017bb4640b3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:33:57.935161 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aecc92f-7f18-4e43-bf16-4e9fe8ec3dc6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:33:57.935405 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aecc92f-7f18-4e43-bf16-4e9fe8ec3dc6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:33:57.935670 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aecc92f-7f18-4e43-bf16-4e9fe8ec3dc6 admin admin] PATH_INFO: `/v3/projects/152f125031724eafaa495afbec83d9cb` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:33:57.936756 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aecc92f-7f18-4e43-bf16-4e9fe8ec3dc6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=152f125031724eafaa495afbec83d9cb)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:33:57.938664 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aecc92f-7f18-4e43-bf16-4e9fe8ec3dc6 admin admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:33:57.966808 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:33:57.984464 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-4aecc92f-7f18-4e43-bf16-4e9fe8ec3dc6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '152f125031724eafaa495afbec83d9cb', 'request_id': 'req-4aecc92f-7f18-4e43-bf16-4e9fe8ec3dc6'} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:33:57.990552 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-4aecc92f-7f18-4e43-bf16-4e9fe8ec3dc6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '152f125031724eafaa495afbec83d9cb', 'request_id': 'req-4aecc92f-7f18-4e43-bf16-4e9fe8ec3dc6'} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:33:57.996621 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-4aecc92f-7f18-4e43-bf16-4e9fe8ec3dc6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '152f125031724eafaa495afbec83d9cb', 'request_id': 'req-4aecc92f-7f18-4e43-bf16-4e9fe8ec3dc6'} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:33:58.008610 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-4aecc92f-7f18-4e43-bf16-4e9fe8ec3dc6 admin admin] The token cache is being invalidate because project 152f125031724eafaa495afbec83d9cb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81399) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 03:33:58.008897 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-4aecc92f-7f18-4e43-bf16-4e9fe8ec3dc6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:33:58.009564 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-4aecc92f-7f18-4e43-bf16-4e9fe8ec3dc6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:33:58.011041 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 1344/2681] 10.222.0.18 () {58 vars in 1233 bytes} [Tue Sep 13 03:33:57 2022] DELETE /identity/v3/projects/152f125031724eafaa495afbec83d9cb => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 03:33:58.015980 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4184a8fa-2e1e-4c4a-b55b-4ff18f37f4f6 None None] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:33:58.016419 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4184a8fa-2e1e-4c4a-b55b-4ff18f37f4f6 None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:33:58.016524 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4184a8fa-2e1e-4c4a-b55b-4ff18f37f4f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:33:58.040305 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.auth.core [None req-4184a8fa-2e1e-4c4a-b55b-4ff18f37f4f6 None None] MFA Rules not processed for user `49f280ab086b4610855c4ffa378517d9`. Rule list: `[]` (Enabled: `True`). {{(pid=81400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 03:33:58.041529 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-4184a8fa-2e1e-4c4a-b55b-4ff18f37f4f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:33:58.110642 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 1338/2682] 10.222.0.18 () {58 vars in 958 bytes} [Tue Sep 13 03:33:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 03:33:58.117669 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d162b80c-8d2a-4c54-9772-8494eab44cfd None None] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:33:58.117954 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d162b80c-8d2a-4c54-9772-8494eab44cfd None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:33:58.118225 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d162b80c-8d2a-4c54-9772-8494eab44cfd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:33:58.138703 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.auth.core [None req-d162b80c-8d2a-4c54-9772-8494eab44cfd None None] MFA Rules not processed for user `49f280ab086b4610855c4ffa378517d9`. Rule list: `[]` (Enabled: `True`). {{(pid=81399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 03:33:58.139950 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-d162b80c-8d2a-4c54-9772-8494eab44cfd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:33:58.148404 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 1345/2683] 10.222.0.18 () {58 vars in 958 bytes} [Tue Sep 13 03:33:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 13 03:33:58.152978 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4184a8fa-2e1e-4c4a-b55b-4ff18f37f4f6 None None] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:33:58.164500 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4184a8fa-2e1e-4c4a-b55b-4ff18f37f4f6 None None] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:33:58.169089 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f2b54e0-32b4-4d7e-9328-29e62b9a2c74 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76c5181fc7be45279ac00017bb4640b3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:33:58.169689 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f2b54e0-32b4-4d7e-9328-29e62b9a2c74 admin admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:33:58.169969 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f2b54e0-32b4-4d7e-9328-29e62b9a2c74 admin admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:33:58.170237 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f2b54e0-32b4-4d7e-9328-29e62b9a2c74 admin admin] PATH_INFO: `/v3/domains` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:33:58.170918 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f2b54e0-32b4-4d7e-9328-29e62b9a2c74 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 03:33:58.171218 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f2b54e0-32b4-4d7e-9328-29e62b9a2c74 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:33:58.172864 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f2b54e0-32b4-4d7e-9328-29e62b9a2c74 admin admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:33:58.177897 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d162b80c-8d2a-4c54-9772-8494eab44cfd None None] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:33:58.179281 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-d162b80c-8d2a-4c54-9772-8494eab44cfd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:33:58.186651 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 1339/2684] 10.222.0.18 () {58 vars in 1153 bytes} [Tue Sep 13 03:33:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 03:33:58.189370 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f2b54e0-32b4-4d7e-9328-29e62b9a2c74 admin admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:33:58.191600 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d162b80c-8d2a-4c54-9772-8494eab44cfd None None] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:33:58.196993 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-731c6c16-98ca-45cf-9b4d-c91a5813be4f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76c5181fc7be45279ac00017bb4640b3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:33:58.197613 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-731c6c16-98ca-45cf-9b4d-c91a5813be4f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:33:58.197885 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-731c6c16-98ca-45cf-9b4d-c91a5813be4f admin admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:33:58.198154 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-731c6c16-98ca-45cf-9b4d-c91a5813be4f admin admin] PATH_INFO: `/v3/users/400f426af7b64b1aadfb9efd1bffc948` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:33:58.199298 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-731c6c16-98ca-45cf-9b4d-c91a5813be4f admin admin] RBAC: Authorizing `identity:delete_user(user_id=400f426af7b64b1aadfb9efd1bffc948)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:33:58.199415 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f2b54e0-32b4-4d7e-9328-29e62b9a2c74 admin admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:33:58.201244 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-731c6c16-98ca-45cf-9b4d-c91a5813be4f admin admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:33:58.203888 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90972297-006e-498e-80dd-41372d21c940 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76c5181fc7be45279ac00017bb4640b3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:33:58.204463 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90972297-006e-498e-80dd-41372d21c940 admin admin] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:33:58.204738 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90972297-006e-498e-80dd-41372d21c940 admin admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:33:58.205006 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90972297-006e-498e-80dd-41372d21c940 admin admin] PATH_INFO: `/v3/projects` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:33:58.205618 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90972297-006e-498e-80dd-41372d21c940 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:33:58.208236 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90972297-006e-498e-80dd-41372d21c940 admin admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:33:58.236305 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 1340/2685] 10.222.0.18 () {60 vars in 1153 bytes} [Tue Sep 13 03:33:58 2022] POST /identity/v3/projects => generated 367 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 13 03:33:58.240293 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90972297-006e-498e-80dd-41372d21c940 admin admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:33:58.249024 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:33:58.251287 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-731c6c16-98ca-45cf-9b4d-c91a5813be4f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '400f426af7b64b1aadfb9efd1bffc948'} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:33:58.260441 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-731c6c16-98ca-45cf-9b4d-c91a5813be4f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '400f426af7b64b1aadfb9efd1bffc948'} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:33:58.273438 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-731c6c16-98ca-45cf-9b4d-c91a5813be4f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '400f426af7b64b1aadfb9efd1bffc948'} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:33:58.274147 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-731c6c16-98ca-45cf-9b4d-c91a5813be4f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '400f426af7b64b1aadfb9efd1bffc948'} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:33:58.281776 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-731c6c16-98ca-45cf-9b4d-c91a5813be4f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '400f426af7b64b1aadfb9efd1bffc948'} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:33:58.288658 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 1346/2686] 10.222.0.18 () {58 vars in 1224 bytes} [Tue Sep 13 03:33:58 2022] DELETE /identity/v3/users/400f426af7b64b1aadfb9efd1bffc948 => generated 0 bytes in 111 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 03:33:58.293013 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61306b02-997f-49cf-acb7-3ddc0511b7fc None None] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:33:58.293359 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61306b02-997f-49cf-acb7-3ddc0511b7fc None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:33:58.293709 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61306b02-997f-49cf-acb7-3ddc0511b7fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:33:58.308214 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90972297-006e-498e-80dd-41372d21c940 admin admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:33:58.312634 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.auth.core [None req-61306b02-997f-49cf-acb7-3ddc0511b7fc None None] MFA Rules not processed for user `49f280ab086b4610855c4ffa378517d9`. Rule list: `[]` (Enabled: `True`). {{(pid=81399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 03:33:58.313812 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-61306b02-997f-49cf-acb7-3ddc0511b7fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:33:58.365596 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1044ce4-5358-4c28-9715-4e964548110c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76c5181fc7be45279ac00017bb4640b3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:33:58.366114 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1044ce4-5358-4c28-9715-4e964548110c admin admin] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:33:58.366387 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1044ce4-5358-4c28-9715-4e964548110c admin admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:33:58.366685 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1044ce4-5358-4c28-9715-4e964548110c admin admin] PATH_INFO: `/v3/users` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:33:58.367429 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1044ce4-5358-4c28-9715-4e964548110c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:33:58.369352 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1044ce4-5358-4c28-9715-4e964548110c admin admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:33:58.371121 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 1347/2687] 10.222.0.18 () {58 vars in 958 bytes} [Tue Sep 13 03:33:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 13 03:33:58.381495 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61306b02-997f-49cf-acb7-3ddc0511b7fc None None] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:33:58.383089 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-61306b02-997f-49cf-acb7-3ddc0511b7fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:33:58.393303 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 1341/2688] 10.222.0.18 () {60 vars in 1144 bytes} [Tue Sep 13 03:33:58 2022] POST /identity/v3/users => generated 284 bytes in 153 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 13 03:33:58.397753 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1044ce4-5358-4c28-9715-4e964548110c admin admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:33:58.399829 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-c1044ce4-5358-4c28-9715-4e964548110c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:33:58.407371 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:DELETE=16,keystone:SELECT=82,keystone:INSERT=2 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:33:58.413309 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1044ce4-5358-4c28-9715-4e964548110c admin admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:33:58.418730 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dce004f2-6411-4263-b89b-eac98cb161e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76c5181fc7be45279ac00017bb4640b3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:33:58.419331 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dce004f2-6411-4263-b89b-eac98cb161e0 admin admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:33:58.419608 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dce004f2-6411-4263-b89b-eac98cb161e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:33:58.419911 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dce004f2-6411-4263-b89b-eac98cb161e0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:33:58.420503 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dce004f2-6411-4263-b89b-eac98cb161e0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:33:58.422084 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dce004f2-6411-4263-b89b-eac98cb161e0 admin admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:33:58.432082 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 1342/2689] 10.222.0.18 () {58 vars in 1122 bytes} [Tue Sep 13 03:33:58 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 13 03:33:58.436067 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dce004f2-6411-4263-b89b-eac98cb161e0 admin admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:33:58.445442 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dce004f2-6411-4263-b89b-eac98cb161e0 admin admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:33:58.448415 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61306b02-997f-49cf-acb7-3ddc0511b7fc None None] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:33:58.449968 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30b5140d-4ea1-4243-b7ed-7b2162c73b96 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76c5181fc7be45279ac00017bb4640b3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:33:58.450533 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30b5140d-4ea1-4243-b7ed-7b2162c73b96 admin admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:33:58.450834 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30b5140d-4ea1-4243-b7ed-7b2162c73b96 admin admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:33:58.451099 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30b5140d-4ea1-4243-b7ed-7b2162c73b96 admin admin] PATH_INFO: `/v3/roles` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:33:58.451707 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30b5140d-4ea1-4243-b7ed-7b2162c73b96 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:33:58.452036 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a41ec660-3e4a-4401-a0cf-1a99ae0140e4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdffbc851086441ebd01022d1e4c2609', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b9d94f6a68f7470d9494f112175620dc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:33:58.452635 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a41ec660-3e4a-4401-a0cf-1a99ae0140e4 service neutron] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:33:58.452912 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a41ec660-3e4a-4401-a0cf-1a99ae0140e4 service neutron] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:33:58.453183 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a41ec660-3e4a-4401-a0cf-1a99ae0140e4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:33:58.453282 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30b5140d-4ea1-4243-b7ed-7b2162c73b96 admin admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:33:58.462547 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a41ec660-3e4a-4401-a0cf-1a99ae0140e4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:33:58.464228 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a41ec660-3e4a-4401-a0cf-1a99ae0140e4 service neutron] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:33:58.464348 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 1343/2690] 10.222.0.18 () {58 vars in 1122 bytes} [Tue Sep 13 03:33:58 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 13 03:33:58.467924 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30b5140d-4ea1-4243-b7ed-7b2162c73b96 admin admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:33:58.474049 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 1348/2691] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Sep 13 03:33:58 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 13 03:33:58.477728 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30b5140d-4ea1-4243-b7ed-7b2162c73b96 admin admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:33:58.482788 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfd01f89-0c97-4760-a3f0-20771c7d5eb8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76c5181fc7be45279ac00017bb4640b3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:33:58.483079 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd01f89-0c97-4760-a3f0-20771c7d5eb8 admin admin] REQUEST_METHOD: `PUT` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:33:58.483352 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd01f89-0c97-4760-a3f0-20771c7d5eb8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:33:58.483615 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd01f89-0c97-4760-a3f0-20771c7d5eb8 admin admin] PATH_INFO: `/v3/projects/0b6c84750c02479c8d8269c39a9fbe4c/users/071f2a03df3944a7aa6e4953ff1e23ac/roles/1eb7a5b5203945c29e6d22d653f47847` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:33:58.497946 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfd01f89-0c97-4760-a3f0-20771c7d5eb8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0b6c84750c02479c8d8269c39a9fbe4c, user_id=071f2a03df3944a7aa6e4953ff1e23ac, role_id=1eb7a5b5203945c29e6d22d653f47847)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:33:58.500612 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfd01f89-0c97-4760-a3f0-20771c7d5eb8 admin admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:33:58.513710 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 1344/2692] 10.222.0.18 () {60 vars in 1483 bytes} [Tue Sep 13 03:33:58 2022] PUT /identity/v3/projects/0b6c84750c02479c8d8269c39a9fbe4c/users/071f2a03df3944a7aa6e4953ff1e23ac/roles/1eb7a5b5203945c29e6d22d653f47847 => generated 0 bytes in 46 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 13 03:33:58.524581 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc94a632-a7d3-4272-bae0-c513e695bfb1 None None] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:33:58.524890 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc94a632-a7d3-4272-bae0-c513e695bfb1 None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:33:58.525165 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc94a632-a7d3-4272-bae0-c513e695bfb1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:33:58.554391 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.auth.core [None req-fc94a632-a7d3-4272-bae0-c513e695bfb1 None None] MFA Rules not processed for user `071f2a03df3944a7aa6e4953ff1e23ac`. Rule list: `[]` (Enabled: `True`). {{(pid=81399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 03:33:58.555691 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-fc94a632-a7d3-4272-bae0-c513e695bfb1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:33:58.627035 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 1349/2693] 10.222.0.18 () {58 vars in 958 bytes} [Tue Sep 13 03:33:58 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 03:33:58.629755 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfd01f89-0c97-4760-a3f0-20771c7d5eb8 admin admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:33:58.688844 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:33:58.693344 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfd01f89-0c97-4760-a3f0-20771c7d5eb8 admin admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:33:58.698680 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7429ee3e-45da-4c86-953d-f93d1d475811 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76c5181fc7be45279ac00017bb4640b3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:33:58.699356 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7429ee3e-45da-4c86-953d-f93d1d475811 admin admin] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:33:58.699691 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7429ee3e-45da-4c86-953d-f93d1d475811 admin admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:33:58.700028 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7429ee3e-45da-4c86-953d-f93d1d475811 admin admin] PATH_INFO: `/v3/projects` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:33:58.700723 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7429ee3e-45da-4c86-953d-f93d1d475811 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:33:58.702709 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7429ee3e-45da-4c86-953d-f93d1d475811 admin admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:33:58.730539 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 1345/2694] 10.222.0.18 () {60 vars in 1153 bytes} [Tue Sep 13 03:33:58 2022] POST /identity/v3/projects => generated 365 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 03:33:58.734877 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc94a632-a7d3-4272-bae0-c513e695bfb1 None None] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:33:58.743288 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7429ee3e-45da-4c86-953d-f93d1d475811 admin admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:33:58.745112 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-7429ee3e-45da-4c86-953d-f93d1d475811 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:33:58.748008 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc94a632-a7d3-4272-bae0-c513e695bfb1 None None] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:33:58.815918 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c32d6c86-3594-4ef9-b75a-41cc30cbc30b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76c5181fc7be45279ac00017bb4640b3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:33:58.817070 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c32d6c86-3594-4ef9-b75a-41cc30cbc30b admin admin] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:33:58.817340 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c32d6c86-3594-4ef9-b75a-41cc30cbc30b admin admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:33:58.817604 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c32d6c86-3594-4ef9-b75a-41cc30cbc30b admin admin] PATH_INFO: `/v3/users` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:33:58.818230 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c32d6c86-3594-4ef9-b75a-41cc30cbc30b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:33:58.819240 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7429ee3e-45da-4c86-953d-f93d1d475811 admin admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:33:58.820174 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c32d6c86-3594-4ef9-b75a-41cc30cbc30b admin admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:33:58.823736 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a14751dc-c601-4698-b291-c88fe59410f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76c5181fc7be45279ac00017bb4640b3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:33:58.824360 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a14751dc-c601-4698-b291-c88fe59410f6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:33:58.824634 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a14751dc-c601-4698-b291-c88fe59410f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:33:58.824901 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a14751dc-c601-4698-b291-c88fe59410f6 admin admin] PATH_INFO: `/v3/projects/37a8009f1fa647b8be089db7d4c59fe4` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:33:58.826066 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a14751dc-c601-4698-b291-c88fe59410f6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=37a8009f1fa647b8be089db7d4c59fe4)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:33:58.828031 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a14751dc-c601-4698-b291-c88fe59410f6 admin admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:33:58.842228 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 1350/2695] 10.222.0.18 () {60 vars in 1144 bytes} [Tue Sep 13 03:33:58 2022] POST /identity/v3/users => generated 289 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 03:33:58.845983 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c32d6c86-3594-4ef9-b75a-41cc30cbc30b admin admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:33:58.855298 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c32d6c86-3594-4ef9-b75a-41cc30cbc30b admin admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:33:58.860250 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e42aa47d-e43e-4d50-b0af-20933b86263b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76c5181fc7be45279ac00017bb4640b3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:33:58.860789 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e42aa47d-e43e-4d50-b0af-20933b86263b admin admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:33:58.861061 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e42aa47d-e43e-4d50-b0af-20933b86263b admin admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:33:58.861449 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e42aa47d-e43e-4d50-b0af-20933b86263b admin admin] PATH_INFO: `/v3/roles` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:33:58.862182 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e42aa47d-e43e-4d50-b0af-20933b86263b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:33:58.864623 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e42aa47d-e43e-4d50-b0af-20933b86263b admin admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:33:58.879434 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 1351/2696] 10.222.0.18 () {58 vars in 1122 bytes} [Tue Sep 13 03:33:58 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 13 03:33:58.880321 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e42aa47d-e43e-4d50-b0af-20933b86263b admin admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:33:58.881076 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-a14751dc-c601-4698-b291-c88fe59410f6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '37a8009f1fa647b8be089db7d4c59fe4', 'request_id': 'req-a14751dc-c601-4698-b291-c88fe59410f6'} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:33:58.886922 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-a14751dc-c601-4698-b291-c88fe59410f6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '37a8009f1fa647b8be089db7d4c59fe4', 'request_id': 'req-a14751dc-c601-4698-b291-c88fe59410f6'} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:33:58.889589 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e42aa47d-e43e-4d50-b0af-20933b86263b admin admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:33:58.893198 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-a14751dc-c601-4698-b291-c88fe59410f6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '37a8009f1fa647b8be089db7d4c59fe4', 'request_id': 'req-a14751dc-c601-4698-b291-c88fe59410f6'} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:33:58.906007 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-a14751dc-c601-4698-b291-c88fe59410f6 admin admin] The token cache is being invalidate because project 37a8009f1fa647b8be089db7d4c59fe4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81400) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 03:33:58.906286 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-a14751dc-c601-4698-b291-c88fe59410f6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:33:58.906953 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-a14751dc-c601-4698-b291-c88fe59410f6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:33:58.908497 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 1346/2697] 10.222.0.18 () {58 vars in 1233 bytes} [Tue Sep 13 03:33:58 2022] DELETE /identity/v3/projects/37a8009f1fa647b8be089db7d4c59fe4 => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 03:33:58.912513 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a14751dc-c601-4698-b291-c88fe59410f6 admin admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:33:58.913818 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-a14751dc-c601-4698-b291-c88fe59410f6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:33:58.922318 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:33:58.952779 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94c6b845-e0d5-4f6f-bf0e-2b7b5aeb51bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76c5181fc7be45279ac00017bb4640b3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:33:58.953440 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94c6b845-e0d5-4f6f-bf0e-2b7b5aeb51bc admin admin] REQUEST_METHOD: `PUT` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:33:58.953711 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94c6b845-e0d5-4f6f-bf0e-2b7b5aeb51bc admin admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:33:58.953981 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94c6b845-e0d5-4f6f-bf0e-2b7b5aeb51bc admin admin] PATH_INFO: `/v3/projects/2e1fe27fe08e4bc39b45c7a9d94510fe/users/e514837a23094f798c4730563b87d6af/roles/425d4b481c7846e0bc07e7e805d5b15f` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:33:58.968514 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94c6b845-e0d5-4f6f-bf0e-2b7b5aeb51bc admin admin] RBAC: Authorizing `identity:create_grant(project_id=2e1fe27fe08e4bc39b45c7a9d94510fe, user_id=e514837a23094f798c4730563b87d6af, role_id=425d4b481c7846e0bc07e7e805d5b15f)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:33:58.971386 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94c6b845-e0d5-4f6f-bf0e-2b7b5aeb51bc admin admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:33:58.979583 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a14751dc-c601-4698-b291-c88fe59410f6 admin admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:33:58.984866 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 1352/2698] 10.222.0.18 () {60 vars in 1483 bytes} [Tue Sep 13 03:33:58 2022] PUT /identity/v3/projects/2e1fe27fe08e4bc39b45c7a9d94510fe/users/e514837a23094f798c4730563b87d6af/roles/425d4b481c7846e0bc07e7e805d5b15f => generated 0 bytes in 106 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 13 03:33:58.989173 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94c6b845-e0d5-4f6f-bf0e-2b7b5aeb51bc admin admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:33:58.990852 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-94c6b845-e0d5-4f6f-bf0e-2b7b5aeb51bc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:33:58.998791 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:33:59.044216 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb53dd42-043f-4135-8ed7-27a0d1c9416d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76c5181fc7be45279ac00017bb4640b3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:33:59.044851 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb53dd42-043f-4135-8ed7-27a0d1c9416d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:33:59.045118 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb53dd42-043f-4135-8ed7-27a0d1c9416d admin admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:33:59.045392 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb53dd42-043f-4135-8ed7-27a0d1c9416d admin admin] PATH_INFO: `/v3/users/2d684ed7b44f4236aacd46fdeda37f1e` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:33:59.046519 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb53dd42-043f-4135-8ed7-27a0d1c9416d admin admin] RBAC: Authorizing `identity:delete_user(user_id=2d684ed7b44f4236aacd46fdeda37f1e)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:33:59.048479 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb53dd42-043f-4135-8ed7-27a0d1c9416d admin admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:33:59.057599 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94c6b845-e0d5-4f6f-bf0e-2b7b5aeb51bc admin admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:33:59.062299 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaf664eb-fb16-4fbe-83e4-0f948a5e7bdb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76c5181fc7be45279ac00017bb4640b3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:33:59.062944 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaf664eb-fb16-4fbe-83e4-0f948a5e7bdb admin admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:33:59.063234 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaf664eb-fb16-4fbe-83e4-0f948a5e7bdb admin admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:33:59.063502 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaf664eb-fb16-4fbe-83e4-0f948a5e7bdb admin admin] PATH_INFO: `/v3/roles` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:33:59.064142 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaf664eb-fb16-4fbe-83e4-0f948a5e7bdb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:33:59.066219 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaf664eb-fb16-4fbe-83e4-0f948a5e7bdb admin admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:33:59.077954 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 1353/2699] 10.222.0.18 () {58 vars in 1122 bytes} [Tue Sep 13 03:33:58 2022] GET /identity/v3/roles => generated 1420 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 13 03:33:59.082226 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaf664eb-fb16-4fbe-83e4-0f948a5e7bdb admin admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:33:59.092938 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaf664eb-fb16-4fbe-83e4-0f948a5e7bdb admin admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:33:59.097769 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e0a64af-874d-4914-a5d9-c36057ce0f09 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76c5181fc7be45279ac00017bb4640b3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:33:59.098363 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e0a64af-874d-4914-a5d9-c36057ce0f09 admin admin] REQUEST_METHOD: `PUT` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:33:59.098664 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e0a64af-874d-4914-a5d9-c36057ce0f09 admin admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:33:59.098939 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e0a64af-874d-4914-a5d9-c36057ce0f09 admin admin] PATH_INFO: `/v3/projects/2e1fe27fe08e4bc39b45c7a9d94510fe/users/e514837a23094f798c4730563b87d6af/roles/425d4b481c7846e0bc07e7e805d5b15f` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:33:59.100835 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e0a64af-874d-4914-a5d9-c36057ce0f09 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2e1fe27fe08e4bc39b45c7a9d94510fe, user_id=e514837a23094f798c4730563b87d6af, role_id=425d4b481c7846e0bc07e7e805d5b15f)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:33:59.102015 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-cb53dd42-043f-4135-8ed7-27a0d1c9416d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2d684ed7b44f4236aacd46fdeda37f1e'} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:33:59.103479 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e0a64af-874d-4914-a5d9-c36057ce0f09 admin admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:33:59.109084 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-cb53dd42-043f-4135-8ed7-27a0d1c9416d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2d684ed7b44f4236aacd46fdeda37f1e'} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:33:59.109706 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-cb53dd42-043f-4135-8ed7-27a0d1c9416d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2d684ed7b44f4236aacd46fdeda37f1e'} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:33:59.114418 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 1354/2700] 10.222.0.18 () {60 vars in 1483 bytes} [Tue Sep 13 03:33:59 2022] PUT /identity/v3/projects/2e1fe27fe08e4bc39b45c7a9d94510fe/users/e514837a23094f798c4730563b87d6af/roles/425d4b481c7846e0bc07e7e805d5b15f => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 13 03:33:59.123176 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-cb53dd42-043f-4135-8ed7-27a0d1c9416d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2d684ed7b44f4236aacd46fdeda37f1e'} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:33:59.124876 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe755b6-b663-4db7-9f01-7d5ed40c0434 None None] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:33:59.125164 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe755b6-b663-4db7-9f01-7d5ed40c0434 None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:33:59.125429 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe755b6-b663-4db7-9f01-7d5ed40c0434 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:33:59.130966 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-cb53dd42-043f-4135-8ed7-27a0d1c9416d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2d684ed7b44f4236aacd46fdeda37f1e'} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:33:59.137802 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 1347/2701] 10.222.0.18 () {58 vars in 1224 bytes} [Tue Sep 13 03:33:58 2022] DELETE /identity/v3/users/2d684ed7b44f4236aacd46fdeda37f1e => generated 0 bytes in 226 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 13 03:33:59.159380 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.auth.core [None req-dbe755b6-b663-4db7-9f01-7d5ed40c0434 None None] MFA Rules not processed for user `e514837a23094f798c4730563b87d6af`. Rule list: `[]` (Enabled: `True`). {{(pid=81399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 03:33:59.160625 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-dbe755b6-b663-4db7-9f01-7d5ed40c0434 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:33:59.232328 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 1355/2702] 10.222.0.18 () {58 vars in 958 bytes} [Tue Sep 13 03:33:59 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 13 03:33:59.244531 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb53dd42-043f-4135-8ed7-27a0d1c9416d admin admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:33:59.246174 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-cb53dd42-043f-4135-8ed7-27a0d1c9416d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:33:59.311005 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb53dd42-043f-4135-8ed7-27a0d1c9416d admin admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:33:59.314942 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-297c3070-b552-417b-9d2f-400e2105c9c1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '924ee3f608d34a18b0ddc4695f102a56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b9d94f6a68f7470d9494f112175620dc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:33:59.315550 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-297c3070-b552-417b-9d2f-400e2105c9c1 service cinder] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:33:59.315826 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-297c3070-b552-417b-9d2f-400e2105c9c1 service cinder] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:33:59.316097 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-297c3070-b552-417b-9d2f-400e2105c9c1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:33:59.325789 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-297c3070-b552-417b-9d2f-400e2105c9c1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:33:59.327271 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-297c3070-b552-417b-9d2f-400e2105c9c1 service cinder] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:33:59.337018 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 1348/2703] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Sep 13 03:33:59 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 03:33:59.425431 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbe755b6-b663-4db7-9f01-7d5ed40c0434 None None] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:33:59.427209 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-297c3070-b552-417b-9d2f-400e2105c9c1 service cinder] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:33:59.427572 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-dbe755b6-b663-4db7-9f01-7d5ed40c0434 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:33:59.437145 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-297c3070-b552-417b-9d2f-400e2105c9c1 service cinder] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:33:59.441550 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-385d3009-6e87-4152-b2a6-efd1a147c00a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '924ee3f608d34a18b0ddc4695f102a56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b9d94f6a68f7470d9494f112175620dc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:33:59.442154 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-385d3009-6e87-4152-b2a6-efd1a147c00a service cinder] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:33:59.442428 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-385d3009-6e87-4152-b2a6-efd1a147c00a service cinder] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:33:59.442726 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-385d3009-6e87-4152-b2a6-efd1a147c00a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:33:59.444535 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-385d3009-6e87-4152-b2a6-efd1a147c00a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:33:59.456155 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:33:59.491379 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:33:59.502179 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbe755b6-b663-4db7-9f01-7d5ed40c0434 None None] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:33:59.506978 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc311ae7-3b0a-4296-bfc0-d26defcbdcf2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76c5181fc7be45279ac00017bb4640b3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:33:59.507609 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc311ae7-3b0a-4296-bfc0-d26defcbdcf2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:33:59.507887 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc311ae7-3b0a-4296-bfc0-d26defcbdcf2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:33:59.508177 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc311ae7-3b0a-4296-bfc0-d26defcbdcf2 admin admin] PATH_INFO: `/v3/projects/8ab195c79b6c4e6a951a3921bca2a456` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:33:59.509311 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc311ae7-3b0a-4296-bfc0-d26defcbdcf2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8ab195c79b6c4e6a951a3921bca2a456)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:33:59.511261 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc311ae7-3b0a-4296-bfc0-d26defcbdcf2 admin admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:33:59.512485 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-385d3009-6e87-4152-b2a6-efd1a147c00a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:33:59.513926 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-385d3009-6e87-4152-b2a6-efd1a147c00a service cinder] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:33:59.523239 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 1349/2704] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Sep 13 03:33:59 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 13 03:33:59.560391 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-dc311ae7-3b0a-4296-bfc0-d26defcbdcf2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8ab195c79b6c4e6a951a3921bca2a456', 'request_id': 'req-dc311ae7-3b0a-4296-bfc0-d26defcbdcf2'} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:33:59.566357 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-dc311ae7-3b0a-4296-bfc0-d26defcbdcf2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8ab195c79b6c4e6a951a3921bca2a456', 'request_id': 'req-dc311ae7-3b0a-4296-bfc0-d26defcbdcf2'} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:33:59.572864 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-dc311ae7-3b0a-4296-bfc0-d26defcbdcf2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8ab195c79b6c4e6a951a3921bca2a456', 'request_id': 'req-dc311ae7-3b0a-4296-bfc0-d26defcbdcf2'} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:33:59.585567 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-dc311ae7-3b0a-4296-bfc0-d26defcbdcf2 admin admin] The token cache is being invalidate because project 8ab195c79b6c4e6a951a3921bca2a456 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81399) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 03:33:59.585855 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-dc311ae7-3b0a-4296-bfc0-d26defcbdcf2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:33:59.586480 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-dc311ae7-3b0a-4296-bfc0-d26defcbdcf2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:33:59.587997 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 1356/2705] 10.222.0.18 () {58 vars in 1233 bytes} [Tue Sep 13 03:33:59 2022] DELETE /identity/v3/projects/8ab195c79b6c4e6a951a3921bca2a456 => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 03:33:59.592822 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d521aee9-0aec-4111-b196-41c59b6dbcc3 None None] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:33:59.593134 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d521aee9-0aec-4111-b196-41c59b6dbcc3 None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:33:59.593405 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d521aee9-0aec-4111-b196-41c59b6dbcc3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:33:59.614920 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.auth.core [None req-d521aee9-0aec-4111-b196-41c59b6dbcc3 None None] MFA Rules not processed for user `49f280ab086b4610855c4ffa378517d9`. Rule list: `[]` (Enabled: `True`). {{(pid=81400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 03:33:59.616115 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-d521aee9-0aec-4111-b196-41c59b6dbcc3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:33:59.678076 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 1350/2706] 10.222.0.18 () {58 vars in 958 bytes} [Tue Sep 13 03:33:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 03:33:59.686356 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-063e9e85-8ff8-4934-bf24-454125c24b35 None None] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:33:59.686667 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-063e9e85-8ff8-4934-bf24-454125c24b35 None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:33:59.686970 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-063e9e85-8ff8-4934-bf24-454125c24b35 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:33:59.708227 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.auth.core [None req-063e9e85-8ff8-4934-bf24-454125c24b35 None None] MFA Rules not processed for user `49f280ab086b4610855c4ffa378517d9`. Rule list: `[]` (Enabled: `True`). {{(pid=81399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 03:33:59.709421 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-063e9e85-8ff8-4934-bf24-454125c24b35 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:33:59.717811 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 1357/2707] 10.222.0.18 () {58 vars in 958 bytes} [Tue Sep 13 03:33:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 03:33:59.722184 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d521aee9-0aec-4111-b196-41c59b6dbcc3 None None] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:33:59.733071 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d521aee9-0aec-4111-b196-41c59b6dbcc3 None None] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:33:59.739579 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-947e00ad-b3e9-403e-80ac-4fc0657e5b2d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76c5181fc7be45279ac00017bb4640b3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:33:59.740137 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-947e00ad-b3e9-403e-80ac-4fc0657e5b2d admin admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:33:59.740413 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-947e00ad-b3e9-403e-80ac-4fc0657e5b2d admin admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:33:59.740721 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-947e00ad-b3e9-403e-80ac-4fc0657e5b2d admin admin] PATH_INFO: `/v3/domains` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:33:59.741316 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-947e00ad-b3e9-403e-80ac-4fc0657e5b2d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 03:33:59.741617 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-947e00ad-b3e9-403e-80ac-4fc0657e5b2d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:33:59.743285 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-947e00ad-b3e9-403e-80ac-4fc0657e5b2d admin admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:33:59.755243 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 1351/2708] 10.222.0.18 () {58 vars in 1153 bytes} [Tue Sep 13 03:33:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 03:33:59.759706 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-063e9e85-8ff8-4934-bf24-454125c24b35 None None] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:33:59.770884 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-063e9e85-8ff8-4934-bf24-454125c24b35 None None] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:33:59.776362 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22bc3e05-afa7-41da-81aa-2a1b97b9546a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76c5181fc7be45279ac00017bb4640b3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:33:59.777177 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22bc3e05-afa7-41da-81aa-2a1b97b9546a admin admin] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:33:59.777508 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22bc3e05-afa7-41da-81aa-2a1b97b9546a admin admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:33:59.777906 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22bc3e05-afa7-41da-81aa-2a1b97b9546a admin admin] PATH_INFO: `/v3/projects` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:33:59.778850 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22bc3e05-afa7-41da-81aa-2a1b97b9546a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:33:59.781177 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22bc3e05-afa7-41da-81aa-2a1b97b9546a admin admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:33:59.812875 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 1358/2709] 10.222.0.18 () {60 vars in 1153 bytes} [Tue Sep 13 03:33:59 2022] POST /identity/v3/projects => generated 377 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 03:33:59.816660 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-947e00ad-b3e9-403e-80ac-4fc0657e5b2d admin admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:33:59.838812 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-947e00ad-b3e9-403e-80ac-4fc0657e5b2d admin admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:33:59.891056 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7b48234-c4ec-452f-bb27-242d11bcb656 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76c5181fc7be45279ac00017bb4640b3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:33:59.891666 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b48234-c4ec-452f-bb27-242d11bcb656 admin admin] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:33:59.891942 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b48234-c4ec-452f-bb27-242d11bcb656 admin admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:33:59.892213 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b48234-c4ec-452f-bb27-242d11bcb656 admin admin] PATH_INFO: `/v3/users` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:33:59.892855 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7b48234-c4ec-452f-bb27-242d11bcb656 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:33:59.895327 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7b48234-c4ec-452f-bb27-242d11bcb656 admin admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:33:59.916559 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 1352/2710] 10.222.0.18 () {60 vars in 1144 bytes} [Tue Sep 13 03:33:59 2022] POST /identity/v3/users => generated 289 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 13 03:33:59.920666 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22bc3e05-afa7-41da-81aa-2a1b97b9546a admin admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:33:59.930997 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22bc3e05-afa7-41da-81aa-2a1b97b9546a admin admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:33:59.935448 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0832632b-206d-49ea-8b8c-9abe5041ffb2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76c5181fc7be45279ac00017bb4640b3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:33:59.936035 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0832632b-206d-49ea-8b8c-9abe5041ffb2 admin admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:33:59.936314 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0832632b-206d-49ea-8b8c-9abe5041ffb2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:33:59.936587 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0832632b-206d-49ea-8b8c-9abe5041ffb2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:33:59.937227 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0832632b-206d-49ea-8b8c-9abe5041ffb2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:33:59.938850 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0832632b-206d-49ea-8b8c-9abe5041ffb2 admin admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:33:59.948787 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 1359/2711] 10.222.0.18 () {58 vars in 1122 bytes} [Tue Sep 13 03:33:59 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 03:33:59.952820 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7b48234-c4ec-452f-bb27-242d11bcb656 admin admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:33:59.963346 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7b48234-c4ec-452f-bb27-242d11bcb656 admin admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:33:59.967912 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d72ef4c-1d56-4331-aaba-55d4c9b4dc7f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76c5181fc7be45279ac00017bb4640b3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:33:59.968507 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d72ef4c-1d56-4331-aaba-55d4c9b4dc7f admin admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:33:59.968784 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d72ef4c-1d56-4331-aaba-55d4c9b4dc7f admin admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:33:59.969057 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d72ef4c-1d56-4331-aaba-55d4c9b4dc7f admin admin] PATH_INFO: `/v3/roles` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:33:59.969691 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d72ef4c-1d56-4331-aaba-55d4c9b4dc7f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:33:59.971328 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d72ef4c-1d56-4331-aaba-55d4c9b4dc7f admin admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:33:59.982646 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 1353/2712] 10.222.0.18 () {58 vars in 1122 bytes} [Tue Sep 13 03:33:59 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 03:33:59.986379 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0832632b-206d-49ea-8b8c-9abe5041ffb2 admin admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:33:59.996528 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0832632b-206d-49ea-8b8c-9abe5041ffb2 admin admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:34:00.001138 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fea5b472-3a4f-4b3a-a4ee-e66662924dcb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76c5181fc7be45279ac00017bb4640b3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:34:00.001784 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fea5b472-3a4f-4b3a-a4ee-e66662924dcb admin admin] REQUEST_METHOD: `PUT` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:34:00.002065 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fea5b472-3a4f-4b3a-a4ee-e66662924dcb admin admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:34:00.002344 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fea5b472-3a4f-4b3a-a4ee-e66662924dcb admin admin] PATH_INFO: `/v3/projects/a645f57266034232b7bf7a20a4d2c3cd/users/2e5fa847a02c4d52ad426126018eadac/roles/1eb7a5b5203945c29e6d22d653f47847` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:34:00.017004 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fea5b472-3a4f-4b3a-a4ee-e66662924dcb admin admin] RBAC: Authorizing `identity:create_grant(project_id=a645f57266034232b7bf7a20a4d2c3cd, user_id=2e5fa847a02c4d52ad426126018eadac, role_id=1eb7a5b5203945c29e6d22d653f47847)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:34:00.020329 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fea5b472-3a4f-4b3a-a4ee-e66662924dcb admin admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:34:00.033120 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 1360/2713] 10.222.0.18 () {60 vars in 1483 bytes} [Tue Sep 13 03:33:59 2022] PUT /identity/v3/projects/a645f57266034232b7bf7a20a4d2c3cd/users/2e5fa847a02c4d52ad426126018eadac/roles/1eb7a5b5203945c29e6d22d653f47847 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 03:34:00.044482 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16bda134-a7af-499a-8c09-66bebe2ea060 None None] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:34:00.044961 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16bda134-a7af-499a-8c09-66bebe2ea060 None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:34:00.045060 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16bda134-a7af-499a-8c09-66bebe2ea060 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:34:00.073456 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.auth.core [None req-16bda134-a7af-499a-8c09-66bebe2ea060 None None] MFA Rules not processed for user `2e5fa847a02c4d52ad426126018eadac`. Rule list: `[]` (Enabled: `True`). {{(pid=81400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 03:34:00.074672 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-16bda134-a7af-499a-8c09-66bebe2ea060 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:34:00.108649 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:34:00.139858 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 1354/2714] 10.222.0.18 () {58 vars in 958 bytes} [Tue Sep 13 03:34:00 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 13 03:34:00.145005 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fea5b472-3a4f-4b3a-a4ee-e66662924dcb admin admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:34:00.191722 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:34:00.205876 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fea5b472-3a4f-4b3a-a4ee-e66662924dcb admin admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:34:00.211148 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae7c247b-4d48-4a9d-b81c-2f7e7f725ace admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76c5181fc7be45279ac00017bb4640b3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:34:00.211741 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae7c247b-4d48-4a9d-b81c-2f7e7f725ace admin admin] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:34:00.212039 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae7c247b-4d48-4a9d-b81c-2f7e7f725ace admin admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:34:00.212332 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae7c247b-4d48-4a9d-b81c-2f7e7f725ace admin admin] PATH_INFO: `/v3/projects` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:34:00.212947 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae7c247b-4d48-4a9d-b81c-2f7e7f725ace admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:34:00.214866 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae7c247b-4d48-4a9d-b81c-2f7e7f725ace admin admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:34:00.239946 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 1361/2715] 10.222.0.18 () {60 vars in 1153 bytes} [Tue Sep 13 03:34:00 2022] POST /identity/v3/projects => generated 375 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 03:34:00.244056 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16bda134-a7af-499a-8c09-66bebe2ea060 None None] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:34:00.255508 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16bda134-a7af-499a-8c09-66bebe2ea060 None None] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:34:00.311392 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cecc2368-00a1-41e2-842c-da0dba04b662 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76c5181fc7be45279ac00017bb4640b3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:34:00.312024 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cecc2368-00a1-41e2-842c-da0dba04b662 admin admin] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:34:00.312302 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cecc2368-00a1-41e2-842c-da0dba04b662 admin admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:34:00.312618 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cecc2368-00a1-41e2-842c-da0dba04b662 admin admin] PATH_INFO: `/v3/users` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:34:00.313215 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cecc2368-00a1-41e2-842c-da0dba04b662 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:34:00.315224 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cecc2368-00a1-41e2-842c-da0dba04b662 admin admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:34:00.337843 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 1355/2716] 10.222.0.18 () {60 vars in 1144 bytes} [Tue Sep 13 03:34:00 2022] POST /identity/v3/users => generated 294 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 13 03:34:00.341921 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae7c247b-4d48-4a9d-b81c-2f7e7f725ace admin admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:34:00.352696 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae7c247b-4d48-4a9d-b81c-2f7e7f725ace admin admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:34:00.357403 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0029f2a-c91c-437b-bc0b-40869202aee6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76c5181fc7be45279ac00017bb4640b3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:34:00.358003 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0029f2a-c91c-437b-bc0b-40869202aee6 admin admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:34:00.358288 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0029f2a-c91c-437b-bc0b-40869202aee6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:34:00.358565 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0029f2a-c91c-437b-bc0b-40869202aee6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:34:00.359243 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0029f2a-c91c-437b-bc0b-40869202aee6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:34:00.360895 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0029f2a-c91c-437b-bc0b-40869202aee6 admin admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:34:00.373680 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 1362/2717] 10.222.0.18 () {58 vars in 1122 bytes} [Tue Sep 13 03:34:00 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 03:34:00.377970 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cecc2368-00a1-41e2-842c-da0dba04b662 admin admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:34:00.390871 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cecc2368-00a1-41e2-842c-da0dba04b662 admin admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:34:00.396185 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1bfdf25-06ca-48c2-b1ae-43814ccbb418 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76c5181fc7be45279ac00017bb4640b3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:34:00.397015 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1bfdf25-06ca-48c2-b1ae-43814ccbb418 admin admin] REQUEST_METHOD: `PUT` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:34:00.397299 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1bfdf25-06ca-48c2-b1ae-43814ccbb418 admin admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:34:00.397645 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1bfdf25-06ca-48c2-b1ae-43814ccbb418 admin admin] PATH_INFO: `/v3/projects/014ce25b5703425aa2df2976a58fcd7e/users/583ccc31a5544cbfb29df485ba79b092/roles/425d4b481c7846e0bc07e7e805d5b15f` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:34:00.413764 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1bfdf25-06ca-48c2-b1ae-43814ccbb418 admin admin] RBAC: Authorizing `identity:create_grant(project_id=014ce25b5703425aa2df2976a58fcd7e, user_id=583ccc31a5544cbfb29df485ba79b092, role_id=425d4b481c7846e0bc07e7e805d5b15f)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:34:00.416475 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1bfdf25-06ca-48c2-b1ae-43814ccbb418 admin admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:34:00.432775 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 1356/2718] 10.222.0.18 () {60 vars in 1483 bytes} [Tue Sep 13 03:34:00 2022] PUT /identity/v3/projects/014ce25b5703425aa2df2976a58fcd7e/users/583ccc31a5544cbfb29df485ba79b092/roles/425d4b481c7846e0bc07e7e805d5b15f => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 13 03:34:00.437337 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0029f2a-c91c-437b-bc0b-40869202aee6 admin admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:34:00.448741 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0029f2a-c91c-437b-bc0b-40869202aee6 admin admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:34:00.507522 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3286c4bd-a8c0-420e-b72b-9f463a3211ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76c5181fc7be45279ac00017bb4640b3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:34:00.508133 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3286c4bd-a8c0-420e-b72b-9f463a3211ba admin admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:34:00.508412 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3286c4bd-a8c0-420e-b72b-9f463a3211ba admin admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:34:00.508682 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3286c4bd-a8c0-420e-b72b-9f463a3211ba admin admin] PATH_INFO: `/v3/roles` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:34:00.509351 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3286c4bd-a8c0-420e-b72b-9f463a3211ba admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:34:00.510978 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3286c4bd-a8c0-420e-b72b-9f463a3211ba admin admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:34:00.521000 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 1363/2719] 10.222.0.18 () {58 vars in 1122 bytes} [Tue Sep 13 03:34:00 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 13 03:34:00.525388 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1bfdf25-06ca-48c2-b1ae-43814ccbb418 admin admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:34:00.536673 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1bfdf25-06ca-48c2-b1ae-43814ccbb418 admin admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:34:00.541541 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc274353-e397-4554-8064-0379721576ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76c5181fc7be45279ac00017bb4640b3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:34:00.542201 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc274353-e397-4554-8064-0379721576ae admin admin] REQUEST_METHOD: `PUT` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:34:00.542565 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc274353-e397-4554-8064-0379721576ae admin admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:34:00.542887 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc274353-e397-4554-8064-0379721576ae admin admin] PATH_INFO: `/v3/projects/014ce25b5703425aa2df2976a58fcd7e/users/583ccc31a5544cbfb29df485ba79b092/roles/425d4b481c7846e0bc07e7e805d5b15f` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:34:00.545211 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc274353-e397-4554-8064-0379721576ae admin admin] RBAC: Authorizing `identity:create_grant(project_id=014ce25b5703425aa2df2976a58fcd7e, user_id=583ccc31a5544cbfb29df485ba79b092, role_id=425d4b481c7846e0bc07e7e805d5b15f)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:34:00.547961 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc274353-e397-4554-8064-0379721576ae admin admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:34:00.559254 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 1357/2720] 10.222.0.18 () {60 vars in 1483 bytes} [Tue Sep 13 03:34:00 2022] PUT /identity/v3/projects/014ce25b5703425aa2df2976a58fcd7e/users/583ccc31a5544cbfb29df485ba79b092/roles/425d4b481c7846e0bc07e7e805d5b15f => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 13 03:34:00.570049 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4991fe6-6938-4dc0-a200-eaf167b5f954 None None] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:34:00.570308 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4991fe6-6938-4dc0-a200-eaf167b5f954 None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:34:00.570606 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4991fe6-6938-4dc0-a200-eaf167b5f954 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:34:00.599199 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.auth.core [None req-c4991fe6-6938-4dc0-a200-eaf167b5f954 None None] MFA Rules not processed for user `583ccc31a5544cbfb29df485ba79b092`. Rule list: `[]` (Enabled: `True`). {{(pid=81399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 03:34:00.600576 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-c4991fe6-6938-4dc0-a200-eaf167b5f954 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:34:00.674682 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 1364/2721] 10.222.0.18 () {58 vars in 958 bytes} [Tue Sep 13 03:34:00 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 03:34:00.686700 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc274353-e397-4554-8064-0379721576ae admin admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:34:00.688441 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-fc274353-e397-4554-8064-0379721576ae admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:34:00.750416 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc274353-e397-4554-8064-0379721576ae admin admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:34:00.754082 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93fdfd83-005f-44c9-b29c-18b5c8a16a8b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '924ee3f608d34a18b0ddc4695f102a56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b9d94f6a68f7470d9494f112175620dc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:34:00.754750 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93fdfd83-005f-44c9-b29c-18b5c8a16a8b service cinder] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:34:00.755031 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93fdfd83-005f-44c9-b29c-18b5c8a16a8b service cinder] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:34:00.755299 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93fdfd83-005f-44c9-b29c-18b5c8a16a8b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:34:00.800066 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:34:00.818153 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93fdfd83-005f-44c9-b29c-18b5c8a16a8b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:34:00.819748 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93fdfd83-005f-44c9-b29c-18b5c8a16a8b service cinder] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:34:00.828878 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 1358/2722] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Sep 13 03:34:00 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 143 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 03:34:07.192413 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4991fe6-6938-4dc0-a200-eaf167b5f954 None None] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:34:07.193866 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-c4991fe6-6938-4dc0-a200-eaf167b5f954 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:34:07.240144 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:34:07.262928 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4991fe6-6938-4dc0-a200-eaf167b5f954 None None] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:34:07.267790 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f50717b-ec66-4802-9d0c-96a943024a09 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b99f1b2374d4d2f8db78306316180d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b9d94f6a68f7470d9494f112175620dc', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:34:07.268390 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f50717b-ec66-4802-9d0c-96a943024a09 service placement] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:34:07.268667 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f50717b-ec66-4802-9d0c-96a943024a09 service placement] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:34:07.268957 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f50717b-ec66-4802-9d0c-96a943024a09 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:34:07.271170 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-1f50717b-ec66-4802-9d0c-96a943024a09 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:34:07.282782 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f50717b-ec66-4802-9d0c-96a943024a09 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:34:07.284411 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f50717b-ec66-4802-9d0c-96a943024a09 service placement] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:34:07.296228 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 1365/2723] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Sep 13 03:34:07 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 03:34:07.439704 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93fdfd83-005f-44c9-b29c-18b5c8a16a8b service cinder] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:34:07.442054 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-93fdfd83-005f-44c9-b29c-18b5c8a16a8b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:34:07.509525 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93fdfd83-005f-44c9-b29c-18b5c8a16a8b service cinder] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:34:07.514294 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc185228-1639-4896-ab58-afc842986091 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76c5181fc7be45279ac00017bb4640b3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:34:07.514951 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc185228-1639-4896-ab58-afc842986091 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:34:07.515228 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc185228-1639-4896-ab58-afc842986091 admin admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:34:07.515498 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc185228-1639-4896-ab58-afc842986091 admin admin] PATH_INFO: `/v3/users/071f2a03df3944a7aa6e4953ff1e23ac` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:34:07.516632 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc185228-1639-4896-ab58-afc842986091 admin admin] RBAC: Authorizing `identity:delete_user(user_id=071f2a03df3944a7aa6e4953ff1e23ac)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:34:07.518515 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f50717b-ec66-4802-9d0c-96a943024a09 service placement] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:34:07.518655 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc185228-1639-4896-ab58-afc842986091 admin admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:34:07.530483 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f50717b-ec66-4802-9d0c-96a943024a09 service placement] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:34:07.534057 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6b50dc8-f58e-4589-accd-85bbc44e18a8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '924ee3f608d34a18b0ddc4695f102a56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b9d94f6a68f7470d9494f112175620dc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:34:07.534683 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6b50dc8-f58e-4589-accd-85bbc44e18a8 service cinder] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:34:07.535489 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6b50dc8-f58e-4589-accd-85bbc44e18a8 service cinder] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:34:07.535759 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6b50dc8-f58e-4589-accd-85bbc44e18a8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:34:07.545127 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6b50dc8-f58e-4589-accd-85bbc44e18a8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:34:07.546657 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6b50dc8-f58e-4589-accd-85bbc44e18a8 service cinder] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:34:07.557049 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 1366/2724] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Sep 13 03:34:07 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 13 03:34:07.566309 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-bc185228-1639-4896-ab58-afc842986091 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '071f2a03df3944a7aa6e4953ff1e23ac'} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:34:07.573978 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-bc185228-1639-4896-ab58-afc842986091 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '071f2a03df3944a7aa6e4953ff1e23ac'} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:34:07.574748 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-bc185228-1639-4896-ab58-afc842986091 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '071f2a03df3944a7aa6e4953ff1e23ac'} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:34:07.588044 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-bc185228-1639-4896-ab58-afc842986091 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '071f2a03df3944a7aa6e4953ff1e23ac'} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:34:07.594808 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-bc185228-1639-4896-ab58-afc842986091 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '071f2a03df3944a7aa6e4953ff1e23ac'} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:34:07.601244 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 1359/2725] 10.222.0.18 () {58 vars in 1224 bytes} [Tue Sep 13 03:34:07 2022] DELETE /identity/v3/users/071f2a03df3944a7aa6e4953ff1e23ac => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 03:34:07.605977 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da86083-3cbd-46e1-ba17-1d7eba06718b None None] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:34:07.606252 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da86083-3cbd-46e1-ba17-1d7eba06718b None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:34:07.606525 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da86083-3cbd-46e1-ba17-1d7eba06718b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:34:07.625931 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.auth.core [None req-3da86083-3cbd-46e1-ba17-1d7eba06718b None None] MFA Rules not processed for user `49f280ab086b4610855c4ffa378517d9`. Rule list: `[]` (Enabled: `True`). {{(pid=81399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 03:34:07.627258 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-3da86083-3cbd-46e1-ba17-1d7eba06718b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:34:07.686848 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 1367/2726] 10.222.0.18 () {58 vars in 958 bytes} [Tue Sep 13 03:34:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 13 03:34:07.696708 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc185228-1639-4896-ab58-afc842986091 admin admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:34:07.698209 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-bc185228-1639-4896-ab58-afc842986091 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:34:07.757942 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc185228-1639-4896-ab58-afc842986091 admin admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:34:07.762311 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12bedabc-97b7-46e3-8565-c90690e1d1da service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdffbc851086441ebd01022d1e4c2609', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b9d94f6a68f7470d9494f112175620dc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:34:07.762945 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12bedabc-97b7-46e3-8565-c90690e1d1da service neutron] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:34:07.763223 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12bedabc-97b7-46e3-8565-c90690e1d1da service neutron] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:34:07.763499 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12bedabc-97b7-46e3-8565-c90690e1d1da service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:34:07.772646 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12bedabc-97b7-46e3-8565-c90690e1d1da service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:34:07.774073 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12bedabc-97b7-46e3-8565-c90690e1d1da service neutron] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:34:07.784062 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 1360/2727] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Sep 13 03:34:07 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 03:34:08.057123 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3da86083-3cbd-46e1-ba17-1d7eba06718b None None] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:34:08.058559 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-3da86083-3cbd-46e1-ba17-1d7eba06718b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:34:08.076994 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3da86083-3cbd-46e1-ba17-1d7eba06718b None None] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:34:08.082758 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-928adff1-f778-4c2b-8c97-ede0198e88c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76c5181fc7be45279ac00017bb4640b3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:34:08.083913 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-928adff1-f778-4c2b-8c97-ede0198e88c7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:34:08.084190 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-928adff1-f778-4c2b-8c97-ede0198e88c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:34:08.084464 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-928adff1-f778-4c2b-8c97-ede0198e88c7 admin admin] PATH_INFO: `/v3/projects/0b6c84750c02479c8d8269c39a9fbe4c` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:34:08.085647 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-928adff1-f778-4c2b-8c97-ede0198e88c7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0b6c84750c02479c8d8269c39a9fbe4c)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:34:08.087791 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-928adff1-f778-4c2b-8c97-ede0198e88c7 admin admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:34:08.132769 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-928adff1-f778-4c2b-8c97-ede0198e88c7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0b6c84750c02479c8d8269c39a9fbe4c', 'request_id': 'req-928adff1-f778-4c2b-8c97-ede0198e88c7'} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:34:08.138013 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-928adff1-f778-4c2b-8c97-ede0198e88c7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0b6c84750c02479c8d8269c39a9fbe4c', 'request_id': 'req-928adff1-f778-4c2b-8c97-ede0198e88c7'} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:34:08.144133 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-928adff1-f778-4c2b-8c97-ede0198e88c7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0b6c84750c02479c8d8269c39a9fbe4c', 'request_id': 'req-928adff1-f778-4c2b-8c97-ede0198e88c7'} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:34:08.156849 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-928adff1-f778-4c2b-8c97-ede0198e88c7 admin admin] The token cache is being invalidate because project 0b6c84750c02479c8d8269c39a9fbe4c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81399) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 03:34:08.157137 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-928adff1-f778-4c2b-8c97-ede0198e88c7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:34:08.157749 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-928adff1-f778-4c2b-8c97-ede0198e88c7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:34:08.159391 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 1368/2728] 10.222.0.18 () {58 vars in 1233 bytes} [Tue Sep 13 03:34:08 2022] DELETE /identity/v3/projects/0b6c84750c02479c8d8269c39a9fbe4c => generated 0 bytes in 103 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 03:34:08.164141 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12bedabc-97b7-46e3-8565-c90690e1d1da service neutron] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:34:08.170126 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:34:08.174301 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12bedabc-97b7-46e3-8565-c90690e1d1da service neutron] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:34:08.232133 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b0de997-7ea6-4cf2-867e-e1df5aeca203 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76c5181fc7be45279ac00017bb4640b3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:34:08.232759 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b0de997-7ea6-4cf2-867e-e1df5aeca203 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:34:08.233034 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b0de997-7ea6-4cf2-867e-e1df5aeca203 admin admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:34:08.233306 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b0de997-7ea6-4cf2-867e-e1df5aeca203 admin admin] PATH_INFO: `/v3/users/e514837a23094f798c4730563b87d6af` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:34:08.234428 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b0de997-7ea6-4cf2-867e-e1df5aeca203 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e514837a23094f798c4730563b87d6af)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:34:08.236375 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b0de997-7ea6-4cf2-867e-e1df5aeca203 admin admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:34:08.282296 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-9b0de997-7ea6-4cf2-867e-e1df5aeca203 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e514837a23094f798c4730563b87d6af'} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:34:08.289823 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-9b0de997-7ea6-4cf2-867e-e1df5aeca203 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e514837a23094f798c4730563b87d6af'} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:34:08.290450 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-9b0de997-7ea6-4cf2-867e-e1df5aeca203 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e514837a23094f798c4730563b87d6af'} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:34:08.303120 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-9b0de997-7ea6-4cf2-867e-e1df5aeca203 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e514837a23094f798c4730563b87d6af'} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:34:08.309992 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-9b0de997-7ea6-4cf2-867e-e1df5aeca203 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e514837a23094f798c4730563b87d6af'} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:34:08.320748 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 1361/2729] 10.222.0.18 () {58 vars in 1224 bytes} [Tue Sep 13 03:34:08 2022] DELETE /identity/v3/users/e514837a23094f798c4730563b87d6af => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 03:34:08.590296 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-928adff1-f778-4c2b-8c97-ede0198e88c7 admin admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:34:08.591968 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-928adff1-f778-4c2b-8c97-ede0198e88c7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:34:08.607179 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-928adff1-f778-4c2b-8c97-ede0198e88c7 admin admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:34:08.649466 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:34:08.665825 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c092b0e-2a6a-4a90-8b4b-78001be6ba4d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76c5181fc7be45279ac00017bb4640b3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:34:08.666389 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c092b0e-2a6a-4a90-8b4b-78001be6ba4d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:34:08.666709 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c092b0e-2a6a-4a90-8b4b-78001be6ba4d admin admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:34:08.666984 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c092b0e-2a6a-4a90-8b4b-78001be6ba4d admin admin] PATH_INFO: `/v3/projects/2e1fe27fe08e4bc39b45c7a9d94510fe` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:34:08.668100 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c092b0e-2a6a-4a90-8b4b-78001be6ba4d admin admin] RBAC: Authorizing `identity:delete_project(project_id=2e1fe27fe08e4bc39b45c7a9d94510fe)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:34:08.670067 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c092b0e-2a6a-4a90-8b4b-78001be6ba4d admin admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:34:08.715195 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-7c092b0e-2a6a-4a90-8b4b-78001be6ba4d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2e1fe27fe08e4bc39b45c7a9d94510fe', 'request_id': 'req-7c092b0e-2a6a-4a90-8b4b-78001be6ba4d'} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:34:08.722001 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-7c092b0e-2a6a-4a90-8b4b-78001be6ba4d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2e1fe27fe08e4bc39b45c7a9d94510fe', 'request_id': 'req-7c092b0e-2a6a-4a90-8b4b-78001be6ba4d'} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:34:08.734657 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-7c092b0e-2a6a-4a90-8b4b-78001be6ba4d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2e1fe27fe08e4bc39b45c7a9d94510fe', 'request_id': 'req-7c092b0e-2a6a-4a90-8b4b-78001be6ba4d'} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:34:08.747905 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-7c092b0e-2a6a-4a90-8b4b-78001be6ba4d admin admin] The token cache is being invalidate because project 2e1fe27fe08e4bc39b45c7a9d94510fe was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81399) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 03:34:08.748193 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-7c092b0e-2a6a-4a90-8b4b-78001be6ba4d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:34:08.748827 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-7c092b0e-2a6a-4a90-8b4b-78001be6ba4d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:34:08.750293 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 1369/2730] 10.222.0.18 () {58 vars in 1233 bytes} [Tue Sep 13 03:34:08 2022] DELETE /identity/v3/projects/2e1fe27fe08e4bc39b45c7a9d94510fe => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 03:34:08.755740 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-478640a5-fd2e-4195-b2b0-9276df695861 None None] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:34:08.756235 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-478640a5-fd2e-4195-b2b0-9276df695861 None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:34:08.756344 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-478640a5-fd2e-4195-b2b0-9276df695861 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:34:08.775208 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.auth.core [None req-478640a5-fd2e-4195-b2b0-9276df695861 None None] MFA Rules not processed for user `49f280ab086b4610855c4ffa378517d9`. Rule list: `[]` (Enabled: `True`). {{(pid=81400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 03:34:08.776384 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-478640a5-fd2e-4195-b2b0-9276df695861 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:34:08.834538 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 1362/2731] 10.222.0.18 () {58 vars in 958 bytes} [Tue Sep 13 03:34:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 03:34:08.843223 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4faf0daf-4661-4d23-8eae-c10acd33a841 None None] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:34:08.843505 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4faf0daf-4661-4d23-8eae-c10acd33a841 None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:34:08.843775 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4faf0daf-4661-4d23-8eae-c10acd33a841 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:34:08.863278 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.auth.core [None req-4faf0daf-4661-4d23-8eae-c10acd33a841 None None] MFA Rules not processed for user `49f280ab086b4610855c4ffa378517d9`. Rule list: `[]` (Enabled: `True`). {{(pid=81399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 03:34:08.864975 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-4faf0daf-4661-4d23-8eae-c10acd33a841 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:34:08.872814 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 1370/2732] 10.222.0.18 () {58 vars in 958 bytes} [Tue Sep 13 03:34:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 03:34:08.877038 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-478640a5-fd2e-4195-b2b0-9276df695861 None None] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:34:08.887127 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-478640a5-fd2e-4195-b2b0-9276df695861 None None] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:34:08.891575 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a6112b1-c078-4547-8c1a-50a470e2ff20 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76c5181fc7be45279ac00017bb4640b3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:34:08.892156 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a6112b1-c078-4547-8c1a-50a470e2ff20 admin admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:34:08.892429 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a6112b1-c078-4547-8c1a-50a470e2ff20 admin admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:34:08.892693 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a6112b1-c078-4547-8c1a-50a470e2ff20 admin admin] PATH_INFO: `/v3/domains` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:34:08.893316 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a6112b1-c078-4547-8c1a-50a470e2ff20 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 03:34:08.893614 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a6112b1-c078-4547-8c1a-50a470e2ff20 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:34:08.895221 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a6112b1-c078-4547-8c1a-50a470e2ff20 admin admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:34:08.907159 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 1363/2733] 10.222.0.18 () {58 vars in 1153 bytes} [Tue Sep 13 03:34:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 03:34:08.911460 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4faf0daf-4661-4d23-8eae-c10acd33a841 None None] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:34:08.921155 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4faf0daf-4661-4d23-8eae-c10acd33a841 None None] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:34:08.925828 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e0be3e0-8ea1-42b8-88a3-7c54112bc268 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76c5181fc7be45279ac00017bb4640b3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:34:08.926417 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e0be3e0-8ea1-42b8-88a3-7c54112bc268 admin admin] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:34:08.927226 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e0be3e0-8ea1-42b8-88a3-7c54112bc268 admin admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:34:08.927498 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e0be3e0-8ea1-42b8-88a3-7c54112bc268 admin admin] PATH_INFO: `/v3/projects` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:34:08.928131 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e0be3e0-8ea1-42b8-88a3-7c54112bc268 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:34:08.929996 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e0be3e0-8ea1-42b8-88a3-7c54112bc268 admin admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:34:08.956977 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 1371/2734] 10.222.0.18 () {60 vars in 1153 bytes} [Tue Sep 13 03:34:08 2022] POST /identity/v3/projects => generated 385 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 13 03:34:08.961278 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a6112b1-c078-4547-8c1a-50a470e2ff20 admin admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:34:08.971131 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a6112b1-c078-4547-8c1a-50a470e2ff20 admin admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:34:08.979313 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:34:09.028249 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62c60524-ecd4-4107-887a-9443db952f9e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76c5181fc7be45279ac00017bb4640b3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:34:09.028844 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62c60524-ecd4-4107-887a-9443db952f9e admin admin] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:34:09.029121 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62c60524-ecd4-4107-887a-9443db952f9e admin admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:34:09.029390 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62c60524-ecd4-4107-887a-9443db952f9e admin admin] PATH_INFO: `/v3/users` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:34:09.030004 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62c60524-ecd4-4107-887a-9443db952f9e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:34:09.031944 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62c60524-ecd4-4107-887a-9443db952f9e admin admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:34:09.054397 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 1364/2735] 10.222.0.18 () {60 vars in 1144 bytes} [Tue Sep 13 03:34:08 2022] POST /identity/v3/users => generated 293 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 03:34:09.058476 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e0be3e0-8ea1-42b8-88a3-7c54112bc268 admin admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:34:09.069010 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e0be3e0-8ea1-42b8-88a3-7c54112bc268 admin admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:34:09.073770 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3cc60b7-3742-4176-9d17-15da0df5c386 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76c5181fc7be45279ac00017bb4640b3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:34:09.074373 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3cc60b7-3742-4176-9d17-15da0df5c386 admin admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:34:09.074678 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3cc60b7-3742-4176-9d17-15da0df5c386 admin admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:34:09.074954 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3cc60b7-3742-4176-9d17-15da0df5c386 admin admin] PATH_INFO: `/v3/roles` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:34:09.075590 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3cc60b7-3742-4176-9d17-15da0df5c386 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:34:09.077210 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3cc60b7-3742-4176-9d17-15da0df5c386 admin admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:34:09.087150 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 1372/2736] 10.222.0.18 () {58 vars in 1122 bytes} [Tue Sep 13 03:34:09 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 13 03:34:09.091202 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62c60524-ecd4-4107-887a-9443db952f9e admin admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:34:09.102171 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62c60524-ecd4-4107-887a-9443db952f9e admin admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:34:09.106980 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b0e759a-f622-40cc-b155-577094a7297c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76c5181fc7be45279ac00017bb4640b3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:34:09.107575 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b0e759a-f622-40cc-b155-577094a7297c admin admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:34:09.107917 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b0e759a-f622-40cc-b155-577094a7297c admin admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:34:09.108233 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b0e759a-f622-40cc-b155-577094a7297c admin admin] PATH_INFO: `/v3/roles` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:34:09.108871 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b0e759a-f622-40cc-b155-577094a7297c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:34:09.110458 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b0e759a-f622-40cc-b155-577094a7297c admin admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:34:09.121676 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 1365/2737] 10.222.0.18 () {58 vars in 1122 bytes} [Tue Sep 13 03:34:09 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 03:34:09.125755 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3cc60b7-3742-4176-9d17-15da0df5c386 admin admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:34:09.135539 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3cc60b7-3742-4176-9d17-15da0df5c386 admin admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:34:09.140147 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5122f8b6-f870-4a8e-9d52-aa17bdae3a28 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76c5181fc7be45279ac00017bb4640b3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:34:09.140762 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5122f8b6-f870-4a8e-9d52-aa17bdae3a28 admin admin] REQUEST_METHOD: `PUT` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:34:09.141038 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5122f8b6-f870-4a8e-9d52-aa17bdae3a28 admin admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:34:09.141306 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5122f8b6-f870-4a8e-9d52-aa17bdae3a28 admin admin] PATH_INFO: `/v3/projects/33fc66045c5442b5a2b1e8ad5d41fcaf/users/301ed911116b4c22a3dd86f24a77bdfd/roles/1eb7a5b5203945c29e6d22d653f47847` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:34:09.156288 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5122f8b6-f870-4a8e-9d52-aa17bdae3a28 admin admin] RBAC: Authorizing `identity:create_grant(project_id=33fc66045c5442b5a2b1e8ad5d41fcaf, user_id=301ed911116b4c22a3dd86f24a77bdfd, role_id=1eb7a5b5203945c29e6d22d653f47847)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:34:09.158958 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5122f8b6-f870-4a8e-9d52-aa17bdae3a28 admin admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:34:09.172057 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 1373/2738] 10.222.0.18 () {60 vars in 1483 bytes} [Tue Sep 13 03:34:09 2022] PUT /identity/v3/projects/33fc66045c5442b5a2b1e8ad5d41fcaf/users/301ed911116b4c22a3dd86f24a77bdfd/roles/1eb7a5b5203945c29e6d22d653f47847 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 03:34:09.182659 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06af3337-5652-4049-9838-4365af3b7001 None None] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:34:09.183053 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06af3337-5652-4049-9838-4365af3b7001 None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:34:09.183495 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06af3337-5652-4049-9838-4365af3b7001 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:34:09.214643 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.auth.core [None req-06af3337-5652-4049-9838-4365af3b7001 None None] MFA Rules not processed for user `301ed911116b4c22a3dd86f24a77bdfd`. Rule list: `[]` (Enabled: `True`). {{(pid=81400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 03:34:09.215843 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-06af3337-5652-4049-9838-4365af3b7001 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:34:09.278158 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 1366/2739] 10.222.0.18 () {58 vars in 958 bytes} [Tue Sep 13 03:34:09 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 03:34:09.282721 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5122f8b6-f870-4a8e-9d52-aa17bdae3a28 admin admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:34:09.343760 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5122f8b6-f870-4a8e-9d52-aa17bdae3a28 admin admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:34:09.348473 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b60b4420-219d-4018-969f-de9d67c4ef6d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76c5181fc7be45279ac00017bb4640b3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:34:09.349607 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b60b4420-219d-4018-969f-de9d67c4ef6d admin admin] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:34:09.349895 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b60b4420-219d-4018-969f-de9d67c4ef6d admin admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:34:09.350284 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b60b4420-219d-4018-969f-de9d67c4ef6d admin admin] PATH_INFO: `/v3/projects` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:34:09.350870 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b60b4420-219d-4018-969f-de9d67c4ef6d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:34:09.352696 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b60b4420-219d-4018-969f-de9d67c4ef6d admin admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:34:09.368661 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:34:09.385288 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 1374/2740] 10.222.0.18 () {60 vars in 1153 bytes} [Tue Sep 13 03:34:09 2022] POST /identity/v3/projects => generated 387 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 13 03:34:09.386486 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06af3337-5652-4049-9838-4365af3b7001 None None] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:34:09.397972 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06af3337-5652-4049-9838-4365af3b7001 None None] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:34:09.453852 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83d87b9c-2f35-4607-a48c-29d6f6152eb7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76c5181fc7be45279ac00017bb4640b3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:34:09.454447 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d87b9c-2f35-4607-a48c-29d6f6152eb7 admin admin] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:34:09.454753 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d87b9c-2f35-4607-a48c-29d6f6152eb7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:34:09.455103 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d87b9c-2f35-4607-a48c-29d6f6152eb7 admin admin] PATH_INFO: `/v3/users` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:34:09.455713 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83d87b9c-2f35-4607-a48c-29d6f6152eb7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:34:09.457621 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83d87b9c-2f35-4607-a48c-29d6f6152eb7 admin admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:34:09.478412 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 1367/2741] 10.222.0.18 () {60 vars in 1144 bytes} [Tue Sep 13 03:34:09 2022] POST /identity/v3/users => generated 300 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 13 03:34:09.482538 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b60b4420-219d-4018-969f-de9d67c4ef6d admin admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:34:09.492985 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b60b4420-219d-4018-969f-de9d67c4ef6d admin admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:34:09.497752 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c513099f-10b2-4b5e-83cb-b716ce6e87de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76c5181fc7be45279ac00017bb4640b3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:34:09.498364 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c513099f-10b2-4b5e-83cb-b716ce6e87de admin admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:34:09.498681 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c513099f-10b2-4b5e-83cb-b716ce6e87de admin admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:34:09.498950 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c513099f-10b2-4b5e-83cb-b716ce6e87de admin admin] PATH_INFO: `/v3/roles` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:34:09.499580 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c513099f-10b2-4b5e-83cb-b716ce6e87de admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:34:09.501143 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c513099f-10b2-4b5e-83cb-b716ce6e87de admin admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:34:09.511449 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 1375/2742] 10.222.0.18 () {58 vars in 1122 bytes} [Tue Sep 13 03:34:09 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 03:34:09.515452 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83d87b9c-2f35-4607-a48c-29d6f6152eb7 admin admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:34:09.525685 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83d87b9c-2f35-4607-a48c-29d6f6152eb7 admin admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:34:09.530239 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f4990c1-3241-414b-b3d8-9901a054ab10 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76c5181fc7be45279ac00017bb4640b3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:34:09.530915 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f4990c1-3241-414b-b3d8-9901a054ab10 admin admin] REQUEST_METHOD: `PUT` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:34:09.531194 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f4990c1-3241-414b-b3d8-9901a054ab10 admin admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:34:09.531463 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f4990c1-3241-414b-b3d8-9901a054ab10 admin admin] PATH_INFO: `/v3/projects/464a003c947c445c8092c1bdd20050c0/users/06827865ef86474d93e6f33ae232691c/roles/425d4b481c7846e0bc07e7e805d5b15f` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:34:09.534423 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:34:09.547815 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f4990c1-3241-414b-b3d8-9901a054ab10 admin admin] RBAC: Authorizing `identity:create_grant(project_id=464a003c947c445c8092c1bdd20050c0, user_id=06827865ef86474d93e6f33ae232691c, role_id=425d4b481c7846e0bc07e7e805d5b15f)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:34:09.550462 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f4990c1-3241-414b-b3d8-9901a054ab10 admin admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:34:09.562976 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 1368/2743] 10.222.0.18 () {60 vars in 1483 bytes} [Tue Sep 13 03:34:09 2022] PUT /identity/v3/projects/464a003c947c445c8092c1bdd20050c0/users/06827865ef86474d93e6f33ae232691c/roles/425d4b481c7846e0bc07e7e805d5b15f => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 13 03:34:09.566876 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c513099f-10b2-4b5e-83cb-b716ce6e87de admin admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:34:09.576531 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c513099f-10b2-4b5e-83cb-b716ce6e87de admin admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:34:09.630836 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e718e27a-a612-49ea-8a7a-80b1a4ca21fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76c5181fc7be45279ac00017bb4640b3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:34:09.631466 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e718e27a-a612-49ea-8a7a-80b1a4ca21fd admin admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:34:09.631688 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e718e27a-a612-49ea-8a7a-80b1a4ca21fd admin admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:34:09.631951 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e718e27a-a612-49ea-8a7a-80b1a4ca21fd admin admin] PATH_INFO: `/v3/roles` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:34:09.632574 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e718e27a-a612-49ea-8a7a-80b1a4ca21fd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:34:09.634152 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e718e27a-a612-49ea-8a7a-80b1a4ca21fd admin admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:34:09.643552 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 1376/2744] 10.222.0.18 () {58 vars in 1122 bytes} [Tue Sep 13 03:34:09 2022] GET /identity/v3/roles => generated 1420 bytes in 77 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 03:34:09.648301 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f4990c1-3241-414b-b3d8-9901a054ab10 admin admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:34:09.658537 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f4990c1-3241-414b-b3d8-9901a054ab10 admin admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:34:09.664022 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d461a548-acdb-44b4-835e-7f15d96acbb1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76c5181fc7be45279ac00017bb4640b3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:34:09.664669 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d461a548-acdb-44b4-835e-7f15d96acbb1 admin admin] REQUEST_METHOD: `PUT` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:34:09.664948 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d461a548-acdb-44b4-835e-7f15d96acbb1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:34:09.665222 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d461a548-acdb-44b4-835e-7f15d96acbb1 admin admin] PATH_INFO: `/v3/projects/464a003c947c445c8092c1bdd20050c0/users/06827865ef86474d93e6f33ae232691c/roles/425d4b481c7846e0bc07e7e805d5b15f` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:34:09.667156 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d461a548-acdb-44b4-835e-7f15d96acbb1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=464a003c947c445c8092c1bdd20050c0, user_id=06827865ef86474d93e6f33ae232691c, role_id=425d4b481c7846e0bc07e7e805d5b15f)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:34:09.669812 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d461a548-acdb-44b4-835e-7f15d96acbb1 admin admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:34:09.679819 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 1369/2745] 10.222.0.18 () {60 vars in 1483 bytes} [Tue Sep 13 03:34:09 2022] PUT /identity/v3/projects/464a003c947c445c8092c1bdd20050c0/users/06827865ef86474d93e6f33ae232691c/roles/425d4b481c7846e0bc07e7e805d5b15f => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 03:34:09.690576 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96eeec5e-d35a-45b4-80da-43b90e46ecee None None] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:34:09.690896 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96eeec5e-d35a-45b4-80da-43b90e46ecee None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:34:09.691167 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96eeec5e-d35a-45b4-80da-43b90e46ecee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:34:09.719272 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.auth.core [None req-96eeec5e-d35a-45b4-80da-43b90e46ecee None None] MFA Rules not processed for user `06827865ef86474d93e6f33ae232691c`. Rule list: `[]` (Enabled: `True`). {{(pid=81399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 03:34:09.720396 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-96eeec5e-d35a-45b4-80da-43b90e46ecee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:34:09.787523 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 1377/2746] 10.222.0.18 () {58 vars in 958 bytes} [Tue Sep 13 03:34:09 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 03:34:09.798121 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d461a548-acdb-44b4-835e-7f15d96acbb1 admin admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:34:09.799902 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-d461a548-acdb-44b4-835e-7f15d96acbb1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:34:09.857697 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d461a548-acdb-44b4-835e-7f15d96acbb1 admin admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:34:09.861960 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7596679-4b9d-4cda-a33e-113999e5363b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '924ee3f608d34a18b0ddc4695f102a56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b9d94f6a68f7470d9494f112175620dc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:34:09.862835 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7596679-4b9d-4cda-a33e-113999e5363b service cinder] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:34:09.863100 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7596679-4b9d-4cda-a33e-113999e5363b service cinder] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:34:09.863404 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7596679-4b9d-4cda-a33e-113999e5363b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:34:09.872483 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7596679-4b9d-4cda-a33e-113999e5363b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:34:09.873899 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7596679-4b9d-4cda-a33e-113999e5363b service cinder] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:34:09.883727 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 1370/2747] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Sep 13 03:34:09 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 03:34:10.413440 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96eeec5e-d35a-45b4-80da-43b90e46ecee None None] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:34:10.474440 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96eeec5e-d35a-45b4-80da-43b90e46ecee None None] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:34:10.478981 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d52c5ff8-58fb-42b5-8edd-340c2c53dd6a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76c5181fc7be45279ac00017bb4640b3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:34:10.479598 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d52c5ff8-58fb-42b5-8edd-340c2c53dd6a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:34:10.479879 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d52c5ff8-58fb-42b5-8edd-340c2c53dd6a admin admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:34:10.480149 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d52c5ff8-58fb-42b5-8edd-340c2c53dd6a admin admin] PATH_INFO: `/v3/users/301ed911116b4c22a3dd86f24a77bdfd` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:34:10.481255 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d52c5ff8-58fb-42b5-8edd-340c2c53dd6a admin admin] RBAC: Authorizing `identity:delete_user(user_id=301ed911116b4c22a3dd86f24a77bdfd)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:34:10.483272 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d52c5ff8-58fb-42b5-8edd-340c2c53dd6a admin admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:34:10.488567 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:34:10.527025 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-d52c5ff8-58fb-42b5-8edd-340c2c53dd6a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '301ed911116b4c22a3dd86f24a77bdfd'} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:34:10.533853 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-d52c5ff8-58fb-42b5-8edd-340c2c53dd6a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '301ed911116b4c22a3dd86f24a77bdfd'} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:34:10.545843 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-d52c5ff8-58fb-42b5-8edd-340c2c53dd6a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '301ed911116b4c22a3dd86f24a77bdfd'} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:34:10.546465 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-d52c5ff8-58fb-42b5-8edd-340c2c53dd6a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '301ed911116b4c22a3dd86f24a77bdfd'} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:34:10.553746 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-d52c5ff8-58fb-42b5-8edd-340c2c53dd6a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '301ed911116b4c22a3dd86f24a77bdfd'} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:34:10.559473 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 1378/2748] 10.222.0.18 () {58 vars in 1224 bytes} [Tue Sep 13 03:34:10 2022] DELETE /identity/v3/users/301ed911116b4c22a3dd86f24a77bdfd => generated 0 bytes in 147 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 03:34:10.564155 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac0bb34e-8550-459c-9ad7-e6282d30c222 None None] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:34:10.564458 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac0bb34e-8550-459c-9ad7-e6282d30c222 None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:34:10.564732 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac0bb34e-8550-459c-9ad7-e6282d30c222 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:34:10.583106 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.auth.core [None req-ac0bb34e-8550-459c-9ad7-e6282d30c222 None None] MFA Rules not processed for user `49f280ab086b4610855c4ffa378517d9`. Rule list: `[]` (Enabled: `True`). {{(pid=81400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 03:34:10.584286 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-ac0bb34e-8550-459c-9ad7-e6282d30c222 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:34:10.642731 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 1371/2749] 10.222.0.18 () {58 vars in 958 bytes} [Tue Sep 13 03:34:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 13 03:34:10.652170 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d52c5ff8-58fb-42b5-8edd-340c2c53dd6a admin admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:34:10.653558 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-d52c5ff8-58fb-42b5-8edd-340c2c53dd6a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:34:10.709425 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d52c5ff8-58fb-42b5-8edd-340c2c53dd6a admin admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:34:10.713197 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a4143e2-86b7-4117-bfd2-30efe4d0e01e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdffbc851086441ebd01022d1e4c2609', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b9d94f6a68f7470d9494f112175620dc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:34:10.713798 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a4143e2-86b7-4117-bfd2-30efe4d0e01e service neutron] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:34:10.714077 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a4143e2-86b7-4117-bfd2-30efe4d0e01e service neutron] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:34:10.714346 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a4143e2-86b7-4117-bfd2-30efe4d0e01e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:34:10.723330 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a4143e2-86b7-4117-bfd2-30efe4d0e01e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:34:10.725272 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a4143e2-86b7-4117-bfd2-30efe4d0e01e service neutron] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:34:10.734589 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 1379/2750] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Sep 13 03:34:10 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 03:34:10.996287 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac0bb34e-8550-459c-9ad7-e6282d30c222 None None] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:34:10.997750 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-ac0bb34e-8550-459c-9ad7-e6282d30c222 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:34:11.009923 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac0bb34e-8550-459c-9ad7-e6282d30c222 None None] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:34:11.014645 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da579c88-2256-4976-8efb-19d9dcebe926 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76c5181fc7be45279ac00017bb4640b3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:34:11.015225 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da579c88-2256-4976-8efb-19d9dcebe926 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:34:11.015497 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da579c88-2256-4976-8efb-19d9dcebe926 admin admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:34:11.015759 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da579c88-2256-4976-8efb-19d9dcebe926 admin admin] PATH_INFO: `/v3/projects/33fc66045c5442b5a2b1e8ad5d41fcaf` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:34:11.017446 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da579c88-2256-4976-8efb-19d9dcebe926 admin admin] RBAC: Authorizing `identity:delete_project(project_id=33fc66045c5442b5a2b1e8ad5d41fcaf)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:34:11.019787 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da579c88-2256-4976-8efb-19d9dcebe926 admin admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:34:11.072310 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-da579c88-2256-4976-8efb-19d9dcebe926 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '33fc66045c5442b5a2b1e8ad5d41fcaf', 'request_id': 'req-da579c88-2256-4976-8efb-19d9dcebe926'} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:34:11.077628 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-da579c88-2256-4976-8efb-19d9dcebe926 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '33fc66045c5442b5a2b1e8ad5d41fcaf', 'request_id': 'req-da579c88-2256-4976-8efb-19d9dcebe926'} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:34:11.084243 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-da579c88-2256-4976-8efb-19d9dcebe926 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '33fc66045c5442b5a2b1e8ad5d41fcaf', 'request_id': 'req-da579c88-2256-4976-8efb-19d9dcebe926'} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:34:11.092320 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=90,keystone:INSERT=3,keystone:DELETE=7 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:34:11.097275 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-da579c88-2256-4976-8efb-19d9dcebe926 admin admin] The token cache is being invalidate because project 33fc66045c5442b5a2b1e8ad5d41fcaf was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81400) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 03:34:11.097563 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-da579c88-2256-4976-8efb-19d9dcebe926 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:34:11.098470 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-da579c88-2256-4976-8efb-19d9dcebe926 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:34:11.100888 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 1372/2751] 10.222.0.18 () {58 vars in 1233 bytes} [Tue Sep 13 03:34:10 2022] DELETE /identity/v3/projects/33fc66045c5442b5a2b1e8ad5d41fcaf => generated 0 bytes in 105 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 03:34:11.104824 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a4143e2-86b7-4117-bfd2-30efe4d0e01e service neutron] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:34:11.120206 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a4143e2-86b7-4117-bfd2-30efe4d0e01e service neutron] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:34:11.175619 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7985906f-8380-4c6c-93fd-bc2c1733645c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76c5181fc7be45279ac00017bb4640b3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:34:11.176268 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7985906f-8380-4c6c-93fd-bc2c1733645c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:34:11.176546 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7985906f-8380-4c6c-93fd-bc2c1733645c admin admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:34:11.176817 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7985906f-8380-4c6c-93fd-bc2c1733645c admin admin] PATH_INFO: `/v3/users/06827865ef86474d93e6f33ae232691c` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:34:11.177922 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7985906f-8380-4c6c-93fd-bc2c1733645c admin admin] RBAC: Authorizing `identity:delete_user(user_id=06827865ef86474d93e6f33ae232691c)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:34:11.179861 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7985906f-8380-4c6c-93fd-bc2c1733645c admin admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:34:11.213975 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=84,keystone:DELETE=15,keystone:INSERT=1 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:34:11.229335 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-7985906f-8380-4c6c-93fd-bc2c1733645c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '06827865ef86474d93e6f33ae232691c'} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:34:11.237343 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-7985906f-8380-4c6c-93fd-bc2c1733645c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '06827865ef86474d93e6f33ae232691c'} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:34:11.249286 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-7985906f-8380-4c6c-93fd-bc2c1733645c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '06827865ef86474d93e6f33ae232691c'} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:34:11.249862 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-7985906f-8380-4c6c-93fd-bc2c1733645c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '06827865ef86474d93e6f33ae232691c'} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:34:11.257545 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-7985906f-8380-4c6c-93fd-bc2c1733645c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '06827865ef86474d93e6f33ae232691c'} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:34:11.263471 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 1380/2752] 10.222.0.18 () {58 vars in 1224 bytes} [Tue Sep 13 03:34:11 2022] DELETE /identity/v3/users/06827865ef86474d93e6f33ae232691c => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 03:34:11.530570 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da579c88-2256-4976-8efb-19d9dcebe926 admin admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:34:11.532103 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-da579c88-2256-4976-8efb-19d9dcebe926 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:34:11.546407 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da579c88-2256-4976-8efb-19d9dcebe926 admin admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:34:11.603547 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eb64331-d891-4527-a486-4205265d1b17 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76c5181fc7be45279ac00017bb4640b3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:34:11.604178 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb64331-d891-4527-a486-4205265d1b17 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:34:11.604454 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb64331-d891-4527-a486-4205265d1b17 admin admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:34:11.604725 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb64331-d891-4527-a486-4205265d1b17 admin admin] PATH_INFO: `/v3/projects/464a003c947c445c8092c1bdd20050c0` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:34:11.605860 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eb64331-d891-4527-a486-4205265d1b17 admin admin] RBAC: Authorizing `identity:delete_project(project_id=464a003c947c445c8092c1bdd20050c0)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:34:11.607799 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eb64331-d891-4527-a486-4205265d1b17 admin admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:34:11.653035 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-2eb64331-d891-4527-a486-4205265d1b17 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '464a003c947c445c8092c1bdd20050c0', 'request_id': 'req-2eb64331-d891-4527-a486-4205265d1b17'} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:34:11.658439 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-2eb64331-d891-4527-a486-4205265d1b17 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '464a003c947c445c8092c1bdd20050c0', 'request_id': 'req-2eb64331-d891-4527-a486-4205265d1b17'} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:34:11.664828 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-2eb64331-d891-4527-a486-4205265d1b17 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '464a003c947c445c8092c1bdd20050c0', 'request_id': 'req-2eb64331-d891-4527-a486-4205265d1b17'} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:34:11.677820 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-2eb64331-d891-4527-a486-4205265d1b17 admin admin] The token cache is being invalidate because project 464a003c947c445c8092c1bdd20050c0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81400) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 03:34:11.678105 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-2eb64331-d891-4527-a486-4205265d1b17 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:34:11.678750 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-2eb64331-d891-4527-a486-4205265d1b17 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:34:11.680260 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 1373/2753] 10.222.0.18 () {58 vars in 1233 bytes} [Tue Sep 13 03:34:11 2022] DELETE /identity/v3/projects/464a003c947c445c8092c1bdd20050c0 => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 03:34:11.685155 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96704e47-1630-4644-99eb-d9cac612ce16 None None] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:34:11.685656 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96704e47-1630-4644-99eb-d9cac612ce16 None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:34:11.685764 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96704e47-1630-4644-99eb-d9cac612ce16 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:34:11.705650 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.auth.core [None req-96704e47-1630-4644-99eb-d9cac612ce16 None None] MFA Rules not processed for user `49f280ab086b4610855c4ffa378517d9`. Rule list: `[]` (Enabled: `True`). {{(pid=81399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 03:34:11.706877 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-96704e47-1630-4644-99eb-d9cac612ce16 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:34:11.766222 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 1381/2754] 10.222.0.18 () {58 vars in 958 bytes} [Tue Sep 13 03:34:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 03:34:11.774090 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0354e598-eec9-4c4c-9e4d-8b1590061ed8 None None] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:34:11.774318 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0354e598-eec9-4c4c-9e4d-8b1590061ed8 None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:34:11.774592 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0354e598-eec9-4c4c-9e4d-8b1590061ed8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:34:11.794166 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.auth.core [None req-0354e598-eec9-4c4c-9e4d-8b1590061ed8 None None] MFA Rules not processed for user `49f280ab086b4610855c4ffa378517d9`. Rule list: `[]` (Enabled: `True`). {{(pid=81400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 03:34:11.795376 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-0354e598-eec9-4c4c-9e4d-8b1590061ed8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:34:11.803691 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 1374/2755] 10.222.0.18 () {58 vars in 958 bytes} [Tue Sep 13 03:34:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 03:34:11.807784 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96704e47-1630-4644-99eb-d9cac612ce16 None None] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:34:11.817992 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96704e47-1630-4644-99eb-d9cac612ce16 None None] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:34:11.822540 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d409da59-9e01-4bb8-913c-572ad2a3b260 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76c5181fc7be45279ac00017bb4640b3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:34:11.823165 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d409da59-9e01-4bb8-913c-572ad2a3b260 admin admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:34:11.823443 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d409da59-9e01-4bb8-913c-572ad2a3b260 admin admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:34:11.823716 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d409da59-9e01-4bb8-913c-572ad2a3b260 admin admin] PATH_INFO: `/v3/domains` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:34:11.824355 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d409da59-9e01-4bb8-913c-572ad2a3b260 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 03:34:11.824658 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d409da59-9e01-4bb8-913c-572ad2a3b260 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:34:11.826247 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d409da59-9e01-4bb8-913c-572ad2a3b260 admin admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:34:11.837684 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 1382/2756] 10.222.0.18 () {58 vars in 1153 bytes} [Tue Sep 13 03:34:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 03:34:11.841768 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0354e598-eec9-4c4c-9e4d-8b1590061ed8 None None] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:34:11.851014 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0354e598-eec9-4c4c-9e4d-8b1590061ed8 None None] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:34:11.855805 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50df3920-01c4-4aaa-9c5e-6b26b5d96fa3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76c5181fc7be45279ac00017bb4640b3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:34:11.856391 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50df3920-01c4-4aaa-9c5e-6b26b5d96fa3 admin admin] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:34:11.856664 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50df3920-01c4-4aaa-9c5e-6b26b5d96fa3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:34:11.856958 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50df3920-01c4-4aaa-9c5e-6b26b5d96fa3 admin admin] PATH_INFO: `/v3/projects` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:34:11.857604 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50df3920-01c4-4aaa-9c5e-6b26b5d96fa3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:34:11.859532 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50df3920-01c4-4aaa-9c5e-6b26b5d96fa3 admin admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:34:11.886865 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 1375/2757] 10.222.0.18 () {60 vars in 1153 bytes} [Tue Sep 13 03:34:11 2022] POST /identity/v3/projects => generated 377 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 03:34:11.891065 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d409da59-9e01-4bb8-913c-572ad2a3b260 admin admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:34:11.901213 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d409da59-9e01-4bb8-913c-572ad2a3b260 admin admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:34:11.955994 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26a4581f-3565-4624-bea8-8c37e69a6638 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76c5181fc7be45279ac00017bb4640b3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:34:11.956588 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26a4581f-3565-4624-bea8-8c37e69a6638 admin admin] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:34:11.956864 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26a4581f-3565-4624-bea8-8c37e69a6638 admin admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:34:11.957132 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26a4581f-3565-4624-bea8-8c37e69a6638 admin admin] PATH_INFO: `/v3/users` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:34:11.957761 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26a4581f-3565-4624-bea8-8c37e69a6638 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:34:11.959731 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26a4581f-3565-4624-bea8-8c37e69a6638 admin admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:34:11.981552 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 1383/2758] 10.222.0.18 () {60 vars in 1144 bytes} [Tue Sep 13 03:34:11 2022] POST /identity/v3/users => generated 289 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 13 03:34:11.985601 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50df3920-01c4-4aaa-9c5e-6b26b5d96fa3 admin admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:34:11.996977 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50df3920-01c4-4aaa-9c5e-6b26b5d96fa3 admin admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:34:12.001662 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e9e6335-5acc-4ee7-90c1-9bbe1441b0f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76c5181fc7be45279ac00017bb4640b3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:34:12.002282 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e9e6335-5acc-4ee7-90c1-9bbe1441b0f9 admin admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:34:12.002560 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e9e6335-5acc-4ee7-90c1-9bbe1441b0f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:34:12.002858 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e9e6335-5acc-4ee7-90c1-9bbe1441b0f9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:34:12.003493 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e9e6335-5acc-4ee7-90c1-9bbe1441b0f9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:34:12.005112 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e9e6335-5acc-4ee7-90c1-9bbe1441b0f9 admin admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:34:12.015451 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 1376/2759] 10.222.0.18 () {58 vars in 1122 bytes} [Tue Sep 13 03:34:11 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 03:34:12.019223 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26a4581f-3565-4624-bea8-8c37e69a6638 admin admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:34:12.029177 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26a4581f-3565-4624-bea8-8c37e69a6638 admin admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:34:12.033682 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb96de80-62e4-48e5-aae7-f27d52d6fe76 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76c5181fc7be45279ac00017bb4640b3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:34:12.034277 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb96de80-62e4-48e5-aae7-f27d52d6fe76 admin admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:34:12.034559 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb96de80-62e4-48e5-aae7-f27d52d6fe76 admin admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:34:12.034868 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb96de80-62e4-48e5-aae7-f27d52d6fe76 admin admin] PATH_INFO: `/v3/roles` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:34:12.035501 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb96de80-62e4-48e5-aae7-f27d52d6fe76 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:34:12.037618 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb96de80-62e4-48e5-aae7-f27d52d6fe76 admin admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:34:12.048373 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 1384/2760] 10.222.0.18 () {58 vars in 1122 bytes} [Tue Sep 13 03:34:12 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 03:34:12.052914 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e9e6335-5acc-4ee7-90c1-9bbe1441b0f9 admin admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:34:12.063240 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e9e6335-5acc-4ee7-90c1-9bbe1441b0f9 admin admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:34:12.067939 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baf7c402-afe4-4884-a63f-533ca43f79f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76c5181fc7be45279ac00017bb4640b3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:34:12.068590 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf7c402-afe4-4884-a63f-533ca43f79f7 admin admin] REQUEST_METHOD: `PUT` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:34:12.068890 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf7c402-afe4-4884-a63f-533ca43f79f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:34:12.069136 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf7c402-afe4-4884-a63f-533ca43f79f7 admin admin] PATH_INFO: `/v3/projects/a9911602e4994d9ebb78a04c162f6b45/users/2a440469f051496089c827052b2e38de/roles/1eb7a5b5203945c29e6d22d653f47847` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:34:12.084066 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baf7c402-afe4-4884-a63f-533ca43f79f7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a9911602e4994d9ebb78a04c162f6b45, user_id=2a440469f051496089c827052b2e38de, role_id=1eb7a5b5203945c29e6d22d653f47847)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:34:12.086752 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baf7c402-afe4-4884-a63f-533ca43f79f7 admin admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:34:12.100412 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 1377/2761] 10.222.0.18 () {60 vars in 1483 bytes} [Tue Sep 13 03:34:12 2022] PUT /identity/v3/projects/a9911602e4994d9ebb78a04c162f6b45/users/2a440469f051496089c827052b2e38de/roles/1eb7a5b5203945c29e6d22d653f47847 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 03:34:12.110788 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e34402b9-2392-41fd-89dc-d5ed5af7c44b None None] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:34:12.111075 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e34402b9-2392-41fd-89dc-d5ed5af7c44b None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:34:12.111344 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e34402b9-2392-41fd-89dc-d5ed5af7c44b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:34:12.136001 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=91,keystone:DELETE=5,keystone:INSERT=4 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:34:12.139108 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.auth.core [None req-e34402b9-2392-41fd-89dc-d5ed5af7c44b None None] MFA Rules not processed for user `2a440469f051496089c827052b2e38de`. Rule list: `[]` (Enabled: `True`). {{(pid=81399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 03:34:12.140940 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-e34402b9-2392-41fd-89dc-d5ed5af7c44b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:34:12.203472 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 1385/2762] 10.222.0.18 () {58 vars in 958 bytes} [Tue Sep 13 03:34:12 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 13 03:34:12.215498 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baf7c402-afe4-4884-a63f-533ca43f79f7 admin admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:34:12.217165 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-baf7c402-afe4-4884-a63f-533ca43f79f7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:34:12.242826 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:34:12.296104 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baf7c402-afe4-4884-a63f-533ca43f79f7 admin admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:34:12.299698 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e05e084-8cfc-41be-bc08-537a83b4a0ad service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b5fdc2ff46a14638b409a13e73ba5500', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b9d94f6a68f7470d9494f112175620dc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:34:12.300293 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e05e084-8cfc-41be-bc08-537a83b4a0ad service glance] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:34:12.300571 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e05e084-8cfc-41be-bc08-537a83b4a0ad service glance] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:34:12.300843 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e05e084-8cfc-41be-bc08-537a83b4a0ad service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:34:12.309048 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e05e084-8cfc-41be-bc08-537a83b4a0ad service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:34:12.310467 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e05e084-8cfc-41be-bc08-537a83b4a0ad service glance] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:34:12.319661 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 1378/2763] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Sep 13 03:34:12 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 13 03:34:22.200634 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=26 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:34:22.315202 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=24 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:34:51.924572 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e34402b9-2392-41fd-89dc-d5ed5af7c44b None None] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:34:51.925983 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-e34402b9-2392-41fd-89dc-d5ed5af7c44b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:34:51.989475 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e34402b9-2392-41fd-89dc-d5ed5af7c44b None None] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:34:51.993984 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8741d461-dc7d-4a45-b2a9-a447b18cb7c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76c5181fc7be45279ac00017bb4640b3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:34:51.994605 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8741d461-dc7d-4a45-b2a9-a447b18cb7c8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:34:51.994913 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8741d461-dc7d-4a45-b2a9-a447b18cb7c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:34:51.995180 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8741d461-dc7d-4a45-b2a9-a447b18cb7c8 admin admin] PATH_INFO: `/v3/users/2e5fa847a02c4d52ad426126018eadac` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:34:51.996290 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8741d461-dc7d-4a45-b2a9-a447b18cb7c8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2e5fa847a02c4d52ad426126018eadac)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:34:51.998196 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8741d461-dc7d-4a45-b2a9-a447b18cb7c8 admin admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:34:52.043500 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-8741d461-dc7d-4a45-b2a9-a447b18cb7c8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2e5fa847a02c4d52ad426126018eadac'} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:34:52.050522 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-8741d461-dc7d-4a45-b2a9-a447b18cb7c8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2e5fa847a02c4d52ad426126018eadac'} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:34:52.063411 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-8741d461-dc7d-4a45-b2a9-a447b18cb7c8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2e5fa847a02c4d52ad426126018eadac'} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:34:52.064083 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-8741d461-dc7d-4a45-b2a9-a447b18cb7c8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2e5fa847a02c4d52ad426126018eadac'} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:34:52.071941 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-8741d461-dc7d-4a45-b2a9-a447b18cb7c8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2e5fa847a02c4d52ad426126018eadac'} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:34:52.078350 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 1386/2764] 10.222.0.18 () {58 vars in 1224 bytes} [Tue Sep 13 03:34:51 2022] DELETE /identity/v3/users/2e5fa847a02c4d52ad426126018eadac => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 03:34:52.082942 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8df1c77-c52e-4423-8120-4364310a4a68 None None] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:34:52.083382 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8df1c77-c52e-4423-8120-4364310a4a68 None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:34:52.083502 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8df1c77-c52e-4423-8120-4364310a4a68 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:34:52.102135 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.auth.core [None req-e8df1c77-c52e-4423-8120-4364310a4a68 None None] MFA Rules not processed for user `49f280ab086b4610855c4ffa378517d9`. Rule list: `[]` (Enabled: `True`). {{(pid=81400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 03:34:52.103359 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-e8df1c77-c52e-4423-8120-4364310a4a68 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:34:52.162589 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 1379/2765] 10.222.0.18 () {58 vars in 958 bytes} [Tue Sep 13 03:34:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 03:34:52.172230 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8741d461-dc7d-4a45-b2a9-a447b18cb7c8 admin admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:34:52.173729 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-8741d461-dc7d-4a45-b2a9-a447b18cb7c8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:34:52.230457 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8741d461-dc7d-4a45-b2a9-a447b18cb7c8 admin admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:34:52.234086 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30b2ef73-fd91-480c-8244-fc511e641884 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdffbc851086441ebd01022d1e4c2609', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b9d94f6a68f7470d9494f112175620dc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:34:52.234716 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30b2ef73-fd91-480c-8244-fc511e641884 service neutron] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:34:52.234995 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30b2ef73-fd91-480c-8244-fc511e641884 service neutron] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:34:52.235259 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30b2ef73-fd91-480c-8244-fc511e641884 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:34:52.244310 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30b2ef73-fd91-480c-8244-fc511e641884 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:34:52.245728 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30b2ef73-fd91-480c-8244-fc511e641884 service neutron] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:34:52.255003 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 1387/2766] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Sep 13 03:34:52 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 03:34:52.515484 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8df1c77-c52e-4423-8120-4364310a4a68 None None] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:34:52.516895 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-e8df1c77-c52e-4423-8120-4364310a4a68 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:34:52.530110 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8df1c77-c52e-4423-8120-4364310a4a68 None None] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:34:52.535135 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8256d68b-45ab-4516-9070-3fd62fe7f88e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76c5181fc7be45279ac00017bb4640b3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:34:52.536024 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8256d68b-45ab-4516-9070-3fd62fe7f88e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:34:52.536298 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8256d68b-45ab-4516-9070-3fd62fe7f88e admin admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:34:52.536568 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8256d68b-45ab-4516-9070-3fd62fe7f88e admin admin] PATH_INFO: `/v3/projects/a645f57266034232b7bf7a20a4d2c3cd` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:34:52.537692 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8256d68b-45ab-4516-9070-3fd62fe7f88e admin admin] RBAC: Authorizing `identity:delete_project(project_id=a645f57266034232b7bf7a20a4d2c3cd)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:34:52.539762 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8256d68b-45ab-4516-9070-3fd62fe7f88e admin admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:34:52.587453 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-8256d68b-45ab-4516-9070-3fd62fe7f88e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a645f57266034232b7bf7a20a4d2c3cd', 'request_id': 'req-8256d68b-45ab-4516-9070-3fd62fe7f88e'} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:34:52.592872 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-8256d68b-45ab-4516-9070-3fd62fe7f88e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a645f57266034232b7bf7a20a4d2c3cd', 'request_id': 'req-8256d68b-45ab-4516-9070-3fd62fe7f88e'} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:34:52.598844 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-8256d68b-45ab-4516-9070-3fd62fe7f88e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a645f57266034232b7bf7a20a4d2c3cd', 'request_id': 'req-8256d68b-45ab-4516-9070-3fd62fe7f88e'} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:34:52.610729 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-8256d68b-45ab-4516-9070-3fd62fe7f88e admin admin] The token cache is being invalidate because project a645f57266034232b7bf7a20a4d2c3cd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81400) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 03:34:52.611012 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-8256d68b-45ab-4516-9070-3fd62fe7f88e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:34:52.611613 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-8256d68b-45ab-4516-9070-3fd62fe7f88e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:34:52.613081 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 1380/2767] 10.222.0.18 () {58 vars in 1233 bytes} [Tue Sep 13 03:34:52 2022] DELETE /identity/v3/projects/a645f57266034232b7bf7a20a4d2c3cd => generated 0 bytes in 98 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 03:34:52.617341 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30b2ef73-fd91-480c-8244-fc511e641884 service neutron] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:34:52.627148 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30b2ef73-fd91-480c-8244-fc511e641884 service neutron] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:34:52.646138 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:34:52.685558 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0db5ba7-6b56-4921-b90b-4076df300468 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76c5181fc7be45279ac00017bb4640b3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:34:52.686182 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0db5ba7-6b56-4921-b90b-4076df300468 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:34:52.686458 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0db5ba7-6b56-4921-b90b-4076df300468 admin admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:34:52.686830 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0db5ba7-6b56-4921-b90b-4076df300468 admin admin] PATH_INFO: `/v3/users/583ccc31a5544cbfb29df485ba79b092` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:34:52.687940 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0db5ba7-6b56-4921-b90b-4076df300468 admin admin] RBAC: Authorizing `identity:delete_user(user_id=583ccc31a5544cbfb29df485ba79b092)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:34:52.689908 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0db5ba7-6b56-4921-b90b-4076df300468 admin admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:34:52.740266 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-d0db5ba7-6b56-4921-b90b-4076df300468 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '583ccc31a5544cbfb29df485ba79b092'} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:34:52.753825 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-d0db5ba7-6b56-4921-b90b-4076df300468 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '583ccc31a5544cbfb29df485ba79b092'} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:34:52.766511 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-d0db5ba7-6b56-4921-b90b-4076df300468 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '583ccc31a5544cbfb29df485ba79b092'} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:34:52.767203 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-d0db5ba7-6b56-4921-b90b-4076df300468 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '583ccc31a5544cbfb29df485ba79b092'} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:34:52.774755 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-d0db5ba7-6b56-4921-b90b-4076df300468 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '583ccc31a5544cbfb29df485ba79b092'} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:34:52.780762 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 1388/2768] 10.222.0.18 () {58 vars in 1224 bytes} [Tue Sep 13 03:34:52 2022] DELETE /identity/v3/users/583ccc31a5544cbfb29df485ba79b092 => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 03:34:53.320911 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8256d68b-45ab-4516-9070-3fd62fe7f88e admin admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:34:53.322411 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-8256d68b-45ab-4516-9070-3fd62fe7f88e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:34:53.344529 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8256d68b-45ab-4516-9070-3fd62fe7f88e admin admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:34:53.400966 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-421a18fb-c4d2-42ea-9fa1-24c221a24e97 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76c5181fc7be45279ac00017bb4640b3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:34:53.401588 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-421a18fb-c4d2-42ea-9fa1-24c221a24e97 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:34:53.401866 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-421a18fb-c4d2-42ea-9fa1-24c221a24e97 admin admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:34:53.402132 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-421a18fb-c4d2-42ea-9fa1-24c221a24e97 admin admin] PATH_INFO: `/v3/projects/014ce25b5703425aa2df2976a58fcd7e` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:34:53.403249 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-421a18fb-c4d2-42ea-9fa1-24c221a24e97 admin admin] RBAC: Authorizing `identity:delete_project(project_id=014ce25b5703425aa2df2976a58fcd7e)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:34:53.405152 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-421a18fb-c4d2-42ea-9fa1-24c221a24e97 admin admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:34:53.435611 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:34:53.452875 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-421a18fb-c4d2-42ea-9fa1-24c221a24e97 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '014ce25b5703425aa2df2976a58fcd7e', 'request_id': 'req-421a18fb-c4d2-42ea-9fa1-24c221a24e97'} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:34:53.458431 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-421a18fb-c4d2-42ea-9fa1-24c221a24e97 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '014ce25b5703425aa2df2976a58fcd7e', 'request_id': 'req-421a18fb-c4d2-42ea-9fa1-24c221a24e97'} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:34:53.464463 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-421a18fb-c4d2-42ea-9fa1-24c221a24e97 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '014ce25b5703425aa2df2976a58fcd7e', 'request_id': 'req-421a18fb-c4d2-42ea-9fa1-24c221a24e97'} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:34:53.476959 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-421a18fb-c4d2-42ea-9fa1-24c221a24e97 admin admin] The token cache is being invalidate because project 014ce25b5703425aa2df2976a58fcd7e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81400) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 03:34:53.477241 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-421a18fb-c4d2-42ea-9fa1-24c221a24e97 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:34:53.477858 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-421a18fb-c4d2-42ea-9fa1-24c221a24e97 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:34:53.479343 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 1381/2769] 10.222.0.18 () {58 vars in 1233 bytes} [Tue Sep 13 03:34:53 2022] DELETE /identity/v3/projects/014ce25b5703425aa2df2976a58fcd7e => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 03:34:53.484281 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-288e297b-ea93-4e47-9fa9-e95bc636aaa4 None None] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:34:53.484761 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-288e297b-ea93-4e47-9fa9-e95bc636aaa4 None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:34:53.484889 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-288e297b-ea93-4e47-9fa9-e95bc636aaa4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:34:53.504674 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.auth.core [None req-288e297b-ea93-4e47-9fa9-e95bc636aaa4 None None] MFA Rules not processed for user `49f280ab086b4610855c4ffa378517d9`. Rule list: `[]` (Enabled: `True`). {{(pid=81399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 03:34:53.505859 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-288e297b-ea93-4e47-9fa9-e95bc636aaa4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:34:53.568190 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 1389/2770] 10.222.0.18 () {58 vars in 958 bytes} [Tue Sep 13 03:34:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 03:34:53.575719 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e9b6016-829e-406c-9fe2-ecc7b563c116 None None] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:34:53.576010 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e9b6016-829e-406c-9fe2-ecc7b563c116 None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:34:53.576270 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e9b6016-829e-406c-9fe2-ecc7b563c116 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:34:53.596031 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.auth.core [None req-2e9b6016-829e-406c-9fe2-ecc7b563c116 None None] MFA Rules not processed for user `49f280ab086b4610855c4ffa378517d9`. Rule list: `[]` (Enabled: `True`). {{(pid=81400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 03:34:53.597249 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-2e9b6016-829e-406c-9fe2-ecc7b563c116 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:34:53.605688 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 1382/2771] 10.222.0.18 () {58 vars in 958 bytes} [Tue Sep 13 03:34:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 03:34:53.610051 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-288e297b-ea93-4e47-9fa9-e95bc636aaa4 None None] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:34:53.620076 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-288e297b-ea93-4e47-9fa9-e95bc636aaa4 None None] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:34:53.624631 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05536deb-e052-4345-a627-34006dde899f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76c5181fc7be45279ac00017bb4640b3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:34:53.625221 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05536deb-e052-4345-a627-34006dde899f admin admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:34:53.625497 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05536deb-e052-4345-a627-34006dde899f admin admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:34:53.625768 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05536deb-e052-4345-a627-34006dde899f admin admin] PATH_INFO: `/v3/domains` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:34:53.626400 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05536deb-e052-4345-a627-34006dde899f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 03:34:53.626728 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05536deb-e052-4345-a627-34006dde899f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:34:53.628324 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05536deb-e052-4345-a627-34006dde899f admin admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:34:53.640151 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 1390/2772] 10.222.0.18 () {58 vars in 1153 bytes} [Tue Sep 13 03:34:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 03:34:53.644413 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e9b6016-829e-406c-9fe2-ecc7b563c116 None None] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:34:53.654157 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e9b6016-829e-406c-9fe2-ecc7b563c116 None None] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:34:53.659149 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61277ac4-f81d-4d04-b414-677284da9600 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76c5181fc7be45279ac00017bb4640b3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:34:53.659832 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61277ac4-f81d-4d04-b414-677284da9600 admin admin] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:34:53.660115 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61277ac4-f81d-4d04-b414-677284da9600 admin admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:34:53.660386 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61277ac4-f81d-4d04-b414-677284da9600 admin admin] PATH_INFO: `/v3/projects` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:34:53.662033 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61277ac4-f81d-4d04-b414-677284da9600 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:34:53.664030 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61277ac4-f81d-4d04-b414-677284da9600 admin admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:34:53.692720 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 1383/2773] 10.222.0.18 () {60 vars in 1153 bytes} [Tue Sep 13 03:34:53 2022] POST /identity/v3/projects => generated 385 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 03:34:53.696903 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05536deb-e052-4345-a627-34006dde899f admin admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:34:53.706802 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05536deb-e052-4345-a627-34006dde899f admin admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:34:53.734710 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:34:53.767206 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f08aabc-ad76-413c-a4e2-2f62213e39c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76c5181fc7be45279ac00017bb4640b3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:34:53.767821 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f08aabc-ad76-413c-a4e2-2f62213e39c8 admin admin] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:34:53.768105 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f08aabc-ad76-413c-a4e2-2f62213e39c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:34:53.768382 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f08aabc-ad76-413c-a4e2-2f62213e39c8 admin admin] PATH_INFO: `/v3/users` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:34:53.769023 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f08aabc-ad76-413c-a4e2-2f62213e39c8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:34:53.771499 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f08aabc-ad76-413c-a4e2-2f62213e39c8 admin admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:34:53.793821 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 1391/2774] 10.222.0.18 () {60 vars in 1144 bytes} [Tue Sep 13 03:34:53 2022] POST /identity/v3/users => generated 293 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 13 03:34:53.797938 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61277ac4-f81d-4d04-b414-677284da9600 admin admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:34:53.808395 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61277ac4-f81d-4d04-b414-677284da9600 admin admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:34:53.813026 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f81efb72-f8a5-4916-b82f-c058e3e7b4e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76c5181fc7be45279ac00017bb4640b3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:34:53.813660 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f81efb72-f8a5-4916-b82f-c058e3e7b4e9 admin admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:34:53.813945 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f81efb72-f8a5-4916-b82f-c058e3e7b4e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:34:53.814215 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f81efb72-f8a5-4916-b82f-c058e3e7b4e9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:34:53.814880 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f81efb72-f8a5-4916-b82f-c058e3e7b4e9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:34:53.816863 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f81efb72-f8a5-4916-b82f-c058e3e7b4e9 admin admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:34:53.827423 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 1384/2775] 10.222.0.18 () {58 vars in 1122 bytes} [Tue Sep 13 03:34:53 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 03:34:53.831221 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f08aabc-ad76-413c-a4e2-2f62213e39c8 admin admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:34:53.842147 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f08aabc-ad76-413c-a4e2-2f62213e39c8 admin admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:34:53.847182 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5319bdd-eaf0-46f3-b554-7e44920ffeb1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76c5181fc7be45279ac00017bb4640b3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:34:53.847817 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5319bdd-eaf0-46f3-b554-7e44920ffeb1 admin admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:34:53.848715 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5319bdd-eaf0-46f3-b554-7e44920ffeb1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:34:53.849006 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5319bdd-eaf0-46f3-b554-7e44920ffeb1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:34:53.849647 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5319bdd-eaf0-46f3-b554-7e44920ffeb1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:34:53.851260 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5319bdd-eaf0-46f3-b554-7e44920ffeb1 admin admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:34:53.862220 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 1392/2776] 10.222.0.18 () {58 vars in 1122 bytes} [Tue Sep 13 03:34:53 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 03:34:53.866777 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f81efb72-f8a5-4916-b82f-c058e3e7b4e9 admin admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:34:53.877533 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f81efb72-f8a5-4916-b82f-c058e3e7b4e9 admin admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:34:53.882464 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5e5b294-f5dc-4d90-98b3-595527a891f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76c5181fc7be45279ac00017bb4640b3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:34:53.883152 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e5b294-f5dc-4d90-98b3-595527a891f8 admin admin] REQUEST_METHOD: `PUT` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:34:53.883409 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e5b294-f5dc-4d90-98b3-595527a891f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:34:53.883752 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e5b294-f5dc-4d90-98b3-595527a891f8 admin admin] PATH_INFO: `/v3/projects/418f2e09bdcd40318cc106e7d98007fc/users/f1c475bfc2b44625bd82fd9a1dac0a8b/roles/1eb7a5b5203945c29e6d22d653f47847` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:34:53.901486 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5e5b294-f5dc-4d90-98b3-595527a891f8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=418f2e09bdcd40318cc106e7d98007fc, user_id=f1c475bfc2b44625bd82fd9a1dac0a8b, role_id=1eb7a5b5203945c29e6d22d653f47847)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:34:53.904365 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5e5b294-f5dc-4d90-98b3-595527a891f8 admin admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:34:53.918741 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 1385/2777] 10.222.0.18 () {60 vars in 1483 bytes} [Tue Sep 13 03:34:53 2022] PUT /identity/v3/projects/418f2e09bdcd40318cc106e7d98007fc/users/f1c475bfc2b44625bd82fd9a1dac0a8b/roles/1eb7a5b5203945c29e6d22d653f47847 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 03:34:53.932420 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-527d1708-e4c0-4962-81a7-bba134323a80 None None] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:34:53.932704 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-527d1708-e4c0-4962-81a7-bba134323a80 None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:34:53.932973 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-527d1708-e4c0-4962-81a7-bba134323a80 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:34:53.961568 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.auth.core [None req-527d1708-e4c0-4962-81a7-bba134323a80 None None] MFA Rules not processed for user `f1c475bfc2b44625bd82fd9a1dac0a8b`. Rule list: `[]` (Enabled: `True`). {{(pid=81399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 03:34:53.962828 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-527d1708-e4c0-4962-81a7-bba134323a80 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:34:54.029146 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 1393/2778] 10.222.0.18 () {58 vars in 958 bytes} [Tue Sep 13 03:34:53 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 03:34:54.033759 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5e5b294-f5dc-4d90-98b3-595527a891f8 admin admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:34:54.099762 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5e5b294-f5dc-4d90-98b3-595527a891f8 admin admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:34:54.104949 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6426bf9b-7db1-47f3-91e8-31c09e6f747d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76c5181fc7be45279ac00017bb4640b3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:34:54.105575 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6426bf9b-7db1-47f3-91e8-31c09e6f747d admin admin] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:34:54.105854 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6426bf9b-7db1-47f3-91e8-31c09e6f747d admin admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:34:54.106125 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6426bf9b-7db1-47f3-91e8-31c09e6f747d admin admin] PATH_INFO: `/v3/projects` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:34:54.106795 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6426bf9b-7db1-47f3-91e8-31c09e6f747d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:34:54.108717 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6426bf9b-7db1-47f3-91e8-31c09e6f747d admin admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:34:54.137810 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 1386/2779] 10.222.0.18 () {60 vars in 1153 bytes} [Tue Sep 13 03:34:54 2022] POST /identity/v3/projects => generated 387 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 03:34:54.141957 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-527d1708-e4c0-4962-81a7-bba134323a80 None None] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:34:54.154322 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-527d1708-e4c0-4962-81a7-bba134323a80 None None] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:34:54.210048 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7ce2597-5ef4-497e-99a6-e252e6de460c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76c5181fc7be45279ac00017bb4640b3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:34:54.210689 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ce2597-5ef4-497e-99a6-e252e6de460c admin admin] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:34:54.210966 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ce2597-5ef4-497e-99a6-e252e6de460c admin admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:34:54.211235 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ce2597-5ef4-497e-99a6-e252e6de460c admin admin] PATH_INFO: `/v3/users` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:34:54.211862 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7ce2597-5ef4-497e-99a6-e252e6de460c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:34:54.213763 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7ce2597-5ef4-497e-99a6-e252e6de460c admin admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:34:54.234376 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 1394/2780] 10.222.0.18 () {60 vars in 1144 bytes} [Tue Sep 13 03:34:54 2022] POST /identity/v3/users => generated 300 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 03:34:54.238459 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6426bf9b-7db1-47f3-91e8-31c09e6f747d admin admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:34:54.248618 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6426bf9b-7db1-47f3-91e8-31c09e6f747d admin admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:34:54.253164 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aabdbd9c-049a-4bf2-8e0a-f028e778538a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76c5181fc7be45279ac00017bb4640b3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:34:54.253735 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aabdbd9c-049a-4bf2-8e0a-f028e778538a admin admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:34:54.254010 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aabdbd9c-049a-4bf2-8e0a-f028e778538a admin admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:34:54.254276 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aabdbd9c-049a-4bf2-8e0a-f028e778538a admin admin] PATH_INFO: `/v3/roles` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:34:54.255460 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aabdbd9c-049a-4bf2-8e0a-f028e778538a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:34:54.257041 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aabdbd9c-049a-4bf2-8e0a-f028e778538a admin admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:34:54.267270 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 1387/2781] 10.222.0.18 () {58 vars in 1122 bytes} [Tue Sep 13 03:34:54 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 03:34:54.271680 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7ce2597-5ef4-497e-99a6-e252e6de460c admin admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:34:54.281216 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7ce2597-5ef4-497e-99a6-e252e6de460c admin admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:34:54.285996 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35adde30-af1e-4159-9eae-3f57909c8583 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76c5181fc7be45279ac00017bb4640b3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:34:54.286653 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35adde30-af1e-4159-9eae-3f57909c8583 admin admin] REQUEST_METHOD: `PUT` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:34:54.286943 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35adde30-af1e-4159-9eae-3f57909c8583 admin admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:34:54.287211 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35adde30-af1e-4159-9eae-3f57909c8583 admin admin] PATH_INFO: `/v3/projects/0a5f6227c54049b8b75b0a7c06d84e68/users/2cb1f7fbb923481bb8182dc7662beb45/roles/425d4b481c7846e0bc07e7e805d5b15f` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:34:54.301814 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35adde30-af1e-4159-9eae-3f57909c8583 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0a5f6227c54049b8b75b0a7c06d84e68, user_id=2cb1f7fbb923481bb8182dc7662beb45, role_id=425d4b481c7846e0bc07e7e805d5b15f)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:34:54.304476 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35adde30-af1e-4159-9eae-3f57909c8583 admin admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:34:54.316773 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 1395/2782] 10.222.0.18 () {60 vars in 1483 bytes} [Tue Sep 13 03:34:54 2022] PUT /identity/v3/projects/0a5f6227c54049b8b75b0a7c06d84e68/users/2cb1f7fbb923481bb8182dc7662beb45/roles/425d4b481c7846e0bc07e7e805d5b15f => generated 0 bytes in 45 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 03:34:54.321250 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aabdbd9c-049a-4bf2-8e0a-f028e778538a admin admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:34:54.330943 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aabdbd9c-049a-4bf2-8e0a-f028e778538a admin admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:34:54.351690 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:DELETE=6,keystone:SELECT=90,keystone:INSERT=4 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:34:54.392674 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dedab871-1abc-41aa-a186-0fd4976460c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76c5181fc7be45279ac00017bb4640b3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:34:54.393276 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dedab871-1abc-41aa-a186-0fd4976460c0 admin admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:34:54.393551 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dedab871-1abc-41aa-a186-0fd4976460c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:34:54.393821 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dedab871-1abc-41aa-a186-0fd4976460c0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:34:54.394455 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dedab871-1abc-41aa-a186-0fd4976460c0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:34:54.396093 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dedab871-1abc-41aa-a186-0fd4976460c0 admin admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:34:54.406151 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 1388/2783] 10.222.0.18 () {58 vars in 1122 bytes} [Tue Sep 13 03:34:54 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 13 03:34:54.410496 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35adde30-af1e-4159-9eae-3f57909c8583 admin admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:34:54.417239 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:34:54.423727 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35adde30-af1e-4159-9eae-3f57909c8583 admin admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:34:54.429091 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bb15c48-5fb3-4d7a-b5ce-6d15387b442b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76c5181fc7be45279ac00017bb4640b3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:34:54.429742 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb15c48-5fb3-4d7a-b5ce-6d15387b442b admin admin] REQUEST_METHOD: `PUT` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:34:54.430022 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb15c48-5fb3-4d7a-b5ce-6d15387b442b admin admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:34:54.430414 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb15c48-5fb3-4d7a-b5ce-6d15387b442b admin admin] PATH_INFO: `/v3/projects/0a5f6227c54049b8b75b0a7c06d84e68/users/2cb1f7fbb923481bb8182dc7662beb45/roles/425d4b481c7846e0bc07e7e805d5b15f` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:34:54.432539 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bb15c48-5fb3-4d7a-b5ce-6d15387b442b admin admin] RBAC: Authorizing `identity:create_grant(project_id=0a5f6227c54049b8b75b0a7c06d84e68, user_id=2cb1f7fbb923481bb8182dc7662beb45, role_id=425d4b481c7846e0bc07e7e805d5b15f)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:34:54.435226 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bb15c48-5fb3-4d7a-b5ce-6d15387b442b admin admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:34:54.447448 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 1396/2784] 10.222.0.18 () {60 vars in 1483 bytes} [Tue Sep 13 03:34:54 2022] PUT /identity/v3/projects/0a5f6227c54049b8b75b0a7c06d84e68/users/2cb1f7fbb923481bb8182dc7662beb45/roles/425d4b481c7846e0bc07e7e805d5b15f => generated 0 bytes in 37 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 03:34:54.512750 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64585e3d-7a64-4c59-8eba-c1fc4636d20e None None] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:34:54.513031 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64585e3d-7a64-4c59-8eba-c1fc4636d20e None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:34:54.513306 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64585e3d-7a64-4c59-8eba-c1fc4636d20e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:34:54.541360 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.auth.core [None req-64585e3d-7a64-4c59-8eba-c1fc4636d20e None None] MFA Rules not processed for user `2cb1f7fbb923481bb8182dc7662beb45`. Rule list: `[]` (Enabled: `True`). {{(pid=81400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 03:34:54.542554 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-64585e3d-7a64-4c59-8eba-c1fc4636d20e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:34:54.608441 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 1389/2785] 10.222.0.18 () {58 vars in 958 bytes} [Tue Sep 13 03:34:54 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 03:34:54.620001 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bb15c48-5fb3-4d7a-b5ce-6d15387b442b admin admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:34:54.621740 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-2bb15c48-5fb3-4d7a-b5ce-6d15387b442b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:34:54.678880 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bb15c48-5fb3-4d7a-b5ce-6d15387b442b admin admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:34:54.682473 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a2b9adf-73f3-46fb-bf66-d0544514522f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '924ee3f608d34a18b0ddc4695f102a56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b9d94f6a68f7470d9494f112175620dc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:34:54.683097 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a2b9adf-73f3-46fb-bf66-d0544514522f service cinder] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:34:54.683374 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a2b9adf-73f3-46fb-bf66-d0544514522f service cinder] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:34:54.683641 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a2b9adf-73f3-46fb-bf66-d0544514522f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:34:54.692344 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a2b9adf-73f3-46fb-bf66-d0544514522f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:34:54.693766 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a2b9adf-73f3-46fb-bf66-d0544514522f service cinder] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:34:54.703113 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 1397/2786] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Sep 13 03:34:54 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 03:34:54.731618 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64585e3d-7a64-4c59-8eba-c1fc4636d20e None None] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:34:54.792947 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64585e3d-7a64-4c59-8eba-c1fc4636d20e None None] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:34:54.797402 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-054f07ae-1885-4b7d-8499-7d84678d4edf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76c5181fc7be45279ac00017bb4640b3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:34:54.798008 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-054f07ae-1885-4b7d-8499-7d84678d4edf admin admin] REQUEST_METHOD: `DELETE` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:34:54.798278 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-054f07ae-1885-4b7d-8499-7d84678d4edf admin admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:34:54.798543 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-054f07ae-1885-4b7d-8499-7d84678d4edf admin admin] PATH_INFO: `/v3/users/f1c475bfc2b44625bd82fd9a1dac0a8b` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:34:54.799680 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-054f07ae-1885-4b7d-8499-7d84678d4edf admin admin] RBAC: Authorizing `identity:delete_user(user_id=f1c475bfc2b44625bd82fd9a1dac0a8b)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:34:54.801582 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-054f07ae-1885-4b7d-8499-7d84678d4edf admin admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:34:54.845865 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-054f07ae-1885-4b7d-8499-7d84678d4edf admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f1c475bfc2b44625bd82fd9a1dac0a8b'} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:34:54.847634 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=93,keystone:DELETE=7 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:34:54.854239 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-054f07ae-1885-4b7d-8499-7d84678d4edf admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f1c475bfc2b44625bd82fd9a1dac0a8b'} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:34:54.855185 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-054f07ae-1885-4b7d-8499-7d84678d4edf admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f1c475bfc2b44625bd82fd9a1dac0a8b'} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:34:54.869640 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-054f07ae-1885-4b7d-8499-7d84678d4edf admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f1c475bfc2b44625bd82fd9a1dac0a8b'} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:34:54.876382 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-054f07ae-1885-4b7d-8499-7d84678d4edf admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f1c475bfc2b44625bd82fd9a1dac0a8b'} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:34:54.882680 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 1390/2787] 10.222.0.18 () {58 vars in 1224 bytes} [Tue Sep 13 03:34:54 2022] DELETE /identity/v3/users/f1c475bfc2b44625bd82fd9a1dac0a8b => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 03:34:54.887080 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-340d04df-4c35-414a-9f99-0f251445df86 None None] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:34:54.887368 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-340d04df-4c35-414a-9f99-0f251445df86 None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:34:54.887640 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-340d04df-4c35-414a-9f99-0f251445df86 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:34:54.906814 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.auth.core [None req-340d04df-4c35-414a-9f99-0f251445df86 None None] MFA Rules not processed for user `49f280ab086b4610855c4ffa378517d9`. Rule list: `[]` (Enabled: `True`). {{(pid=81399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 03:34:54.907997 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-340d04df-4c35-414a-9f99-0f251445df86 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:34:54.966128 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 1398/2788] 10.222.0.18 () {58 vars in 958 bytes} [Tue Sep 13 03:34:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 03:34:54.975600 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-054f07ae-1885-4b7d-8499-7d84678d4edf admin admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:34:54.976996 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-054f07ae-1885-4b7d-8499-7d84678d4edf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:34:55.032828 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-054f07ae-1885-4b7d-8499-7d84678d4edf admin admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:34:55.036717 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6ca74e2-0442-4c02-b550-ac734fc0dabe service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdffbc851086441ebd01022d1e4c2609', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b9d94f6a68f7470d9494f112175620dc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:34:55.037305 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6ca74e2-0442-4c02-b550-ac734fc0dabe service neutron] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:34:55.037578 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6ca74e2-0442-4c02-b550-ac734fc0dabe service neutron] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:34:55.037840 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6ca74e2-0442-4c02-b550-ac734fc0dabe service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:34:55.046620 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6ca74e2-0442-4c02-b550-ac734fc0dabe service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:34:55.048037 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6ca74e2-0442-4c02-b550-ac734fc0dabe service neutron] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:34:55.057645 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 1391/2789] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Sep 13 03:34:54 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 03:34:55.189801 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-340d04df-4c35-414a-9f99-0f251445df86 None None] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:34:55.191237 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-340d04df-4c35-414a-9f99-0f251445df86 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:34:55.204990 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-340d04df-4c35-414a-9f99-0f251445df86 None None] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:34:55.210505 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10f32cb2-0ac8-447b-877f-b2fa65c21dcb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76c5181fc7be45279ac00017bb4640b3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:34:55.211228 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f32cb2-0ac8-447b-877f-b2fa65c21dcb admin admin] REQUEST_METHOD: `DELETE` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:34:55.211509 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f32cb2-0ac8-447b-877f-b2fa65c21dcb admin admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:34:55.211778 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f32cb2-0ac8-447b-877f-b2fa65c21dcb admin admin] PATH_INFO: `/v3/users/2a440469f051496089c827052b2e38de` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:34:55.213138 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10f32cb2-0ac8-447b-877f-b2fa65c21dcb admin admin] RBAC: Authorizing `identity:delete_user(user_id=2a440469f051496089c827052b2e38de)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:34:55.214840 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10f32cb2-0ac8-447b-877f-b2fa65c21dcb admin admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:34:55.264095 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-10f32cb2-0ac8-447b-877f-b2fa65c21dcb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2a440469f051496089c827052b2e38de'} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:34:55.271635 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-10f32cb2-0ac8-447b-877f-b2fa65c21dcb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2a440469f051496089c827052b2e38de'} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:34:55.283640 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-10f32cb2-0ac8-447b-877f-b2fa65c21dcb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2a440469f051496089c827052b2e38de'} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:34:55.284414 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-10f32cb2-0ac8-447b-877f-b2fa65c21dcb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2a440469f051496089c827052b2e38de'} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:34:55.291674 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-10f32cb2-0ac8-447b-877f-b2fa65c21dcb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2a440469f051496089c827052b2e38de'} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:34:55.297446 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 1399/2790] 10.222.0.18 () {58 vars in 1224 bytes} [Tue Sep 13 03:34:55 2022] DELETE /identity/v3/users/2a440469f051496089c827052b2e38de => generated 0 bytes in 108 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 03:34:55.302324 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cce40b30-783b-46c8-b377-f046a8cfc9e0 None None] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:34:55.302621 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cce40b30-783b-46c8-b377-f046a8cfc9e0 None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:34:55.302892 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cce40b30-783b-46c8-b377-f046a8cfc9e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:34:55.323562 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.auth.core [None req-cce40b30-783b-46c8-b377-f046a8cfc9e0 None None] MFA Rules not processed for user `49f280ab086b4610855c4ffa378517d9`. Rule list: `[]` (Enabled: `True`). {{(pid=81400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 03:34:55.325366 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-cce40b30-783b-46c8-b377-f046a8cfc9e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:34:55.347625 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10f32cb2-0ac8-447b-877f-b2fa65c21dcb admin admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:34:55.349089 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-10f32cb2-0ac8-447b-877f-b2fa65c21dcb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:34:55.356445 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:34:55.405442 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 1392/2791] 10.222.0.18 () {58 vars in 958 bytes} [Tue Sep 13 03:34:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 13 03:34:55.415711 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cce40b30-783b-46c8-b377-f046a8cfc9e0 None None] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:34:55.417026 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-cce40b30-783b-46c8-b377-f046a8cfc9e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:34:55.421593 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10f32cb2-0ac8-447b-877f-b2fa65c21dcb admin admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:34:55.426244 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2018c3a-15df-424a-961d-bca7e591630a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76c5181fc7be45279ac00017bb4640b3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:34:55.427520 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2018c3a-15df-424a-961d-bca7e591630a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:34:55.428042 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2018c3a-15df-424a-961d-bca7e591630a admin admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:34:55.428042 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2018c3a-15df-424a-961d-bca7e591630a admin admin] PATH_INFO: `/v3/projects/418f2e09bdcd40318cc106e7d98007fc` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:34:55.429205 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2018c3a-15df-424a-961d-bca7e591630a admin admin] RBAC: Authorizing `identity:delete_project(project_id=418f2e09bdcd40318cc106e7d98007fc)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:34:55.431297 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2018c3a-15df-424a-961d-bca7e591630a admin admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:34:55.473860 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cce40b30-783b-46c8-b377-f046a8cfc9e0 None None] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:34:55.479078 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-b2018c3a-15df-424a-961d-bca7e591630a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '418f2e09bdcd40318cc106e7d98007fc', 'request_id': 'req-b2018c3a-15df-424a-961d-bca7e591630a'} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:34:55.485018 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-b2018c3a-15df-424a-961d-bca7e591630a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '418f2e09bdcd40318cc106e7d98007fc', 'request_id': 'req-b2018c3a-15df-424a-961d-bca7e591630a'} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:34:55.491688 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-b2018c3a-15df-424a-961d-bca7e591630a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '418f2e09bdcd40318cc106e7d98007fc', 'request_id': 'req-b2018c3a-15df-424a-961d-bca7e591630a'} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:34:55.494238 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:INSERT=1,keystone:DELETE=3,keystone:SELECT=96 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:34:55.506752 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-b2018c3a-15df-424a-961d-bca7e591630a admin admin] The token cache is being invalidate because project 418f2e09bdcd40318cc106e7d98007fc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81399) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 03:34:55.507047 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-b2018c3a-15df-424a-961d-bca7e591630a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:34:55.507692 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-b2018c3a-15df-424a-961d-bca7e591630a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:34:55.509413 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 1400/2792] 10.222.0.18 () {58 vars in 1233 bytes} [Tue Sep 13 03:34:55 2022] DELETE /identity/v3/projects/418f2e09bdcd40318cc106e7d98007fc => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 13 03:34:55.513413 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2018c3a-15df-424a-961d-bca7e591630a admin admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:34:55.514745 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-b2018c3a-15df-424a-961d-bca7e591630a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:34:55.528877 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce77749e-62bb-42db-9247-577bf3c481d7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdffbc851086441ebd01022d1e4c2609', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b9d94f6a68f7470d9494f112175620dc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:34:55.529480 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce77749e-62bb-42db-9247-577bf3c481d7 service neutron] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:34:55.529758 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce77749e-62bb-42db-9247-577bf3c481d7 service neutron] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:34:55.530087 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce77749e-62bb-42db-9247-577bf3c481d7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:34:55.531772 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-ce77749e-62bb-42db-9247-577bf3c481d7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:34:55.578058 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2018c3a-15df-424a-961d-bca7e591630a admin admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:34:55.583076 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e168e2fc-8528-4c79-857e-37c9ad9434b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76c5181fc7be45279ac00017bb4640b3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:34:55.583700 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e168e2fc-8528-4c79-857e-37c9ad9434b3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:34:55.583977 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e168e2fc-8528-4c79-857e-37c9ad9434b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:34:55.584249 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e168e2fc-8528-4c79-857e-37c9ad9434b3 admin admin] PATH_INFO: `/v3/users/2cb1f7fbb923481bb8182dc7662beb45` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:34:55.585377 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e168e2fc-8528-4c79-857e-37c9ad9434b3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2cb1f7fbb923481bb8182dc7662beb45)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:34:55.587337 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e168e2fc-8528-4c79-857e-37c9ad9434b3 admin admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:34:55.596778 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce77749e-62bb-42db-9247-577bf3c481d7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:34:55.598865 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce77749e-62bb-42db-9247-577bf3c481d7 service neutron] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:34:55.608382 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 1393/2793] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Sep 13 03:34:55 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 193 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 13 03:34:55.627429 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:34:55.638143 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-e168e2fc-8528-4c79-857e-37c9ad9434b3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2cb1f7fbb923481bb8182dc7662beb45'} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:34:55.646646 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-e168e2fc-8528-4c79-857e-37c9ad9434b3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2cb1f7fbb923481bb8182dc7662beb45'} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:34:55.660029 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-e168e2fc-8528-4c79-857e-37c9ad9434b3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2cb1f7fbb923481bb8182dc7662beb45'} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:34:55.661418 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-e168e2fc-8528-4c79-857e-37c9ad9434b3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2cb1f7fbb923481bb8182dc7662beb45'} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:34:55.669098 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-e168e2fc-8528-4c79-857e-37c9ad9434b3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2cb1f7fbb923481bb8182dc7662beb45'} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:34:55.675425 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 1401/2794] 10.222.0.18 () {58 vars in 1224 bytes} [Tue Sep 13 03:34:55 2022] DELETE /identity/v3/users/2cb1f7fbb923481bb8182dc7662beb45 => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 13 03:34:55.935923 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce77749e-62bb-42db-9247-577bf3c481d7 service neutron] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:34:55.937436 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-ce77749e-62bb-42db-9247-577bf3c481d7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:34:55.949374 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce77749e-62bb-42db-9247-577bf3c481d7 service neutron] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:34:56.004295 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e168e2fc-8528-4c79-857e-37c9ad9434b3 admin admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:34:56.006144 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-e168e2fc-8528-4c79-857e-37c9ad9434b3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:34:56.018187 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-179cc811-c76b-4f73-b55e-2a828fd88d09 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76c5181fc7be45279ac00017bb4640b3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:34:56.018978 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-179cc811-c76b-4f73-b55e-2a828fd88d09 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:34:56.019334 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-179cc811-c76b-4f73-b55e-2a828fd88d09 admin admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:34:56.019670 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-179cc811-c76b-4f73-b55e-2a828fd88d09 admin admin] PATH_INFO: `/v3/projects/a9911602e4994d9ebb78a04c162f6b45` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:34:56.021366 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-179cc811-c76b-4f73-b55e-2a828fd88d09 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a9911602e4994d9ebb78a04c162f6b45)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:34:56.023393 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-179cc811-c76b-4f73-b55e-2a828fd88d09 admin admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:34:56.074130 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-179cc811-c76b-4f73-b55e-2a828fd88d09 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a9911602e4994d9ebb78a04c162f6b45', 'request_id': 'req-179cc811-c76b-4f73-b55e-2a828fd88d09'} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:34:56.079975 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-179cc811-c76b-4f73-b55e-2a828fd88d09 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a9911602e4994d9ebb78a04c162f6b45', 'request_id': 'req-179cc811-c76b-4f73-b55e-2a828fd88d09'} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:34:56.086715 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-179cc811-c76b-4f73-b55e-2a828fd88d09 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a9911602e4994d9ebb78a04c162f6b45', 'request_id': 'req-179cc811-c76b-4f73-b55e-2a828fd88d09'} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:34:56.094305 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e168e2fc-8528-4c79-857e-37c9ad9434b3 admin admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:34:56.101323 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-179cc811-c76b-4f73-b55e-2a828fd88d09 admin admin] The token cache is being invalidate because project a9911602e4994d9ebb78a04c162f6b45 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81400) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 03:34:56.101690 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-179cc811-c76b-4f73-b55e-2a828fd88d09 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:34:56.102433 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-179cc811-c76b-4f73-b55e-2a828fd88d09 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:34:56.104039 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 1394/2795] 10.222.0.18 () {58 vars in 1233 bytes} [Tue Sep 13 03:34:55 2022] DELETE /identity/v3/projects/a9911602e4994d9ebb78a04c162f6b45 => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 03:34:56.108757 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-746c370e-63ec-4252-b89e-9073ccbe7a09 None None] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:34:56.109037 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-746c370e-63ec-4252-b89e-9073ccbe7a09 None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:34:56.109306 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-746c370e-63ec-4252-b89e-9073ccbe7a09 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:34:56.117770 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:34:56.130252 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.auth.core [None req-746c370e-63ec-4252-b89e-9073ccbe7a09 None None] MFA Rules not processed for user `49f280ab086b4610855c4ffa378517d9`. Rule list: `[]` (Enabled: `True`). {{(pid=81400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 03:34:56.131563 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-746c370e-63ec-4252-b89e-9073ccbe7a09 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:34:56.152372 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70c8021f-7241-4028-9dd2-9c3130ffd545 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76c5181fc7be45279ac00017bb4640b3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:34:56.153010 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c8021f-7241-4028-9dd2-9c3130ffd545 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:34:56.153290 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c8021f-7241-4028-9dd2-9c3130ffd545 admin admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:34:56.153565 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c8021f-7241-4028-9dd2-9c3130ffd545 admin admin] PATH_INFO: `/v3/projects/0a5f6227c54049b8b75b0a7c06d84e68` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:34:56.154739 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70c8021f-7241-4028-9dd2-9c3130ffd545 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0a5f6227c54049b8b75b0a7c06d84e68)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:34:56.156652 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70c8021f-7241-4028-9dd2-9c3130ffd545 admin admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:34:56.191617 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 1395/2796] 10.222.0.18 () {58 vars in 958 bytes} [Tue Sep 13 03:34:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 13 03:34:56.199822 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44091db0-eb3d-411c-b98f-b9cb3a30ecde None None] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:34:56.200178 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44091db0-eb3d-411c-b98f-b9cb3a30ecde None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:34:56.200534 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44091db0-eb3d-411c-b98f-b9cb3a30ecde None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:34:56.214402 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-70c8021f-7241-4028-9dd2-9c3130ffd545 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0a5f6227c54049b8b75b0a7c06d84e68', 'request_id': 'req-70c8021f-7241-4028-9dd2-9c3130ffd545'} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:34:56.220213 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-70c8021f-7241-4028-9dd2-9c3130ffd545 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0a5f6227c54049b8b75b0a7c06d84e68', 'request_id': 'req-70c8021f-7241-4028-9dd2-9c3130ffd545'} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:34:56.224454 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.auth.core [None req-44091db0-eb3d-411c-b98f-b9cb3a30ecde None None] MFA Rules not processed for user `49f280ab086b4610855c4ffa378517d9`. Rule list: `[]` (Enabled: `True`). {{(pid=81400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 03:34:56.225664 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-44091db0-eb3d-411c-b98f-b9cb3a30ecde None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:34:56.226473 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-70c8021f-7241-4028-9dd2-9c3130ffd545 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0a5f6227c54049b8b75b0a7c06d84e68', 'request_id': 'req-70c8021f-7241-4028-9dd2-9c3130ffd545'} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:34:56.232698 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:34:56.247828 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-70c8021f-7241-4028-9dd2-9c3130ffd545 admin admin] The token cache is being invalidate because project 0a5f6227c54049b8b75b0a7c06d84e68 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81399) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 03:34:56.248540 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-70c8021f-7241-4028-9dd2-9c3130ffd545 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:34:56.249214 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-70c8021f-7241-4028-9dd2-9c3130ffd545 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:34:56.250726 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 1402/2797] 10.222.0.18 () {58 vars in 1233 bytes} [Tue Sep 13 03:34:56 2022] DELETE /identity/v3/projects/0a5f6227c54049b8b75b0a7c06d84e68 => generated 0 bytes in 247 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 03:34:56.255864 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df9a6d0f-b05d-4b5f-87e0-fce3c04633a3 None None] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:34:56.256148 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df9a6d0f-b05d-4b5f-87e0-fce3c04633a3 None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:34:56.256417 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df9a6d0f-b05d-4b5f-87e0-fce3c04633a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:34:56.276080 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.auth.core [None req-df9a6d0f-b05d-4b5f-87e0-fce3c04633a3 None None] MFA Rules not processed for user `49f280ab086b4610855c4ffa378517d9`. Rule list: `[]` (Enabled: `True`). {{(pid=81399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 03:34:56.277278 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-df9a6d0f-b05d-4b5f-87e0-fce3c04633a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:34:56.289917 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 1396/2798] 10.222.0.18 () {58 vars in 958 bytes} [Tue Sep 13 03:34:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 03:34:56.294060 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44091db0-eb3d-411c-b98f-b9cb3a30ecde None None] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:34:56.303039 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44091db0-eb3d-411c-b98f-b9cb3a30ecde None None] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:34:56.307723 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8134ddb-9a57-4c52-a45c-0bf824a39cc8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76c5181fc7be45279ac00017bb4640b3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:34:56.308307 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8134ddb-9a57-4c52-a45c-0bf824a39cc8 admin admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:34:56.308583 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8134ddb-9a57-4c52-a45c-0bf824a39cc8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:34:56.308859 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8134ddb-9a57-4c52-a45c-0bf824a39cc8 admin admin] PATH_INFO: `/v3/domains` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:34:56.309494 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8134ddb-9a57-4c52-a45c-0bf824a39cc8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 03:34:56.309796 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8134ddb-9a57-4c52-a45c-0bf824a39cc8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:34:56.311425 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8134ddb-9a57-4c52-a45c-0bf824a39cc8 admin admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:34:56.326552 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 1397/2799] 10.222.0.18 () {58 vars in 1153 bytes} [Tue Sep 13 03:34:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 03:34:56.328253 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8134ddb-9a57-4c52-a45c-0bf824a39cc8 admin admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:34:56.337437 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 1403/2800] 10.222.0.18 () {58 vars in 958 bytes} [Tue Sep 13 03:34:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 13 03:34:56.343472 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8134ddb-9a57-4c52-a45c-0bf824a39cc8 admin admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:34:56.343843 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cafcfe4-7fff-4e34-93db-9e36292b277b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76c5181fc7be45279ac00017bb4640b3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:34:56.344433 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cafcfe4-7fff-4e34-93db-9e36292b277b admin admin] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:34:56.344708 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cafcfe4-7fff-4e34-93db-9e36292b277b admin admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:34:56.344815 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca0478cd-0ca6-481d-9573-df20384eb159 None None] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:34:56.344982 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cafcfe4-7fff-4e34-93db-9e36292b277b admin admin] PATH_INFO: `/v3/projects` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:34:56.345146 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca0478cd-0ca6-481d-9573-df20384eb159 None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:34:56.345319 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca0478cd-0ca6-481d-9573-df20384eb159 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:34:56.345600 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cafcfe4-7fff-4e34-93db-9e36292b277b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:34:56.347571 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cafcfe4-7fff-4e34-93db-9e36292b277b admin admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:34:56.364844 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.auth.core [None req-ca0478cd-0ca6-481d-9573-df20384eb159 None None] MFA Rules not processed for user `49f280ab086b4610855c4ffa378517d9`. Rule list: `[]` (Enabled: `True`). {{(pid=81399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 03:34:56.366023 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-ca0478cd-0ca6-481d-9573-df20384eb159 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:34:56.376755 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 1398/2801] 10.222.0.18 () {60 vars in 1153 bytes} [Tue Sep 13 03:34:56 2022] POST /identity/v3/projects => generated 385 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 03:34:56.380681 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 1404/2802] 10.222.0.18 () {58 vars in 958 bytes} [Tue Sep 13 03:34:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 03:34:56.381545 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca0478cd-0ca6-481d-9573-df20384eb159 None None] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:34:56.381668 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cafcfe4-7fff-4e34-93db-9e36292b277b admin admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:34:56.390935 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca0478cd-0ca6-481d-9573-df20384eb159 None None] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:34:56.444886 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cafcfe4-7fff-4e34-93db-9e36292b277b admin admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:34:56.447933 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13e58e0e-4610-4087-9ca6-3394f7adcff5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76c5181fc7be45279ac00017bb4640b3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:34:56.448545 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13e58e0e-4610-4087-9ca6-3394f7adcff5 admin admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:34:56.448826 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13e58e0e-4610-4087-9ca6-3394f7adcff5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:34:56.449094 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13e58e0e-4610-4087-9ca6-3394f7adcff5 admin admin] PATH_INFO: `/v3/domains` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:34:56.449733 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13e58e0e-4610-4087-9ca6-3394f7adcff5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 03:34:56.449869 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58a2b7d7-0bee-488e-8ea9-a90d75720c7d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76c5181fc7be45279ac00017bb4640b3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:34:56.450086 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13e58e0e-4610-4087-9ca6-3394f7adcff5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:34:56.450469 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58a2b7d7-0bee-488e-8ea9-a90d75720c7d admin admin] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:34:56.450771 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58a2b7d7-0bee-488e-8ea9-a90d75720c7d admin admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:34:56.451041 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58a2b7d7-0bee-488e-8ea9-a90d75720c7d admin admin] PATH_INFO: `/v3/users` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:34:56.451645 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13e58e0e-4610-4087-9ca6-3394f7adcff5 admin admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:34:56.451804 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58a2b7d7-0bee-488e-8ea9-a90d75720c7d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:34:56.453769 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58a2b7d7-0bee-488e-8ea9-a90d75720c7d admin admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:34:56.463610 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 1405/2803] 10.222.0.18 () {58 vars in 1153 bytes} [Tue Sep 13 03:34:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 03:34:56.468269 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13e58e0e-4610-4087-9ca6-3394f7adcff5 admin admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:34:56.471867 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:34:56.478859 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 1399/2804] 10.222.0.18 () {60 vars in 1144 bytes} [Tue Sep 13 03:34:56 2022] POST /identity/v3/users => generated 293 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 03:34:56.479034 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13e58e0e-4610-4087-9ca6-3394f7adcff5 admin admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:34:56.482413 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58a2b7d7-0bee-488e-8ea9-a90d75720c7d admin admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:34:56.483799 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ac39af7-dfb7-4bfd-870f-d253318d59b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76c5181fc7be45279ac00017bb4640b3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:34:56.484400 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ac39af7-dfb7-4bfd-870f-d253318d59b8 admin admin] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:34:56.484682 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ac39af7-dfb7-4bfd-870f-d253318d59b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:34:56.484954 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ac39af7-dfb7-4bfd-870f-d253318d59b8 admin admin] PATH_INFO: `/v3/projects` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:34:56.485613 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ac39af7-dfb7-4bfd-870f-d253318d59b8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:34:56.487607 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ac39af7-dfb7-4bfd-870f-d253318d59b8 admin admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:34:56.492557 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58a2b7d7-0bee-488e-8ea9-a90d75720c7d admin admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:34:56.497204 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d52ab66-f53d-493f-acf3-3b0a0172cec1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76c5181fc7be45279ac00017bb4640b3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:34:56.497807 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d52ab66-f53d-493f-acf3-3b0a0172cec1 admin admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:34:56.498082 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d52ab66-f53d-493f-acf3-3b0a0172cec1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:34:56.498346 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d52ab66-f53d-493f-acf3-3b0a0172cec1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:34:56.499009 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d52ab66-f53d-493f-acf3-3b0a0172cec1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:34:56.500603 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d52ab66-f53d-493f-acf3-3b0a0172cec1 admin admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:34:56.511569 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 1400/2805] 10.222.0.18 () {58 vars in 1122 bytes} [Tue Sep 13 03:34:56 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 13 03:34:56.515470 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 1406/2806] 10.222.0.18 () {60 vars in 1153 bytes} [Tue Sep 13 03:34:56 2022] POST /identity/v3/projects => generated 379 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 03:34:56.515631 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d52ab66-f53d-493f-acf3-3b0a0172cec1 admin admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:34:56.519203 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ac39af7-dfb7-4bfd-870f-d253318d59b8 admin admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:34:56.527110 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d52ab66-f53d-493f-acf3-3b0a0172cec1 admin admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:34:56.578066 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:INSERT=2,keystone:DELETE=1,keystone:SELECT=97 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:34:56.582086 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ac39af7-dfb7-4bfd-870f-d253318d59b8 admin admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:34:56.584705 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ef0854f-4556-4206-ad83-2a7fafd8aea8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76c5181fc7be45279ac00017bb4640b3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:34:56.585572 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef0854f-4556-4206-ad83-2a7fafd8aea8 admin admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:34:56.586006 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef0854f-4556-4206-ad83-2a7fafd8aea8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:34:56.586451 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef0854f-4556-4206-ad83-2a7fafd8aea8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:34:56.587212 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ef0854f-4556-4206-ad83-2a7fafd8aea8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:34:56.588144 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c073a4b-13e3-4269-9834-94f60cf0cea6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76c5181fc7be45279ac00017bb4640b3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:34:56.588793 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c073a4b-13e3-4269-9834-94f60cf0cea6 admin admin] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:34:56.589060 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ef0854f-4556-4206-ad83-2a7fafd8aea8 admin admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:34:56.589223 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c073a4b-13e3-4269-9834-94f60cf0cea6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:34:56.589336 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c073a4b-13e3-4269-9834-94f60cf0cea6 admin admin] PATH_INFO: `/v3/users` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:34:56.590220 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c073a4b-13e3-4269-9834-94f60cf0cea6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:34:56.592619 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c073a4b-13e3-4269-9834-94f60cf0cea6 admin admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:34:56.599057 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 1401/2807] 10.222.0.18 () {58 vars in 1122 bytes} [Tue Sep 13 03:34:56 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 13 03:34:56.603237 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ef0854f-4556-4206-ad83-2a7fafd8aea8 admin admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:34:56.614343 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ef0854f-4556-4206-ad83-2a7fafd8aea8 admin admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:34:56.615747 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 1407/2808] 10.222.0.18 () {60 vars in 1144 bytes} [Tue Sep 13 03:34:56 2022] POST /identity/v3/users => generated 290 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 13 03:34:56.619353 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53d3f3b4-c829-4b02-9774-5102cfbb738a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76c5181fc7be45279ac00017bb4640b3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:34:56.619686 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c073a4b-13e3-4269-9834-94f60cf0cea6 admin admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:34:56.619995 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53d3f3b4-c829-4b02-9774-5102cfbb738a admin admin] REQUEST_METHOD: `PUT` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:34:56.620273 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53d3f3b4-c829-4b02-9774-5102cfbb738a admin admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:34:56.620545 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53d3f3b4-c829-4b02-9774-5102cfbb738a admin admin] PATH_INFO: `/v3/projects/1423d25ff51a4eb3810e409baed076f0/users/155ed81d5c9a4e28ba2c6afd1b34fcf6/roles/1eb7a5b5203945c29e6d22d653f47847` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:34:56.630632 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c073a4b-13e3-4269-9834-94f60cf0cea6 admin admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:34:56.635853 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-214ad0c9-f819-4448-8968-296ad0eb2043 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76c5181fc7be45279ac00017bb4640b3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:34:56.636446 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-214ad0c9-f819-4448-8968-296ad0eb2043 admin admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:34:56.636719 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53d3f3b4-c829-4b02-9774-5102cfbb738a admin admin] RBAC: Authorizing `identity:create_grant(project_id=1423d25ff51a4eb3810e409baed076f0, user_id=155ed81d5c9a4e28ba2c6afd1b34fcf6, role_id=1eb7a5b5203945c29e6d22d653f47847)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:34:56.636861 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-214ad0c9-f819-4448-8968-296ad0eb2043 admin admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:34:56.636966 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-214ad0c9-f819-4448-8968-296ad0eb2043 admin admin] PATH_INFO: `/v3/roles` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:34:56.637597 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-214ad0c9-f819-4448-8968-296ad0eb2043 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:34:56.639222 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-214ad0c9-f819-4448-8968-296ad0eb2043 admin admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:34:56.639407 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53d3f3b4-c829-4b02-9774-5102cfbb738a admin admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:34:56.649595 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 1408/2809] 10.222.0.18 () {58 vars in 1122 bytes} [Tue Sep 13 03:34:56 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 13 03:34:56.652022 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 1402/2810] 10.222.0.18 () {60 vars in 1483 bytes} [Tue Sep 13 03:34:56 2022] PUT /identity/v3/projects/1423d25ff51a4eb3810e409baed076f0/users/155ed81d5c9a4e28ba2c6afd1b34fcf6/roles/1eb7a5b5203945c29e6d22d653f47847 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 03:34:56.653636 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-214ad0c9-f819-4448-8968-296ad0eb2043 admin admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:34:56.662637 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-580930ac-a958-45d1-9be3-8ad1a9cc7a7c None None] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:34:56.662927 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-580930ac-a958-45d1-9be3-8ad1a9cc7a7c None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:34:56.663199 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-580930ac-a958-45d1-9be3-8ad1a9cc7a7c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:34:56.664141 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-214ad0c9-f819-4448-8968-296ad0eb2043 admin admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:34:56.690631 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.auth.core [None req-580930ac-a958-45d1-9be3-8ad1a9cc7a7c None None] MFA Rules not processed for user `155ed81d5c9a4e28ba2c6afd1b34fcf6`. Rule list: `[]` (Enabled: `True`). {{(pid=81400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 03:34:56.691898 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-580930ac-a958-45d1-9be3-8ad1a9cc7a7c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:34:56.722915 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ca430cf-b2af-4dd1-a38d-11d45227b461 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76c5181fc7be45279ac00017bb4640b3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:34:56.723478 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca430cf-b2af-4dd1-a38d-11d45227b461 admin admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:34:56.723758 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca430cf-b2af-4dd1-a38d-11d45227b461 admin admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:34:56.724032 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca430cf-b2af-4dd1-a38d-11d45227b461 admin admin] PATH_INFO: `/v3/roles` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:34:56.724664 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ca430cf-b2af-4dd1-a38d-11d45227b461 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:34:56.726248 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ca430cf-b2af-4dd1-a38d-11d45227b461 admin admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:34:56.736283 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 1409/2811] 10.222.0.18 () {58 vars in 1122 bytes} [Tue Sep 13 03:34:56 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 13 03:34:56.740678 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ca430cf-b2af-4dd1-a38d-11d45227b461 admin admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:34:56.750277 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ca430cf-b2af-4dd1-a38d-11d45227b461 admin admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:34:56.755542 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8adf3ff6-0e6f-4d5c-81c4-60e0f4d9ecbe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76c5181fc7be45279ac00017bb4640b3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:34:56.756174 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8adf3ff6-0e6f-4d5c-81c4-60e0f4d9ecbe admin admin] REQUEST_METHOD: `PUT` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:34:56.756450 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8adf3ff6-0e6f-4d5c-81c4-60e0f4d9ecbe admin admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:34:56.756719 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8adf3ff6-0e6f-4d5c-81c4-60e0f4d9ecbe admin admin] PATH_INFO: `/v3/projects/d332ca9b985e4ba692f509b669f69970/users/bb9b6445f2fc482496009f5922446e21/roles/1eb7a5b5203945c29e6d22d653f47847` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:34:56.757679 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 1403/2812] 10.222.0.18 () {58 vars in 958 bytes} [Tue Sep 13 03:34:56 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 13 03:34:56.761814 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-580930ac-a958-45d1-9be3-8ad1a9cc7a7c None None] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:34:56.772173 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8adf3ff6-0e6f-4d5c-81c4-60e0f4d9ecbe admin admin] RBAC: Authorizing `identity:create_grant(project_id=d332ca9b985e4ba692f509b669f69970, user_id=bb9b6445f2fc482496009f5922446e21, role_id=1eb7a5b5203945c29e6d22d653f47847)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:34:56.773111 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-580930ac-a958-45d1-9be3-8ad1a9cc7a7c None None] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:34:56.774870 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8adf3ff6-0e6f-4d5c-81c4-60e0f4d9ecbe admin admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:34:56.777684 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-804f418b-1377-47b9-b596-680c209a7b85 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76c5181fc7be45279ac00017bb4640b3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:34:56.778272 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-804f418b-1377-47b9-b596-680c209a7b85 admin admin] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:34:56.778551 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-804f418b-1377-47b9-b596-680c209a7b85 admin admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:34:56.778853 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-804f418b-1377-47b9-b596-680c209a7b85 admin admin] PATH_INFO: `/v3/projects` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:34:56.779724 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-804f418b-1377-47b9-b596-680c209a7b85 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:34:56.781898 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-804f418b-1377-47b9-b596-680c209a7b85 admin admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:34:56.788031 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 1410/2813] 10.222.0.18 () {60 vars in 1483 bytes} [Tue Sep 13 03:34:56 2022] PUT /identity/v3/projects/d332ca9b985e4ba692f509b669f69970/users/bb9b6445f2fc482496009f5922446e21/roles/1eb7a5b5203945c29e6d22d653f47847 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 03:34:56.797985 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef564a8-c55d-416b-868f-114f6a8ed1bc None None] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:34:56.798270 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef564a8-c55d-416b-868f-114f6a8ed1bc None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:34:56.798546 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef564a8-c55d-416b-868f-114f6a8ed1bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:34:56.810043 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 1404/2814] 10.222.0.18 () {60 vars in 1153 bytes} [Tue Sep 13 03:34:56 2022] POST /identity/v3/projects => generated 383 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 13 03:34:56.813902 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-804f418b-1377-47b9-b596-680c209a7b85 admin admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:34:56.829045 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.auth.core [None req-4ef564a8-c55d-416b-868f-114f6a8ed1bc None None] MFA Rules not processed for user `bb9b6445f2fc482496009f5922446e21`. Rule list: `[]` (Enabled: `True`). {{(pid=81399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 03:34:56.830279 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-4ef564a8-c55d-416b-868f-114f6a8ed1bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:34:56.846389 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:34:56.880608 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-804f418b-1377-47b9-b596-680c209a7b85 admin admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:34:56.885452 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f56bd5e-f95a-4f3e-9074-411dbd37d63e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76c5181fc7be45279ac00017bb4640b3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:34:56.886062 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f56bd5e-f95a-4f3e-9074-411dbd37d63e admin admin] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:34:56.886344 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f56bd5e-f95a-4f3e-9074-411dbd37d63e admin admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:34:56.886775 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f56bd5e-f95a-4f3e-9074-411dbd37d63e admin admin] PATH_INFO: `/v3/users` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:34:56.887406 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f56bd5e-f95a-4f3e-9074-411dbd37d63e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:34:56.889340 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f56bd5e-f95a-4f3e-9074-411dbd37d63e admin admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:34:56.896399 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 1411/2815] 10.222.0.18 () {58 vars in 958 bytes} [Tue Sep 13 03:34:56 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 13 03:34:56.900957 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ef564a8-c55d-416b-868f-114f6a8ed1bc None None] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:34:56.914357 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ef564a8-c55d-416b-868f-114f6a8ed1bc None None] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:34:56.914559 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 1405/2816] 10.222.0.18 () {60 vars in 1144 bytes} [Tue Sep 13 03:34:56 2022] POST /identity/v3/users => generated 292 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 13 03:34:56.915383 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f56bd5e-f95a-4f3e-9074-411dbd37d63e admin admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:34:56.916651 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ac9fc36-cecb-48b5-b338-671d5737a13f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76c5181fc7be45279ac00017bb4640b3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:34:56.917394 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac9fc36-cecb-48b5-b338-671d5737a13f admin admin] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:34:56.917678 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac9fc36-cecb-48b5-b338-671d5737a13f admin admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:34:56.917953 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac9fc36-cecb-48b5-b338-671d5737a13f admin admin] PATH_INFO: `/v3/projects` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:34:56.918634 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ac9fc36-cecb-48b5-b338-671d5737a13f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:34:56.920865 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ac9fc36-cecb-48b5-b338-671d5737a13f admin admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:34:56.925048 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f56bd5e-f95a-4f3e-9074-411dbd37d63e admin admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:34:56.930439 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55a39174-e8d6-4e96-a3d4-df0914f77fb9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76c5181fc7be45279ac00017bb4640b3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:34:56.931043 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55a39174-e8d6-4e96-a3d4-df0914f77fb9 admin admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:34:56.931325 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55a39174-e8d6-4e96-a3d4-df0914f77fb9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:34:56.931592 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55a39174-e8d6-4e96-a3d4-df0914f77fb9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:34:56.932261 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55a39174-e8d6-4e96-a3d4-df0914f77fb9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:34:56.933970 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55a39174-e8d6-4e96-a3d4-df0914f77fb9 admin admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:34:56.947717 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 1406/2817] 10.222.0.18 () {58 vars in 1122 bytes} [Tue Sep 13 03:34:56 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 13 03:34:56.948698 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55a39174-e8d6-4e96-a3d4-df0914f77fb9 admin admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:34:56.950322 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 1412/2818] 10.222.0.18 () {60 vars in 1153 bytes} [Tue Sep 13 03:34:56 2022] POST /identity/v3/projects => generated 381 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 13 03:34:56.954034 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ac9fc36-cecb-48b5-b338-671d5737a13f admin admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:34:56.961482 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55a39174-e8d6-4e96-a3d4-df0914f77fb9 admin admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:34:56.972600 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:34:57.018556 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ac9fc36-cecb-48b5-b338-671d5737a13f admin admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:34:57.018797 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c65f538e-989f-484c-a33d-b47aab0a8d9f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76c5181fc7be45279ac00017bb4640b3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:34:57.019467 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c65f538e-989f-484c-a33d-b47aab0a8d9f admin admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:34:57.019809 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c65f538e-989f-484c-a33d-b47aab0a8d9f admin admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:34:57.020108 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c65f538e-989f-484c-a33d-b47aab0a8d9f admin admin] PATH_INFO: `/v3/roles` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:34:57.020735 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c65f538e-989f-484c-a33d-b47aab0a8d9f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:34:57.022322 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c65f538e-989f-484c-a33d-b47aab0a8d9f admin admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:34:57.023326 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4056e5e-2838-4bdd-af0b-92a31ed5ee7d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76c5181fc7be45279ac00017bb4640b3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:34:57.024037 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4056e5e-2838-4bdd-af0b-92a31ed5ee7d admin admin] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:34:57.024548 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4056e5e-2838-4bdd-af0b-92a31ed5ee7d admin admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:34:57.024675 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4056e5e-2838-4bdd-af0b-92a31ed5ee7d admin admin] PATH_INFO: `/v3/users` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:34:57.025280 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4056e5e-2838-4bdd-af0b-92a31ed5ee7d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:34:57.027263 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4056e5e-2838-4bdd-af0b-92a31ed5ee7d admin admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:34:57.032529 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 1407/2819] 10.222.0.18 () {58 vars in 1122 bytes} [Tue Sep 13 03:34:56 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 13 03:34:57.037240 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c65f538e-989f-484c-a33d-b47aab0a8d9f admin admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:34:57.047943 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c65f538e-989f-484c-a33d-b47aab0a8d9f admin admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:34:57.054198 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 1413/2820] 10.222.0.18 () {60 vars in 1144 bytes} [Tue Sep 13 03:34:56 2022] POST /identity/v3/users => generated 297 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 13 03:34:57.054348 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cf20a51-ef74-4ce9-9ef6-b2de06bc08bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76c5181fc7be45279ac00017bb4640b3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:34:57.054348 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cf20a51-ef74-4ce9-9ef6-b2de06bc08bb admin admin] REQUEST_METHOD: `PUT` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:34:57.054348 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cf20a51-ef74-4ce9-9ef6-b2de06bc08bb admin admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:34:57.054739 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cf20a51-ef74-4ce9-9ef6-b2de06bc08bb admin admin] PATH_INFO: `/v3/projects/6e16a1e859034e9abdce7c22596b9f38/users/c900a8f794f042798a22b8104f9d9fde/roles/1eb7a5b5203945c29e6d22d653f47847` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:34:57.055093 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4056e5e-2838-4bdd-af0b-92a31ed5ee7d admin admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:34:57.065168 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4056e5e-2838-4bdd-af0b-92a31ed5ee7d admin admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:34:57.070433 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f7cd1c5-8a63-4026-8493-2d4118c8710d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76c5181fc7be45279ac00017bb4640b3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:34:57.071079 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f7cd1c5-8a63-4026-8493-2d4118c8710d admin admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:34:57.071407 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f7cd1c5-8a63-4026-8493-2d4118c8710d admin admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:34:57.071741 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f7cd1c5-8a63-4026-8493-2d4118c8710d admin admin] PATH_INFO: `/v3/roles` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:34:57.072435 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f7cd1c5-8a63-4026-8493-2d4118c8710d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:34:57.074028 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f7cd1c5-8a63-4026-8493-2d4118c8710d admin admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:34:57.074544 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cf20a51-ef74-4ce9-9ef6-b2de06bc08bb admin admin] RBAC: Authorizing `identity:create_grant(project_id=6e16a1e859034e9abdce7c22596b9f38, user_id=c900a8f794f042798a22b8104f9d9fde, role_id=1eb7a5b5203945c29e6d22d653f47847)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:34:57.077280 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cf20a51-ef74-4ce9-9ef6-b2de06bc08bb admin admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:34:57.085507 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 1414/2821] 10.222.0.18 () {58 vars in 1122 bytes} [Tue Sep 13 03:34:57 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 13 03:34:57.089442 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f7cd1c5-8a63-4026-8493-2d4118c8710d admin admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:34:57.091151 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 1408/2822] 10.222.0.18 () {60 vars in 1483 bytes} [Tue Sep 13 03:34:57 2022] PUT /identity/v3/projects/6e16a1e859034e9abdce7c22596b9f38/users/c900a8f794f042798a22b8104f9d9fde/roles/1eb7a5b5203945c29e6d22d653f47847 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 13 03:34:57.099562 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f7cd1c5-8a63-4026-8493-2d4118c8710d admin admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:34:57.158225 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac7360b-aeb0-4740-9b7c-dc9782bf868b None None] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:34:57.158794 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac7360b-aeb0-4740-9b7c-dc9782bf868b None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:34:57.159071 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac7360b-aeb0-4740-9b7c-dc9782bf868b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:34:57.162393 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0896c1e8-acb7-4ec6-9ac0-70d07b9be047 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76c5181fc7be45279ac00017bb4640b3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:34:57.163089 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0896c1e8-acb7-4ec6-9ac0-70d07b9be047 admin admin] REQUEST_METHOD: `PUT` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:34:57.163374 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0896c1e8-acb7-4ec6-9ac0-70d07b9be047 admin admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:34:57.163649 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0896c1e8-acb7-4ec6-9ac0-70d07b9be047 admin admin] PATH_INFO: `/v3/projects/f4217643b7c842dda363a27d6ecf62d5/users/e3adf98191ee446f825c636db6da4835/roles/425d4b481c7846e0bc07e7e805d5b15f` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:34:57.181424 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0896c1e8-acb7-4ec6-9ac0-70d07b9be047 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f4217643b7c842dda363a27d6ecf62d5, user_id=e3adf98191ee446f825c636db6da4835, role_id=425d4b481c7846e0bc07e7e805d5b15f)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:34:57.184296 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0896c1e8-acb7-4ec6-9ac0-70d07b9be047 admin admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:34:57.190670 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.auth.core [None req-1ac7360b-aeb0-4740-9b7c-dc9782bf868b None None] MFA Rules not processed for user `c900a8f794f042798a22b8104f9d9fde`. Rule list: `[]` (Enabled: `True`). {{(pid=81400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 03:34:57.192015 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-1ac7360b-aeb0-4740-9b7c-dc9782bf868b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:34:57.199087 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 1415/2823] 10.222.0.18 () {60 vars in 1483 bytes} [Tue Sep 13 03:34:57 2022] PUT /identity/v3/projects/f4217643b7c842dda363a27d6ecf62d5/users/e3adf98191ee446f825c636db6da4835/roles/425d4b481c7846e0bc07e7e805d5b15f => generated 0 bytes in 110 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 13 03:34:57.202860 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0896c1e8-acb7-4ec6-9ac0-70d07b9be047 admin admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:34:57.257877 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 1409/2824] 10.222.0.18 () {58 vars in 958 bytes} [Tue Sep 13 03:34:57 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 13 03:34:57.262297 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ac7360b-aeb0-4740-9b7c-dc9782bf868b None None] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:34:57.269331 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0896c1e8-acb7-4ec6-9ac0-70d07b9be047 admin admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:34:57.272621 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ac7360b-aeb0-4740-9b7c-dc9782bf868b None None] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:34:57.274344 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f673cb2-162e-489d-a9e9-6dbe9d944cf2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76c5181fc7be45279ac00017bb4640b3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:34:57.275021 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f673cb2-162e-489d-a9e9-6dbe9d944cf2 admin admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:34:57.275297 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f673cb2-162e-489d-a9e9-6dbe9d944cf2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:34:57.275562 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f673cb2-162e-489d-a9e9-6dbe9d944cf2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:34:57.276190 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f673cb2-162e-489d-a9e9-6dbe9d944cf2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:34:57.277427 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96ab4f39-84c1-4a93-a5bd-42a9f1e2bd30 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76c5181fc7be45279ac00017bb4640b3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:34:57.277764 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f673cb2-162e-489d-a9e9-6dbe9d944cf2 admin admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:34:57.278019 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ab4f39-84c1-4a93-a5bd-42a9f1e2bd30 admin admin] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:34:57.278300 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ab4f39-84c1-4a93-a5bd-42a9f1e2bd30 admin admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:34:57.278572 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ab4f39-84c1-4a93-a5bd-42a9f1e2bd30 admin admin] PATH_INFO: `/v3/projects` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:34:57.279374 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96ab4f39-84c1-4a93-a5bd-42a9f1e2bd30 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:34:57.281433 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96ab4f39-84c1-4a93-a5bd-42a9f1e2bd30 admin admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:34:57.287470 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:34:57.288447 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 1416/2825] 10.222.0.18 () {58 vars in 1122 bytes} [Tue Sep 13 03:34:57 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 03:34:57.292679 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f673cb2-162e-489d-a9e9-6dbe9d944cf2 admin admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:34:57.303794 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f673cb2-162e-489d-a9e9-6dbe9d944cf2 admin admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:34:57.308894 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99317d3a-8b0e-4675-a26b-8e8e6ae9b263 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76c5181fc7be45279ac00017bb4640b3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:34:57.309545 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99317d3a-8b0e-4675-a26b-8e8e6ae9b263 admin admin] REQUEST_METHOD: `PUT` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:34:57.309837 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99317d3a-8b0e-4675-a26b-8e8e6ae9b263 admin admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:34:57.310115 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99317d3a-8b0e-4675-a26b-8e8e6ae9b263 admin admin] PATH_INFO: `/v3/projects/f4217643b7c842dda363a27d6ecf62d5/users/e3adf98191ee446f825c636db6da4835/roles/425d4b481c7846e0bc07e7e805d5b15f` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:34:57.312085 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99317d3a-8b0e-4675-a26b-8e8e6ae9b263 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f4217643b7c842dda363a27d6ecf62d5, user_id=e3adf98191ee446f825c636db6da4835, role_id=425d4b481c7846e0bc07e7e805d5b15f)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:34:57.315311 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 1410/2826] 10.222.0.18 () {60 vars in 1153 bytes} [Tue Sep 13 03:34:57 2022] POST /identity/v3/projects => generated 383 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 13 03:34:57.315449 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99317d3a-8b0e-4675-a26b-8e8e6ae9b263 admin admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:34:57.316263 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96ab4f39-84c1-4a93-a5bd-42a9f1e2bd30 admin admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:34:57.322104 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:34:57.327315 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 1417/2827] 10.222.0.18 () {60 vars in 1483 bytes} [Tue Sep 13 03:34:57 2022] PUT /identity/v3/projects/f4217643b7c842dda363a27d6ecf62d5/users/e3adf98191ee446f825c636db6da4835/roles/425d4b481c7846e0bc07e7e805d5b15f => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 13 03:34:57.339065 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-177c2202-a44c-41ee-b151-a1bc2c191774 None None] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:34:57.339362 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-177c2202-a44c-41ee-b151-a1bc2c191774 None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:34:57.339654 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-177c2202-a44c-41ee-b151-a1bc2c191774 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:34:57.368333 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.auth.core [None req-177c2202-a44c-41ee-b151-a1bc2c191774 None None] MFA Rules not processed for user `e3adf98191ee446f825c636db6da4835`. Rule list: `[]` (Enabled: `True`). {{(pid=81399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 03:34:57.369630 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-177c2202-a44c-41ee-b151-a1bc2c191774 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:34:57.383322 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96ab4f39-84c1-4a93-a5bd-42a9f1e2bd30 admin admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:34:57.440704 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 1418/2828] 10.222.0.18 () {58 vars in 958 bytes} [Tue Sep 13 03:34:57 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 13 03:34:57.441860 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3838a6b2-30b3-4e40-a296-8e1061626878 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76c5181fc7be45279ac00017bb4640b3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:34:57.442532 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3838a6b2-30b3-4e40-a296-8e1061626878 admin admin] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:34:57.442863 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3838a6b2-30b3-4e40-a296-8e1061626878 admin admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:34:57.443136 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3838a6b2-30b3-4e40-a296-8e1061626878 admin admin] PATH_INFO: `/v3/users` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:34:57.443767 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3838a6b2-30b3-4e40-a296-8e1061626878 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:34:57.445755 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3838a6b2-30b3-4e40-a296-8e1061626878 admin admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:34:57.453379 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-177c2202-a44c-41ee-b151-a1bc2c191774 None None] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:34:57.454751 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-177c2202-a44c-41ee-b151-a1bc2c191774 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:34:57.467851 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 1411/2829] 10.222.0.18 () {60 vars in 1144 bytes} [Tue Sep 13 03:34:57 2022] POST /identity/v3/users => generated 298 bytes in 152 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 13 03:34:57.471713 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3838a6b2-30b3-4e40-a296-8e1061626878 admin admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:34:57.481929 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3838a6b2-30b3-4e40-a296-8e1061626878 admin admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:34:57.488494 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe91ac90-09c3-48c6-93b4-0de167804c2b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76c5181fc7be45279ac00017bb4640b3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:34:57.489108 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe91ac90-09c3-48c6-93b4-0de167804c2b admin admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:34:57.489389 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe91ac90-09c3-48c6-93b4-0de167804c2b admin admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:34:57.489662 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe91ac90-09c3-48c6-93b4-0de167804c2b admin admin] PATH_INFO: `/v3/roles` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:34:57.490298 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe91ac90-09c3-48c6-93b4-0de167804c2b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:34:57.491929 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe91ac90-09c3-48c6-93b4-0de167804c2b admin admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:34:57.504740 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 1412/2830] 10.222.0.18 () {58 vars in 1122 bytes} [Tue Sep 13 03:34:57 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 13 03:34:57.505678 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe91ac90-09c3-48c6-93b4-0de167804c2b admin admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:34:57.515185 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe91ac90-09c3-48c6-93b4-0de167804c2b admin admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:34:57.515546 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-177c2202-a44c-41ee-b151-a1bc2c191774 None None] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:34:57.519111 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31327475-725f-438e-8f96-1c1db4263558 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '924ee3f608d34a18b0ddc4695f102a56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b9d94f6a68f7470d9494f112175620dc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:34:57.519774 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31327475-725f-438e-8f96-1c1db4263558 service cinder] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:34:57.520486 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31327475-725f-438e-8f96-1c1db4263558 service cinder] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:34:57.520764 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31327475-725f-438e-8f96-1c1db4263558 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:34:57.520899 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb5e4ccb-4f76-460f-91c5-c2ee7acf2bfb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76c5181fc7be45279ac00017bb4640b3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:34:57.521650 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb5e4ccb-4f76-460f-91c5-c2ee7acf2bfb admin admin] REQUEST_METHOD: `PUT` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:34:57.521922 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb5e4ccb-4f76-460f-91c5-c2ee7acf2bfb admin admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:34:57.522189 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb5e4ccb-4f76-460f-91c5-c2ee7acf2bfb admin admin] PATH_INFO: `/v3/projects/ff68ec5b48a94c62a45a8cadcf65b1ec/users/34b9c55f45c940deb3818b3093f48d47/roles/425d4b481c7846e0bc07e7e805d5b15f` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:34:57.529763 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31327475-725f-438e-8f96-1c1db4263558 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:34:57.531396 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31327475-725f-438e-8f96-1c1db4263558 service cinder] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:34:57.537802 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb5e4ccb-4f76-460f-91c5-c2ee7acf2bfb admin admin] RBAC: Authorizing `identity:create_grant(project_id=ff68ec5b48a94c62a45a8cadcf65b1ec, user_id=34b9c55f45c940deb3818b3093f48d47, role_id=425d4b481c7846e0bc07e7e805d5b15f)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:34:57.540597 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb5e4ccb-4f76-460f-91c5-c2ee7acf2bfb admin admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:34:57.542048 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 1419/2831] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Sep 13 03:34:57 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 13 03:34:57.553469 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 1413/2832] 10.222.0.18 () {60 vars in 1483 bytes} [Tue Sep 13 03:34:57 2022] PUT /identity/v3/projects/ff68ec5b48a94c62a45a8cadcf65b1ec/users/34b9c55f45c940deb3818b3093f48d47/roles/425d4b481c7846e0bc07e7e805d5b15f => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 13 03:34:57.557456 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31327475-725f-438e-8f96-1c1db4263558 service cinder] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:34:57.568879 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31327475-725f-438e-8f96-1c1db4263558 service cinder] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:34:57.629701 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3c34c18-bca0-4a53-84be-13cc5d59a910 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76c5181fc7be45279ac00017bb4640b3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:34:57.630374 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3c34c18-bca0-4a53-84be-13cc5d59a910 admin admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:34:57.630761 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3c34c18-bca0-4a53-84be-13cc5d59a910 admin admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:34:57.631305 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3c34c18-bca0-4a53-84be-13cc5d59a910 admin admin] PATH_INFO: `/v3/roles` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:34:57.631802 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3c34c18-bca0-4a53-84be-13cc5d59a910 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:34:57.633451 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3c34c18-bca0-4a53-84be-13cc5d59a910 admin admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:34:57.646673 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 1420/2833] 10.222.0.18 () {58 vars in 1122 bytes} [Tue Sep 13 03:34:57 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 03:34:57.648601 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb5e4ccb-4f76-460f-91c5-c2ee7acf2bfb admin admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:34:57.659741 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb5e4ccb-4f76-460f-91c5-c2ee7acf2bfb admin admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:34:57.665404 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7abec9e1-7ef6-4ff4-898f-523676f0912f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76c5181fc7be45279ac00017bb4640b3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:34:57.666066 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7abec9e1-7ef6-4ff4-898f-523676f0912f admin admin] REQUEST_METHOD: `PUT` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:34:57.666344 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7abec9e1-7ef6-4ff4-898f-523676f0912f admin admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:34:57.666660 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7abec9e1-7ef6-4ff4-898f-523676f0912f admin admin] PATH_INFO: `/v3/projects/ff68ec5b48a94c62a45a8cadcf65b1ec/users/34b9c55f45c940deb3818b3093f48d47/roles/425d4b481c7846e0bc07e7e805d5b15f` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:34:57.668576 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7abec9e1-7ef6-4ff4-898f-523676f0912f admin admin] RBAC: Authorizing `identity:create_grant(project_id=ff68ec5b48a94c62a45a8cadcf65b1ec, user_id=34b9c55f45c940deb3818b3093f48d47, role_id=425d4b481c7846e0bc07e7e805d5b15f)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:34:57.671255 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7abec9e1-7ef6-4ff4-898f-523676f0912f admin admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:34:57.682353 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 1414/2834] 10.222.0.18 () {60 vars in 1483 bytes} [Tue Sep 13 03:34:57 2022] PUT /identity/v3/projects/ff68ec5b48a94c62a45a8cadcf65b1ec/users/34b9c55f45c940deb3818b3093f48d47/roles/425d4b481c7846e0bc07e7e805d5b15f => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 03:34:57.692946 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f3d9b63-3297-4867-b3f5-80487fb9e7ca None None] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:34:57.693234 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f3d9b63-3297-4867-b3f5-80487fb9e7ca None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:34:57.693621 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f3d9b63-3297-4867-b3f5-80487fb9e7ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:34:57.721997 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.auth.core [None req-7f3d9b63-3297-4867-b3f5-80487fb9e7ca None None] MFA Rules not processed for user `34b9c55f45c940deb3818b3093f48d47`. Rule list: `[]` (Enabled: `True`). {{(pid=81399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 03:34:57.723272 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-7f3d9b63-3297-4867-b3f5-80487fb9e7ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:34:57.735363 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:34:57.799997 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 1421/2835] 10.222.0.18 () {58 vars in 958 bytes} [Tue Sep 13 03:34:57 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 13 03:34:57.811375 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7abec9e1-7ef6-4ff4-898f-523676f0912f admin admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:34:57.854585 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:34:57.864782 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f3d9b63-3297-4867-b3f5-80487fb9e7ca None None] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:34:57.874375 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7abec9e1-7ef6-4ff4-898f-523676f0912f admin admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:34:57.878119 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cb70948-9780-45f8-a27f-e0b9ff3ee181 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '924ee3f608d34a18b0ddc4695f102a56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b9d94f6a68f7470d9494f112175620dc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:34:57.879069 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cb70948-9780-45f8-a27f-e0b9ff3ee181 service cinder] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:34:57.879835 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cb70948-9780-45f8-a27f-e0b9ff3ee181 service cinder] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:34:57.880101 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cb70948-9780-45f8-a27f-e0b9ff3ee181 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:34:57.928591 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f3d9b63-3297-4867-b3f5-80487fb9e7ca None None] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:34:57.933194 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06612e99-9818-4bd8-a6eb-74b70dbb9e40 tempest-VolumeQuotaClassesTest-1926640548 tempest-VolumeQuotaClassesTest-1926640548-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3adf98191ee446f825c636db6da4835', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f4217643b7c842dda363a27d6ecf62d5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:34:57.933801 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06612e99-9818-4bd8-a6eb-74b70dbb9e40 tempest-VolumeQuotaClassesTest-1926640548 tempest-VolumeQuotaClassesTest-1926640548-project-admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:34:57.934127 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06612e99-9818-4bd8-a6eb-74b70dbb9e40 tempest-VolumeQuotaClassesTest-1926640548 tempest-VolumeQuotaClassesTest-1926640548-project-admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:34:57.934404 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06612e99-9818-4bd8-a6eb-74b70dbb9e40 tempest-VolumeQuotaClassesTest-1926640548 tempest-VolumeQuotaClassesTest-1926640548-project-admin] PATH_INFO: `/v3/domains` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:34:57.935077 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06612e99-9818-4bd8-a6eb-74b70dbb9e40 tempest-VolumeQuotaClassesTest-1926640548 tempest-VolumeQuotaClassesTest-1926640548-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=81399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 03:34:57.935381 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06612e99-9818-4bd8-a6eb-74b70dbb9e40 tempest-VolumeQuotaClassesTest-1926640548 tempest-VolumeQuotaClassesTest-1926640548-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:34:57.936863 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cb70948-9780-45f8-a27f-e0b9ff3ee181 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:34:57.936987 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06612e99-9818-4bd8-a6eb-74b70dbb9e40 tempest-VolumeQuotaClassesTest-1926640548 tempest-VolumeQuotaClassesTest-1926640548-project-admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:34:57.938357 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cb70948-9780-45f8-a27f-e0b9ff3ee181 service cinder] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:34:57.948745 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 1415/2836] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Sep 13 03:34:57 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 138 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 03:34:57.949046 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 1422/2837] 10.222.0.18 () {58 vars in 1153 bytes} [Tue Sep 13 03:34:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 03:34:57.953198 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cb70948-9780-45f8-a27f-e0b9ff3ee181 service cinder] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:34:57.964054 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cb70948-9780-45f8-a27f-e0b9ff3ee181 service cinder] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:34:57.968709 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cffab3ed-74c6-4595-8cc4-dc2ab6eacb4c tempest-VolumeQuotaClassesTest-1926640548 tempest-VolumeQuotaClassesTest-1926640548-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3adf98191ee446f825c636db6da4835', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f4217643b7c842dda363a27d6ecf62d5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:34:57.969306 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cffab3ed-74c6-4595-8cc4-dc2ab6eacb4c tempest-VolumeQuotaClassesTest-1926640548 tempest-VolumeQuotaClassesTest-1926640548-project-admin] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:34:57.969586 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cffab3ed-74c6-4595-8cc4-dc2ab6eacb4c tempest-VolumeQuotaClassesTest-1926640548 tempest-VolumeQuotaClassesTest-1926640548-project-admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:34:57.969856 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cffab3ed-74c6-4595-8cc4-dc2ab6eacb4c tempest-VolumeQuotaClassesTest-1926640548 tempest-VolumeQuotaClassesTest-1926640548-project-admin] PATH_INFO: `/v3/projects` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:34:57.970493 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cffab3ed-74c6-4595-8cc4-dc2ab6eacb4c tempest-VolumeQuotaClassesTest-1926640548 tempest-VolumeQuotaClassesTest-1926640548-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:34:57.972417 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cffab3ed-74c6-4595-8cc4-dc2ab6eacb4c tempest-VolumeQuotaClassesTest-1926640548 tempest-VolumeQuotaClassesTest-1926640548-project-admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:34:57.999394 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 1416/2838] 10.222.0.18 () {60 vars in 1153 bytes} [Tue Sep 13 03:34:57 2022] POST /identity/v3/projects => generated 353 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 13 03:34:58.003540 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06612e99-9818-4bd8-a6eb-74b70dbb9e40 tempest-VolumeQuotaClassesTest-1926640548 tempest-VolumeQuotaClassesTest-1926640548-project-admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:34:58.013889 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06612e99-9818-4bd8-a6eb-74b70dbb9e40 tempest-VolumeQuotaClassesTest-1926640548 tempest-VolumeQuotaClassesTest-1926640548-project-admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:34:58.078384 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e7fe3dd-d03e-4539-876f-f6b1b0201e82 tempest-VolumeQuotaClassesTest-1926640548 tempest-VolumeQuotaClassesTest-1926640548-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3adf98191ee446f825c636db6da4835', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f4217643b7c842dda363a27d6ecf62d5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:34:58.079033 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e7fe3dd-d03e-4539-876f-f6b1b0201e82 tempest-VolumeQuotaClassesTest-1926640548 tempest-VolumeQuotaClassesTest-1926640548-project-admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:34:58.079349 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e7fe3dd-d03e-4539-876f-f6b1b0201e82 tempest-VolumeQuotaClassesTest-1926640548 tempest-VolumeQuotaClassesTest-1926640548-project-admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:34:58.079759 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e7fe3dd-d03e-4539-876f-f6b1b0201e82 tempest-VolumeQuotaClassesTest-1926640548 tempest-VolumeQuotaClassesTest-1926640548-project-admin] PATH_INFO: `/v3/domains` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:34:58.080412 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e7fe3dd-d03e-4539-876f-f6b1b0201e82 tempest-VolumeQuotaClassesTest-1926640548 tempest-VolumeQuotaClassesTest-1926640548-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=81399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 03:34:58.080716 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e7fe3dd-d03e-4539-876f-f6b1b0201e82 tempest-VolumeQuotaClassesTest-1926640548 tempest-VolumeQuotaClassesTest-1926640548-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:34:58.082373 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e7fe3dd-d03e-4539-876f-f6b1b0201e82 tempest-VolumeQuotaClassesTest-1926640548 tempest-VolumeQuotaClassesTest-1926640548-project-admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:34:58.095992 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 1423/2839] 10.222.0.18 () {58 vars in 1153 bytes} [Tue Sep 13 03:34:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 93 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 03:34:58.135940 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cffab3ed-74c6-4595-8cc4-dc2ab6eacb4c tempest-VolumeQuotaClassesTest-1926640548 tempest-VolumeQuotaClassesTest-1926640548-project-admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:34:58.146112 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cffab3ed-74c6-4595-8cc4-dc2ab6eacb4c tempest-VolumeQuotaClassesTest-1926640548 tempest-VolumeQuotaClassesTest-1926640548-project-admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:34:58.150764 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d62c6852-d0e0-4067-b3ae-cc47d85984d4 tempest-VolumeQuotaClassesTest-1926640548 tempest-VolumeQuotaClassesTest-1926640548-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3adf98191ee446f825c636db6da4835', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f4217643b7c842dda363a27d6ecf62d5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:34:58.151392 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d62c6852-d0e0-4067-b3ae-cc47d85984d4 tempest-VolumeQuotaClassesTest-1926640548 tempest-VolumeQuotaClassesTest-1926640548-project-admin] REQUEST_METHOD: `DELETE` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:34:58.151673 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d62c6852-d0e0-4067-b3ae-cc47d85984d4 tempest-VolumeQuotaClassesTest-1926640548 tempest-VolumeQuotaClassesTest-1926640548-project-admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:34:58.151947 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d62c6852-d0e0-4067-b3ae-cc47d85984d4 tempest-VolumeQuotaClassesTest-1926640548 tempest-VolumeQuotaClassesTest-1926640548-project-admin] PATH_INFO: `/v3/projects/3bacae8dd5fe465ba2936413d103fcbe` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:34:58.153074 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d62c6852-d0e0-4067-b3ae-cc47d85984d4 tempest-VolumeQuotaClassesTest-1926640548 tempest-VolumeQuotaClassesTest-1926640548-project-admin] RBAC: Authorizing `identity:delete_project(project_id=3bacae8dd5fe465ba2936413d103fcbe)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:34:58.155921 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d62c6852-d0e0-4067-b3ae-cc47d85984d4 tempest-VolumeQuotaClassesTest-1926640548 tempest-VolumeQuotaClassesTest-1926640548-project-admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:34:58.216374 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-d62c6852-d0e0-4067-b3ae-cc47d85984d4 tempest-VolumeQuotaClassesTest-1926640548 tempest-VolumeQuotaClassesTest-1926640548-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3bacae8dd5fe465ba2936413d103fcbe', 'request_id': 'req-d62c6852-d0e0-4067-b3ae-cc47d85984d4'} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:34:58.222565 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-d62c6852-d0e0-4067-b3ae-cc47d85984d4 tempest-VolumeQuotaClassesTest-1926640548 tempest-VolumeQuotaClassesTest-1926640548-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3bacae8dd5fe465ba2936413d103fcbe', 'request_id': 'req-d62c6852-d0e0-4067-b3ae-cc47d85984d4'} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:34:58.230393 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-d62c6852-d0e0-4067-b3ae-cc47d85984d4 tempest-VolumeQuotaClassesTest-1926640548 tempest-VolumeQuotaClassesTest-1926640548-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3bacae8dd5fe465ba2936413d103fcbe', 'request_id': 'req-d62c6852-d0e0-4067-b3ae-cc47d85984d4'} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:34:58.246224 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-d62c6852-d0e0-4067-b3ae-cc47d85984d4 tempest-VolumeQuotaClassesTest-1926640548 tempest-VolumeQuotaClassesTest-1926640548-project-admin] The token cache is being invalidate because project 3bacae8dd5fe465ba2936413d103fcbe was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81400) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 03:34:58.246499 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-d62c6852-d0e0-4067-b3ae-cc47d85984d4 tempest-VolumeQuotaClassesTest-1926640548 tempest-VolumeQuotaClassesTest-1926640548-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:34:58.247246 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-d62c6852-d0e0-4067-b3ae-cc47d85984d4 tempest-VolumeQuotaClassesTest-1926640548 tempest-VolumeQuotaClassesTest-1926640548-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:34:58.249034 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 1417/2840] 10.222.0.18 () {58 vars in 1233 bytes} [Tue Sep 13 03:34:58 2022] DELETE /identity/v3/projects/3bacae8dd5fe465ba2936413d103fcbe => generated 0 bytes in 114 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 03:34:58.437009 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e7fe3dd-d03e-4539-876f-f6b1b0201e82 tempest-VolumeQuotaClassesTest-1926640548 tempest-VolumeQuotaClassesTest-1926640548-project-admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:34:58.438812 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-8e7fe3dd-d03e-4539-876f-f6b1b0201e82 tempest-VolumeQuotaClassesTest-1926640548 tempest-VolumeQuotaClassesTest-1926640548-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:34:58.479211 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=100 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:34:58.501951 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e7fe3dd-d03e-4539-876f-f6b1b0201e82 tempest-VolumeQuotaClassesTest-1926640548 tempest-VolumeQuotaClassesTest-1926640548-project-admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:34:58.555608 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91077048-a0d2-4f0c-9218-edcadb4f4c18 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76c5181fc7be45279ac00017bb4640b3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:34:58.556226 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91077048-a0d2-4f0c-9218-edcadb4f4c18 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:34:58.556500 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91077048-a0d2-4f0c-9218-edcadb4f4c18 admin admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:34:58.556766 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91077048-a0d2-4f0c-9218-edcadb4f4c18 admin admin] PATH_INFO: `/v3/users/bb9b6445f2fc482496009f5922446e21` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:34:58.557862 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91077048-a0d2-4f0c-9218-edcadb4f4c18 admin admin] RBAC: Authorizing `identity:delete_user(user_id=bb9b6445f2fc482496009f5922446e21)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:34:58.559804 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91077048-a0d2-4f0c-9218-edcadb4f4c18 admin admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:34:58.604716 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-91077048-a0d2-4f0c-9218-edcadb4f4c18 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bb9b6445f2fc482496009f5922446e21'} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:34:58.612051 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-91077048-a0d2-4f0c-9218-edcadb4f4c18 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bb9b6445f2fc482496009f5922446e21'} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:34:58.624615 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-91077048-a0d2-4f0c-9218-edcadb4f4c18 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bb9b6445f2fc482496009f5922446e21'} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:34:58.625257 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-91077048-a0d2-4f0c-9218-edcadb4f4c18 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bb9b6445f2fc482496009f5922446e21'} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:34:58.632089 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-91077048-a0d2-4f0c-9218-edcadb4f4c18 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bb9b6445f2fc482496009f5922446e21'} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:34:58.638207 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 1424/2841] 10.222.0.18 () {58 vars in 1224 bytes} [Tue Sep 13 03:34:58 2022] DELETE /identity/v3/users/bb9b6445f2fc482496009f5922446e21 => generated 0 bytes in 202 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 03:34:58.642953 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc53d9bb-b8dc-4806-94d0-95c9faa059fa None None] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:34:58.643439 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc53d9bb-b8dc-4806-94d0-95c9faa059fa None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:34:58.643545 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc53d9bb-b8dc-4806-94d0-95c9faa059fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:34:58.662456 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.auth.core [None req-cc53d9bb-b8dc-4806-94d0-95c9faa059fa None None] MFA Rules not processed for user `49f280ab086b4610855c4ffa378517d9`. Rule list: `[]` (Enabled: `True`). {{(pid=81400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 03:34:58.663656 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-cc53d9bb-b8dc-4806-94d0-95c9faa059fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:34:58.708126 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:34:58.724653 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 1418/2842] 10.222.0.18 () {58 vars in 958 bytes} [Tue Sep 13 03:34:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 03:34:58.734289 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91077048-a0d2-4f0c-9218-edcadb4f4c18 admin admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:34:58.735831 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-91077048-a0d2-4f0c-9218-edcadb4f4c18 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:34:58.792591 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91077048-a0d2-4f0c-9218-edcadb4f4c18 admin admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:34:58.796270 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc0388bb-5704-4bfd-bff5-4e8dce5653bc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdffbc851086441ebd01022d1e4c2609', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b9d94f6a68f7470d9494f112175620dc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:34:58.796866 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc0388bb-5704-4bfd-bff5-4e8dce5653bc service neutron] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:34:58.797145 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc0388bb-5704-4bfd-bff5-4e8dce5653bc service neutron] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:34:58.797419 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc0388bb-5704-4bfd-bff5-4e8dce5653bc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:34:58.806963 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc0388bb-5704-4bfd-bff5-4e8dce5653bc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:34:58.808401 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc0388bb-5704-4bfd-bff5-4e8dce5653bc service neutron] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:34:58.817737 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 1425/2843] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Sep 13 03:34:58 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 03:34:59.104076 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc53d9bb-b8dc-4806-94d0-95c9faa059fa None None] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:34:59.106397 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-cc53d9bb-b8dc-4806-94d0-95c9faa059fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:34:59.120820 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc53d9bb-b8dc-4806-94d0-95c9faa059fa None None] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:34:59.128206 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73494f5d-55bc-41e2-abc3-a8d2c18a6ffd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76c5181fc7be45279ac00017bb4640b3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:34:59.128854 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73494f5d-55bc-41e2-abc3-a8d2c18a6ffd admin admin] REQUEST_METHOD: `DELETE` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:34:59.129132 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73494f5d-55bc-41e2-abc3-a8d2c18a6ffd admin admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:34:59.129403 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73494f5d-55bc-41e2-abc3-a8d2c18a6ffd admin admin] PATH_INFO: `/v3/projects/d332ca9b985e4ba692f509b669f69970` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:34:59.130579 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73494f5d-55bc-41e2-abc3-a8d2c18a6ffd admin admin] RBAC: Authorizing `identity:delete_project(project_id=d332ca9b985e4ba692f509b669f69970)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:34:59.132518 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73494f5d-55bc-41e2-abc3-a8d2c18a6ffd admin admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:34:59.183130 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-73494f5d-55bc-41e2-abc3-a8d2c18a6ffd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd332ca9b985e4ba692f509b669f69970', 'request_id': 'req-73494f5d-55bc-41e2-abc3-a8d2c18a6ffd'} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:34:59.189260 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-73494f5d-55bc-41e2-abc3-a8d2c18a6ffd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd332ca9b985e4ba692f509b669f69970', 'request_id': 'req-73494f5d-55bc-41e2-abc3-a8d2c18a6ffd'} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:34:59.196316 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-73494f5d-55bc-41e2-abc3-a8d2c18a6ffd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd332ca9b985e4ba692f509b669f69970', 'request_id': 'req-73494f5d-55bc-41e2-abc3-a8d2c18a6ffd'} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:34:59.210431 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-73494f5d-55bc-41e2-abc3-a8d2c18a6ffd admin admin] The token cache is being invalidate because project d332ca9b985e4ba692f509b669f69970 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81400) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 03:34:59.210754 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-73494f5d-55bc-41e2-abc3-a8d2c18a6ffd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:34:59.211372 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-73494f5d-55bc-41e2-abc3-a8d2c18a6ffd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:34:59.212899 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 1419/2844] 10.222.0.18 () {58 vars in 1233 bytes} [Tue Sep 13 03:34:59 2022] DELETE /identity/v3/projects/d332ca9b985e4ba692f509b669f69970 => generated 0 bytes in 110 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 03:34:59.217126 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc0388bb-5704-4bfd-bff5-4e8dce5653bc service neutron] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:34:59.227132 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc0388bb-5704-4bfd-bff5-4e8dce5653bc service neutron] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:34:59.230976 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:34:59.296540 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43368fc3-fabb-4fb4-921c-0d8e57cccabc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76c5181fc7be45279ac00017bb4640b3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:34:59.297179 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43368fc3-fabb-4fb4-921c-0d8e57cccabc admin admin] REQUEST_METHOD: `DELETE` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:34:59.297462 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43368fc3-fabb-4fb4-921c-0d8e57cccabc admin admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:34:59.297730 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43368fc3-fabb-4fb4-921c-0d8e57cccabc admin admin] PATH_INFO: `/v3/users/e3adf98191ee446f825c636db6da4835` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:34:59.298931 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43368fc3-fabb-4fb4-921c-0d8e57cccabc admin admin] RBAC: Authorizing `identity:delete_user(user_id=e3adf98191ee446f825c636db6da4835)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:34:59.300851 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43368fc3-fabb-4fb4-921c-0d8e57cccabc admin admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:34:59.364499 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-43368fc3-fabb-4fb4-921c-0d8e57cccabc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e3adf98191ee446f825c636db6da4835'} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:34:59.380703 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-43368fc3-fabb-4fb4-921c-0d8e57cccabc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e3adf98191ee446f825c636db6da4835'} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:34:59.397811 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-43368fc3-fabb-4fb4-921c-0d8e57cccabc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e3adf98191ee446f825c636db6da4835'} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:34:59.398606 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-43368fc3-fabb-4fb4-921c-0d8e57cccabc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e3adf98191ee446f825c636db6da4835'} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:34:59.408132 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-43368fc3-fabb-4fb4-921c-0d8e57cccabc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e3adf98191ee446f825c636db6da4835'} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:34:59.425135 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 1426/2845] 10.222.0.18 () {58 vars in 1224 bytes} [Tue Sep 13 03:34:59 2022] DELETE /identity/v3/users/e3adf98191ee446f825c636db6da4835 => generated 0 bytes in 206 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 03:34:59.442410 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73494f5d-55bc-41e2-abc3-a8d2c18a6ffd admin admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:34:59.444596 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-73494f5d-55bc-41e2-abc3-a8d2c18a6ffd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:34:59.507143 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43368fc3-fabb-4fb4-921c-0d8e57cccabc admin admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:34:59.509495 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-43368fc3-fabb-4fb4-921c-0d8e57cccabc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:34:59.560700 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73494f5d-55bc-41e2-abc3-a8d2c18a6ffd admin admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:34:59.604608 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43368fc3-fabb-4fb4-921c-0d8e57cccabc admin admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:34:59.611112 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24310de1-f123-487d-9f20-ce3656d5788e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '924ee3f608d34a18b0ddc4695f102a56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b9d94f6a68f7470d9494f112175620dc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:34:59.616084 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24310de1-f123-487d-9f20-ce3656d5788e service cinder] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:34:59.616084 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24310de1-f123-487d-9f20-ce3656d5788e service cinder] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:34:59.616084 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24310de1-f123-487d-9f20-ce3656d5788e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:34:59.620121 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-24310de1-f123-487d-9f20-ce3656d5788e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:34:59.655895 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f38beea6-1243-4b0c-a939-2346497b68b0 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7377f5bee4a49959182e1bd34bc97a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b9d94f6a68f7470d9494f112175620dc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:34:59.656723 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f38beea6-1243-4b0c-a939-2346497b68b0 service nova] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:34:59.657287 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f38beea6-1243-4b0c-a939-2346497b68b0 service nova] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:34:59.657555 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f38beea6-1243-4b0c-a939-2346497b68b0 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:34:59.659889 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-f38beea6-1243-4b0c-a939-2346497b68b0 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:34:59.677893 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f38beea6-1243-4b0c-a939-2346497b68b0 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:34:59.678988 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f38beea6-1243-4b0c-a939-2346497b68b0 service nova] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:34:59.688593 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:34:59.697270 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 1420/2846] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Sep 13 03:34:59 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 256 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 13 03:34:59.720392 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:34:59.747589 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24310de1-f123-487d-9f20-ce3656d5788e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:34:59.749207 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24310de1-f123-487d-9f20-ce3656d5788e service cinder] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:34:59.760530 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 1427/2847] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Sep 13 03:34:59 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 254 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 03:34:59.927578 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f38beea6-1243-4b0c-a939-2346497b68b0 service nova] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:34:59.929747 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-f38beea6-1243-4b0c-a939-2346497b68b0 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:35:00.024737 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f38beea6-1243-4b0c-a939-2346497b68b0 service nova] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:35:00.029972 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24506e20-6e9b-4484-95f2-ef48ceda9091 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76c5181fc7be45279ac00017bb4640b3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:35:00.030851 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24506e20-6e9b-4484-95f2-ef48ceda9091 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:35:00.031086 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24506e20-6e9b-4484-95f2-ef48ceda9091 admin admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:35:00.031404 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24506e20-6e9b-4484-95f2-ef48ceda9091 admin admin] PATH_INFO: `/v3/projects/f4217643b7c842dda363a27d6ecf62d5` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:35:00.032551 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24506e20-6e9b-4484-95f2-ef48ceda9091 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f4217643b7c842dda363a27d6ecf62d5)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:35:00.034575 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24506e20-6e9b-4484-95f2-ef48ceda9091 admin admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:35:00.093856 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-24506e20-6e9b-4484-95f2-ef48ceda9091 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f4217643b7c842dda363a27d6ecf62d5', 'request_id': 'req-24506e20-6e9b-4484-95f2-ef48ceda9091'} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:35:00.101481 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-24506e20-6e9b-4484-95f2-ef48ceda9091 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f4217643b7c842dda363a27d6ecf62d5', 'request_id': 'req-24506e20-6e9b-4484-95f2-ef48ceda9091'} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:35:00.109543 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-24506e20-6e9b-4484-95f2-ef48ceda9091 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f4217643b7c842dda363a27d6ecf62d5', 'request_id': 'req-24506e20-6e9b-4484-95f2-ef48ceda9091'} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:35:00.124533 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-24506e20-6e9b-4484-95f2-ef48ceda9091 admin admin] The token cache is being invalidate because project f4217643b7c842dda363a27d6ecf62d5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81400) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 03:35:00.124824 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-24506e20-6e9b-4484-95f2-ef48ceda9091 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:35:00.125592 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-24506e20-6e9b-4484-95f2-ef48ceda9091 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:35:00.127158 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 1421/2848] 10.222.0.18 () {58 vars in 1233 bytes} [Tue Sep 13 03:34:59 2022] DELETE /identity/v3/projects/f4217643b7c842dda363a27d6ecf62d5 => generated 0 bytes in 201 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 03:35:00.132527 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f71677ad-2974-4db9-bc9e-4fee5ec0009d None None] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:35:00.132527 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f71677ad-2974-4db9-bc9e-4fee5ec0009d None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:35:00.132846 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f71677ad-2974-4db9-bc9e-4fee5ec0009d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:35:00.152486 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.auth.core [None req-f71677ad-2974-4db9-bc9e-4fee5ec0009d None None] MFA Rules not processed for user `49f280ab086b4610855c4ffa378517d9`. Rule list: `[]` (Enabled: `True`). {{(pid=81399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 03:35:00.153888 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-f71677ad-2974-4db9-bc9e-4fee5ec0009d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:35:00.232149 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 1428/2849] 10.222.0.18 () {58 vars in 958 bytes} [Tue Sep 13 03:35:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 13 03:35:00.234179 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f16e309-8f42-4ad6-9062-1d1b0465a439 None None] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:35:00.234471 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f16e309-8f42-4ad6-9062-1d1b0465a439 None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:35:00.235338 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f16e309-8f42-4ad6-9062-1d1b0465a439 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:35:00.263234 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.auth.core [None req-4f16e309-8f42-4ad6-9062-1d1b0465a439 None None] MFA Rules not processed for user `49f280ab086b4610855c4ffa378517d9`. Rule list: `[]` (Enabled: `True`). {{(pid=81400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 03:35:00.265164 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-4f16e309-8f42-4ad6-9062-1d1b0465a439 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:35:00.279750 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 1422/2850] 10.222.0.18 () {58 vars in 958 bytes} [Tue Sep 13 03:35:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 46 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 03:35:00.280529 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f71677ad-2974-4db9-bc9e-4fee5ec0009d None None] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:35:00.293652 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f71677ad-2974-4db9-bc9e-4fee5ec0009d None None] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:35:00.298517 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97959c61-8b3a-43b3-8db9-711b6fdb1be7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76c5181fc7be45279ac00017bb4640b3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:35:00.299088 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97959c61-8b3a-43b3-8db9-711b6fdb1be7 admin admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:35:00.299367 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97959c61-8b3a-43b3-8db9-711b6fdb1be7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:35:00.299804 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97959c61-8b3a-43b3-8db9-711b6fdb1be7 admin admin] PATH_INFO: `/v3/domains` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:35:00.300548 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97959c61-8b3a-43b3-8db9-711b6fdb1be7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 03:35:00.300860 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97959c61-8b3a-43b3-8db9-711b6fdb1be7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:35:00.303453 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97959c61-8b3a-43b3-8db9-711b6fdb1be7 admin admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:35:00.315638 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 1429/2851] 10.222.0.18 () {58 vars in 1153 bytes} [Tue Sep 13 03:35:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 03:35:00.320145 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f16e309-8f42-4ad6-9062-1d1b0465a439 None None] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:35:00.329791 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f16e309-8f42-4ad6-9062-1d1b0465a439 None None] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:35:00.335758 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d0aa659-36fb-4741-96d6-34469ab104e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76c5181fc7be45279ac00017bb4640b3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:35:00.337336 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d0aa659-36fb-4741-96d6-34469ab104e3 admin admin] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:35:00.337998 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d0aa659-36fb-4741-96d6-34469ab104e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:35:00.337998 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d0aa659-36fb-4741-96d6-34469ab104e3 admin admin] PATH_INFO: `/v3/projects` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:35:00.338708 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d0aa659-36fb-4741-96d6-34469ab104e3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:35:00.340853 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d0aa659-36fb-4741-96d6-34469ab104e3 admin admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:35:00.374570 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 1423/2852] 10.222.0.18 () {60 vars in 1153 bytes} [Tue Sep 13 03:35:00 2022] POST /identity/v3/projects => generated 373 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 03:35:00.375840 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97959c61-8b3a-43b3-8db9-711b6fdb1be7 admin admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:35:00.387400 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97959c61-8b3a-43b3-8db9-711b6fdb1be7 admin admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:35:00.456108 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d0aa659-36fb-4741-96d6-34469ab104e3 admin admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:35:00.458484 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-6d0aa659-36fb-4741-96d6-34469ab104e3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:35:00.461211 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a75b13b-8410-417a-a240-077d9483b303 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76c5181fc7be45279ac00017bb4640b3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:35:00.461756 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a75b13b-8410-417a-a240-077d9483b303 admin admin] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:35:00.462034 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a75b13b-8410-417a-a240-077d9483b303 admin admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:35:00.462446 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a75b13b-8410-417a-a240-077d9483b303 admin admin] PATH_INFO: `/v3/users` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:35:00.463191 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a75b13b-8410-417a-a240-077d9483b303 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:35:00.465123 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a75b13b-8410-417a-a240-077d9483b303 admin admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:35:00.490260 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 1430/2853] 10.222.0.18 () {60 vars in 1144 bytes} [Tue Sep 13 03:35:00 2022] POST /identity/v3/users => generated 287 bytes in 115 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 03:35:00.494077 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a75b13b-8410-417a-a240-077d9483b303 admin admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:35:00.504754 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a75b13b-8410-417a-a240-077d9483b303 admin admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:35:00.509614 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5adcd522-4d67-4384-8a45-ffd7252f2d48 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76c5181fc7be45279ac00017bb4640b3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:35:00.510219 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5adcd522-4d67-4384-8a45-ffd7252f2d48 admin admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:35:00.510494 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5adcd522-4d67-4384-8a45-ffd7252f2d48 admin admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:35:00.510874 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5adcd522-4d67-4384-8a45-ffd7252f2d48 admin admin] PATH_INFO: `/v3/roles` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:35:00.511429 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5adcd522-4d67-4384-8a45-ffd7252f2d48 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:35:00.513066 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5adcd522-4d67-4384-8a45-ffd7252f2d48 admin admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:35:00.525586 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d0aa659-36fb-4741-96d6-34469ab104e3 admin admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:35:00.526961 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 1431/2854] 10.222.0.18 () {58 vars in 1122 bytes} [Tue Sep 13 03:35:00 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 13 03:35:00.530420 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5adcd522-4d67-4384-8a45-ffd7252f2d48 admin admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:35:00.533085 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43dcd949-7e16-4bf8-b5e1-5baa0b3708c0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '924ee3f608d34a18b0ddc4695f102a56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b9d94f6a68f7470d9494f112175620dc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:35:00.533437 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43dcd949-7e16-4bf8-b5e1-5baa0b3708c0 service cinder] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:35:00.533717 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43dcd949-7e16-4bf8-b5e1-5baa0b3708c0 service cinder] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:35:00.533985 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43dcd949-7e16-4bf8-b5e1-5baa0b3708c0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:35:00.536685 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-43dcd949-7e16-4bf8-b5e1-5baa0b3708c0 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:35:00.543187 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5adcd522-4d67-4384-8a45-ffd7252f2d48 admin admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:35:00.548281 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a8fb8e9-2c7b-419b-aecb-4e27e5c77fa0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76c5181fc7be45279ac00017bb4640b3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:35:00.548985 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a8fb8e9-2c7b-419b-aecb-4e27e5c77fa0 admin admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:35:00.549421 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a8fb8e9-2c7b-419b-aecb-4e27e5c77fa0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:35:00.549790 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a8fb8e9-2c7b-419b-aecb-4e27e5c77fa0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:35:00.550653 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a8fb8e9-2c7b-419b-aecb-4e27e5c77fa0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:35:00.552278 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a8fb8e9-2c7b-419b-aecb-4e27e5c77fa0 admin admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:35:00.553857 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:35:00.567572 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 1432/2855] 10.222.0.18 () {58 vars in 1122 bytes} [Tue Sep 13 03:35:00 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 03:35:00.569344 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a8fb8e9-2c7b-419b-aecb-4e27e5c77fa0 admin admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:35:00.585376 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a8fb8e9-2c7b-419b-aecb-4e27e5c77fa0 admin admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:35:00.591863 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7877f3d7-4eb0-4362-af47-3bd15f2b4a0a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76c5181fc7be45279ac00017bb4640b3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:35:00.592687 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7877f3d7-4eb0-4362-af47-3bd15f2b4a0a admin admin] REQUEST_METHOD: `PUT` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:35:00.593274 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7877f3d7-4eb0-4362-af47-3bd15f2b4a0a admin admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:35:00.593551 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7877f3d7-4eb0-4362-af47-3bd15f2b4a0a admin admin] PATH_INFO: `/v3/projects/3903eed3df354b8a8e84c4f12d740dba/users/452bed8c90ea4ab8957a8da46f9acc4a/roles/1eb7a5b5203945c29e6d22d653f47847` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:35:00.610234 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7877f3d7-4eb0-4362-af47-3bd15f2b4a0a admin admin] RBAC: Authorizing `identity:create_grant(project_id=3903eed3df354b8a8e84c4f12d740dba, user_id=452bed8c90ea4ab8957a8da46f9acc4a, role_id=1eb7a5b5203945c29e6d22d653f47847)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:35:00.613339 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7877f3d7-4eb0-4362-af47-3bd15f2b4a0a admin admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:35:00.617600 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43dcd949-7e16-4bf8-b5e1-5baa0b3708c0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:35:00.618520 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43dcd949-7e16-4bf8-b5e1-5baa0b3708c0 service cinder] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:35:00.628945 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 1424/2856] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Sep 13 03:35:00 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 173 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 13 03:35:00.631433 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 1433/2857] 10.222.0.18 () {60 vars in 1483 bytes} [Tue Sep 13 03:35:00 2022] PUT /identity/v3/projects/3903eed3df354b8a8e84c4f12d740dba/users/452bed8c90ea4ab8957a8da46f9acc4a/roles/1eb7a5b5203945c29e6d22d653f47847 => generated 0 bytes in 63 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 03:35:00.645607 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b24539-f6e7-44f8-afa5-86c92293ed21 None None] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:35:00.646347 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b24539-f6e7-44f8-afa5-86c92293ed21 None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:35:00.646709 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b24539-f6e7-44f8-afa5-86c92293ed21 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:35:00.680759 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.auth.core [None req-60b24539-f6e7-44f8-afa5-86c92293ed21 None None] MFA Rules not processed for user `452bed8c90ea4ab8957a8da46f9acc4a`. Rule list: `[]` (Enabled: `True`). {{(pid=81400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 03:35:00.682731 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-60b24539-f6e7-44f8-afa5-86c92293ed21 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:35:00.766546 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 1425/2858] 10.222.0.18 () {58 vars in 958 bytes} [Tue Sep 13 03:35:00 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 03:35:00.777552 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7877f3d7-4eb0-4362-af47-3bd15f2b4a0a admin admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:35:00.791146 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:35:00.840399 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7877f3d7-4eb0-4362-af47-3bd15f2b4a0a admin admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:35:00.844202 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb8cf6ac-7c95-4397-859f-3b594a793f92 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '924ee3f608d34a18b0ddc4695f102a56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b9d94f6a68f7470d9494f112175620dc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:35:00.844905 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb8cf6ac-7c95-4397-859f-3b594a793f92 service cinder] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:35:00.845192 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb8cf6ac-7c95-4397-859f-3b594a793f92 service cinder] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:35:00.845464 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb8cf6ac-7c95-4397-859f-3b594a793f92 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:35:00.857687 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb8cf6ac-7c95-4397-859f-3b594a793f92 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:35:00.859284 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb8cf6ac-7c95-4397-859f-3b594a793f92 service cinder] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:35:00.873141 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 1434/2859] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Sep 13 03:35:00 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 03:35:09.804602 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60b24539-f6e7-44f8-afa5-86c92293ed21 None None] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:35:09.806042 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-60b24539-f6e7-44f8-afa5-86c92293ed21 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:35:09.881307 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60b24539-f6e7-44f8-afa5-86c92293ed21 None None] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:35:09.886938 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97eca27b-268c-40d2-91ad-1a169c4e2642 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7377f5bee4a49959182e1bd34bc97a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b9d94f6a68f7470d9494f112175620dc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:35:09.887482 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97eca27b-268c-40d2-91ad-1a169c4e2642 service nova] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:35:09.887762 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97eca27b-268c-40d2-91ad-1a169c4e2642 service nova] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:35:09.888031 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97eca27b-268c-40d2-91ad-1a169c4e2642 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:35:09.889770 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-97eca27b-268c-40d2-91ad-1a169c4e2642 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:35:09.925237 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:35:09.950428 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97eca27b-268c-40d2-91ad-1a169c4e2642 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:35:09.952077 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97eca27b-268c-40d2-91ad-1a169c4e2642 service nova] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:35:09.961633 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 1426/2860] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Sep 13 03:35:09 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 158 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 03:35:10.867240 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=21 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:35:17.155303 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb8cf6ac-7c95-4397-859f-3b594a793f92 service cinder] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:35:17.156839 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-cb8cf6ac-7c95-4397-859f-3b594a793f92 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:35:17.223672 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb8cf6ac-7c95-4397-859f-3b594a793f92 service cinder] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:35:17.228258 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e11cbf65-e3b8-41d4-92e3-40f213861a1d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76c5181fc7be45279ac00017bb4640b3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:35:17.228875 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e11cbf65-e3b8-41d4-92e3-40f213861a1d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:35:17.229148 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e11cbf65-e3b8-41d4-92e3-40f213861a1d admin admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:35:17.229415 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e11cbf65-e3b8-41d4-92e3-40f213861a1d admin admin] PATH_INFO: `/v3/users/155ed81d5c9a4e28ba2c6afd1b34fcf6` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:35:17.230517 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e11cbf65-e3b8-41d4-92e3-40f213861a1d admin admin] RBAC: Authorizing `identity:delete_user(user_id=155ed81d5c9a4e28ba2c6afd1b34fcf6)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:35:17.232436 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e11cbf65-e3b8-41d4-92e3-40f213861a1d admin admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:35:17.278541 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-e11cbf65-e3b8-41d4-92e3-40f213861a1d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '155ed81d5c9a4e28ba2c6afd1b34fcf6'} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:35:17.287011 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-e11cbf65-e3b8-41d4-92e3-40f213861a1d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '155ed81d5c9a4e28ba2c6afd1b34fcf6'} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:35:17.299246 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-e11cbf65-e3b8-41d4-92e3-40f213861a1d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '155ed81d5c9a4e28ba2c6afd1b34fcf6'} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:35:17.299882 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-e11cbf65-e3b8-41d4-92e3-40f213861a1d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '155ed81d5c9a4e28ba2c6afd1b34fcf6'} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:35:17.308157 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-e11cbf65-e3b8-41d4-92e3-40f213861a1d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '155ed81d5c9a4e28ba2c6afd1b34fcf6'} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:35:17.314280 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 1435/2861] 10.222.0.18 () {58 vars in 1224 bytes} [Tue Sep 13 03:35:17 2022] DELETE /identity/v3/users/155ed81d5c9a4e28ba2c6afd1b34fcf6 => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 03:35:17.319262 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad48382-470b-4617-8ffa-790d43a44b0b None None] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:35:17.319750 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad48382-470b-4617-8ffa-790d43a44b0b None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:35:17.319868 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad48382-470b-4617-8ffa-790d43a44b0b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:35:17.339726 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.auth.core [None req-6ad48382-470b-4617-8ffa-790d43a44b0b None None] MFA Rules not processed for user `49f280ab086b4610855c4ffa378517d9`. Rule list: `[]` (Enabled: `True`). {{(pid=81400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 03:35:17.340913 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-6ad48382-470b-4617-8ffa-790d43a44b0b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:35:17.403047 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 1427/2862] 10.222.0.18 () {58 vars in 958 bytes} [Tue Sep 13 03:35:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 03:35:17.412994 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e11cbf65-e3b8-41d4-92e3-40f213861a1d admin admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:35:17.414493 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-e11cbf65-e3b8-41d4-92e3-40f213861a1d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:35:17.472960 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e11cbf65-e3b8-41d4-92e3-40f213861a1d admin admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:35:17.476948 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9359291-aa98-4c5b-a062-2fafa5a2b385 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdffbc851086441ebd01022d1e4c2609', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b9d94f6a68f7470d9494f112175620dc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:35:17.477553 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9359291-aa98-4c5b-a062-2fafa5a2b385 service neutron] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:35:17.477827 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9359291-aa98-4c5b-a062-2fafa5a2b385 service neutron] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:35:17.480766 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9359291-aa98-4c5b-a062-2fafa5a2b385 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:35:17.491927 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9359291-aa98-4c5b-a062-2fafa5a2b385 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:35:17.493429 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9359291-aa98-4c5b-a062-2fafa5a2b385 service neutron] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:35:17.502838 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 1436/2863] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Sep 13 03:35:17 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 03:35:17.773095 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ad48382-470b-4617-8ffa-790d43a44b0b None None] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:35:17.774601 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-6ad48382-470b-4617-8ffa-790d43a44b0b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:35:17.787331 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ad48382-470b-4617-8ffa-790d43a44b0b None None] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:35:17.791941 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c70d342a-b497-4df7-84c3-3f9f1be8eb73 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76c5181fc7be45279ac00017bb4640b3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:35:17.792562 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c70d342a-b497-4df7-84c3-3f9f1be8eb73 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:35:17.792838 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c70d342a-b497-4df7-84c3-3f9f1be8eb73 admin admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:35:17.793101 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c70d342a-b497-4df7-84c3-3f9f1be8eb73 admin admin] PATH_INFO: `/v3/projects/1423d25ff51a4eb3810e409baed076f0` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:35:17.794569 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c70d342a-b497-4df7-84c3-3f9f1be8eb73 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1423d25ff51a4eb3810e409baed076f0)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:35:17.796492 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c70d342a-b497-4df7-84c3-3f9f1be8eb73 admin admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:35:17.843684 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-c70d342a-b497-4df7-84c3-3f9f1be8eb73 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1423d25ff51a4eb3810e409baed076f0', 'request_id': 'req-c70d342a-b497-4df7-84c3-3f9f1be8eb73'} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:35:17.849359 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-c70d342a-b497-4df7-84c3-3f9f1be8eb73 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1423d25ff51a4eb3810e409baed076f0', 'request_id': 'req-c70d342a-b497-4df7-84c3-3f9f1be8eb73'} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:35:17.855417 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-c70d342a-b497-4df7-84c3-3f9f1be8eb73 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1423d25ff51a4eb3810e409baed076f0', 'request_id': 'req-c70d342a-b497-4df7-84c3-3f9f1be8eb73'} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:35:17.868857 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-c70d342a-b497-4df7-84c3-3f9f1be8eb73 admin admin] The token cache is being invalidate because project 1423d25ff51a4eb3810e409baed076f0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81400) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 03:35:17.869145 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-c70d342a-b497-4df7-84c3-3f9f1be8eb73 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:35:17.869768 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-c70d342a-b497-4df7-84c3-3f9f1be8eb73 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:35:17.871308 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 1428/2864] 10.222.0.18 () {58 vars in 1233 bytes} [Tue Sep 13 03:35:17 2022] DELETE /identity/v3/projects/1423d25ff51a4eb3810e409baed076f0 => generated 0 bytes in 99 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 03:35:17.875396 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9359291-aa98-4c5b-a062-2fafa5a2b385 service neutron] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:35:17.885566 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9359291-aa98-4c5b-a062-2fafa5a2b385 service neutron] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:35:17.903163 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:35:17.943155 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c45ac70-8434-4218-bf45-e74010bf7cf8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76c5181fc7be45279ac00017bb4640b3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:35:17.943778 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c45ac70-8434-4218-bf45-e74010bf7cf8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:35:17.944054 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c45ac70-8434-4218-bf45-e74010bf7cf8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:35:17.944324 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c45ac70-8434-4218-bf45-e74010bf7cf8 admin admin] PATH_INFO: `/v3/users/c900a8f794f042798a22b8104f9d9fde` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:35:17.945424 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c45ac70-8434-4218-bf45-e74010bf7cf8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c900a8f794f042798a22b8104f9d9fde)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:35:17.947361 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c45ac70-8434-4218-bf45-e74010bf7cf8 admin admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:35:17.993905 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-6c45ac70-8434-4218-bf45-e74010bf7cf8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c900a8f794f042798a22b8104f9d9fde'} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:35:18.001356 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-6c45ac70-8434-4218-bf45-e74010bf7cf8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c900a8f794f042798a22b8104f9d9fde'} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:35:18.013150 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-6c45ac70-8434-4218-bf45-e74010bf7cf8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c900a8f794f042798a22b8104f9d9fde'} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:35:18.013787 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-6c45ac70-8434-4218-bf45-e74010bf7cf8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c900a8f794f042798a22b8104f9d9fde'} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:35:18.021201 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-6c45ac70-8434-4218-bf45-e74010bf7cf8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c900a8f794f042798a22b8104f9d9fde'} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:35:18.027052 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 1437/2865] 10.222.0.18 () {58 vars in 1224 bytes} [Tue Sep 13 03:35:17 2022] DELETE /identity/v3/users/c900a8f794f042798a22b8104f9d9fde => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 03:35:18.297359 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c70d342a-b497-4df7-84c3-3f9f1be8eb73 admin admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:35:18.298796 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-c70d342a-b497-4df7-84c3-3f9f1be8eb73 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:35:18.313794 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c70d342a-b497-4df7-84c3-3f9f1be8eb73 admin admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:35:18.370923 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1e78041-f327-46ea-9f80-653825ce0b98 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76c5181fc7be45279ac00017bb4640b3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:35:18.371609 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e78041-f327-46ea-9f80-653825ce0b98 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:35:18.371827 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e78041-f327-46ea-9f80-653825ce0b98 admin admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:35:18.372103 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e78041-f327-46ea-9f80-653825ce0b98 admin admin] PATH_INFO: `/v3/projects/6e16a1e859034e9abdce7c22596b9f38` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:35:18.373182 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1e78041-f327-46ea-9f80-653825ce0b98 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6e16a1e859034e9abdce7c22596b9f38)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:35:18.375181 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1e78041-f327-46ea-9f80-653825ce0b98 admin admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:35:18.376552 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:35:18.427884 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-d1e78041-f327-46ea-9f80-653825ce0b98 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6e16a1e859034e9abdce7c22596b9f38', 'request_id': 'req-d1e78041-f327-46ea-9f80-653825ce0b98'} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:35:18.433283 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-d1e78041-f327-46ea-9f80-653825ce0b98 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6e16a1e859034e9abdce7c22596b9f38', 'request_id': 'req-d1e78041-f327-46ea-9f80-653825ce0b98'} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:35:18.439792 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-d1e78041-f327-46ea-9f80-653825ce0b98 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6e16a1e859034e9abdce7c22596b9f38', 'request_id': 'req-d1e78041-f327-46ea-9f80-653825ce0b98'} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:35:18.451467 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-d1e78041-f327-46ea-9f80-653825ce0b98 admin admin] The token cache is being invalidate because project 6e16a1e859034e9abdce7c22596b9f38 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81400) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 03:35:18.451759 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-d1e78041-f327-46ea-9f80-653825ce0b98 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:35:18.452373 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-d1e78041-f327-46ea-9f80-653825ce0b98 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:35:18.453914 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 1429/2866] 10.222.0.18 () {58 vars in 1233 bytes} [Tue Sep 13 03:35:18 2022] DELETE /identity/v3/projects/6e16a1e859034e9abdce7c22596b9f38 => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 03:35:18.458427 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c45ac70-8434-4218-bf45-e74010bf7cf8 admin admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:35:18.459962 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-6c45ac70-8434-4218-bf45-e74010bf7cf8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:35:18.472443 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c45ac70-8434-4218-bf45-e74010bf7cf8 admin admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:35:18.527514 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c3aeabd-6c95-4114-aeac-6cc9e9c008c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76c5181fc7be45279ac00017bb4640b3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:35:18.528147 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c3aeabd-6c95-4114-aeac-6cc9e9c008c6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:35:18.528426 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c3aeabd-6c95-4114-aeac-6cc9e9c008c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:35:18.528696 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c3aeabd-6c95-4114-aeac-6cc9e9c008c6 admin admin] PATH_INFO: `/v3/users/34b9c55f45c940deb3818b3093f48d47` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:35:18.529799 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c3aeabd-6c95-4114-aeac-6cc9e9c008c6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=34b9c55f45c940deb3818b3093f48d47)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:35:18.531780 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c3aeabd-6c95-4114-aeac-6cc9e9c008c6 admin admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:35:18.574407 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=83,keystone:DELETE=16,keystone:INSERT=1 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:35:18.577238 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-6c3aeabd-6c95-4114-aeac-6cc9e9c008c6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '34b9c55f45c940deb3818b3093f48d47'} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:35:18.585384 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-6c3aeabd-6c95-4114-aeac-6cc9e9c008c6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '34b9c55f45c940deb3818b3093f48d47'} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:35:18.597272 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-6c3aeabd-6c95-4114-aeac-6cc9e9c008c6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '34b9c55f45c940deb3818b3093f48d47'} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:35:18.597894 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-6c3aeabd-6c95-4114-aeac-6cc9e9c008c6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '34b9c55f45c940deb3818b3093f48d47'} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:35:18.605593 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-6c3aeabd-6c95-4114-aeac-6cc9e9c008c6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '34b9c55f45c940deb3818b3093f48d47'} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:35:18.612135 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 1438/2867] 10.222.0.18 () {58 vars in 1224 bytes} [Tue Sep 13 03:35:18 2022] DELETE /identity/v3/users/34b9c55f45c940deb3818b3093f48d47 => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 03:35:18.880862 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1e78041-f327-46ea-9f80-653825ce0b98 admin admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:35:18.882235 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-d1e78041-f327-46ea-9f80-653825ce0b98 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:35:18.895017 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1e78041-f327-46ea-9f80-653825ce0b98 admin admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:35:18.951993 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5c3ca10-5d9b-483d-baf0-aeded1bb733c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76c5181fc7be45279ac00017bb4640b3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:35:18.952616 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c3ca10-5d9b-483d-baf0-aeded1bb733c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:35:18.952893 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c3ca10-5d9b-483d-baf0-aeded1bb733c admin admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:35:18.953162 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c3ca10-5d9b-483d-baf0-aeded1bb733c admin admin] PATH_INFO: `/v3/projects/ff68ec5b48a94c62a45a8cadcf65b1ec` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:35:18.954261 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5c3ca10-5d9b-483d-baf0-aeded1bb733c admin admin] RBAC: Authorizing `identity:delete_project(project_id=ff68ec5b48a94c62a45a8cadcf65b1ec)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:35:18.956198 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5c3ca10-5d9b-483d-baf0-aeded1bb733c admin admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:35:19.000086 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-d5c3ca10-5d9b-483d-baf0-aeded1bb733c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ff68ec5b48a94c62a45a8cadcf65b1ec', 'request_id': 'req-d5c3ca10-5d9b-483d-baf0-aeded1bb733c'} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:35:19.006017 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-d5c3ca10-5d9b-483d-baf0-aeded1bb733c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ff68ec5b48a94c62a45a8cadcf65b1ec', 'request_id': 'req-d5c3ca10-5d9b-483d-baf0-aeded1bb733c'} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:35:19.011962 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-d5c3ca10-5d9b-483d-baf0-aeded1bb733c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ff68ec5b48a94c62a45a8cadcf65b1ec', 'request_id': 'req-d5c3ca10-5d9b-483d-baf0-aeded1bb733c'} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:35:19.024226 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-d5c3ca10-5d9b-483d-baf0-aeded1bb733c admin admin] The token cache is being invalidate because project ff68ec5b48a94c62a45a8cadcf65b1ec was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81400) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 03:35:19.024513 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-d5c3ca10-5d9b-483d-baf0-aeded1bb733c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:35:19.025127 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-d5c3ca10-5d9b-483d-baf0-aeded1bb733c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:35:19.026593 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 1430/2868] 10.222.0.18 () {58 vars in 1233 bytes} [Tue Sep 13 03:35:18 2022] DELETE /identity/v3/projects/ff68ec5b48a94c62a45a8cadcf65b1ec => generated 0 bytes in 147 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 03:35:19.031547 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad66f35d-5f81-4669-b824-86973045b940 None None] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:35:19.031857 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad66f35d-5f81-4669-b824-86973045b940 None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:35:19.032128 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad66f35d-5f81-4669-b824-86973045b940 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:35:19.052133 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.auth.core [None req-ad66f35d-5f81-4669-b824-86973045b940 None None] MFA Rules not processed for user `49f280ab086b4610855c4ffa378517d9`. Rule list: `[]` (Enabled: `True`). {{(pid=81399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 03:35:19.053311 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-ad66f35d-5f81-4669-b824-86973045b940 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:35:19.117378 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 1439/2869] 10.222.0.18 () {58 vars in 958 bytes} [Tue Sep 13 03:35:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 03:35:19.124988 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9bfa93c-83ce-484f-b263-f451682b068b None None] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:35:19.125274 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9bfa93c-83ce-484f-b263-f451682b068b None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:35:19.125547 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9bfa93c-83ce-484f-b263-f451682b068b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:35:19.145661 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.auth.core [None req-b9bfa93c-83ce-484f-b263-f451682b068b None None] MFA Rules not processed for user `49f280ab086b4610855c4ffa378517d9`. Rule list: `[]` (Enabled: `True`). {{(pid=81400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 03:35:19.146834 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-b9bfa93c-83ce-484f-b263-f451682b068b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:35:19.154681 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 1431/2870] 10.222.0.18 () {58 vars in 958 bytes} [Tue Sep 13 03:35:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 03:35:19.158735 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad66f35d-5f81-4669-b824-86973045b940 None None] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:35:19.168490 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad66f35d-5f81-4669-b824-86973045b940 None None] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:35:19.172977 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-805c12bc-1e9b-4312-8980-f2e19da0df22 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76c5181fc7be45279ac00017bb4640b3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:35:19.173572 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-805c12bc-1e9b-4312-8980-f2e19da0df22 admin admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:35:19.173849 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-805c12bc-1e9b-4312-8980-f2e19da0df22 admin admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:35:19.174119 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-805c12bc-1e9b-4312-8980-f2e19da0df22 admin admin] PATH_INFO: `/v3/domains` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:35:19.174784 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-805c12bc-1e9b-4312-8980-f2e19da0df22 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 03:35:19.175087 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-805c12bc-1e9b-4312-8980-f2e19da0df22 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:35:19.177172 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-805c12bc-1e9b-4312-8980-f2e19da0df22 admin admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:35:19.188704 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 1440/2871] 10.222.0.18 () {58 vars in 1153 bytes} [Tue Sep 13 03:35:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 03:35:19.193429 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9bfa93c-83ce-484f-b263-f451682b068b None None] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:35:19.202952 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9bfa93c-83ce-484f-b263-f451682b068b None None] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:35:19.207579 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-708fd17e-d1df-4387-b9a6-8acbb159515a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76c5181fc7be45279ac00017bb4640b3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:35:19.208167 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-708fd17e-d1df-4387-b9a6-8acbb159515a admin admin] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:35:19.208448 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-708fd17e-d1df-4387-b9a6-8acbb159515a admin admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:35:19.208720 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-708fd17e-d1df-4387-b9a6-8acbb159515a admin admin] PATH_INFO: `/v3/projects` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:35:19.209347 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-708fd17e-d1df-4387-b9a6-8acbb159515a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:35:19.211302 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-708fd17e-d1df-4387-b9a6-8acbb159515a admin admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:35:19.222414 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:35:19.243006 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 1432/2872] 10.222.0.18 () {60 vars in 1153 bytes} [Tue Sep 13 03:35:19 2022] POST /identity/v3/projects => generated 371 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 03:35:19.244222 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-805c12bc-1e9b-4312-8980-f2e19da0df22 admin admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:35:19.254192 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-805c12bc-1e9b-4312-8980-f2e19da0df22 admin admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:35:19.310330 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d133d38-900e-4be3-b061-58e1254160b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76c5181fc7be45279ac00017bb4640b3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:35:19.311011 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d133d38-900e-4be3-b061-58e1254160b2 admin admin] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:35:19.311291 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d133d38-900e-4be3-b061-58e1254160b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:35:19.311564 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d133d38-900e-4be3-b061-58e1254160b2 admin admin] PATH_INFO: `/v3/users` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:35:19.312258 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d133d38-900e-4be3-b061-58e1254160b2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:35:19.314198 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d133d38-900e-4be3-b061-58e1254160b2 admin admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:35:19.335502 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 1441/2873] 10.222.0.18 () {60 vars in 1144 bytes} [Tue Sep 13 03:35:19 2022] POST /identity/v3/users => generated 286 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 03:35:19.341295 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-708fd17e-d1df-4387-b9a6-8acbb159515a admin admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:35:19.351496 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-708fd17e-d1df-4387-b9a6-8acbb159515a admin admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:35:19.357175 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fee286d-89cb-407d-9479-66b0ae0af452 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76c5181fc7be45279ac00017bb4640b3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:35:19.358187 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fee286d-89cb-407d-9479-66b0ae0af452 admin admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:35:19.358736 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fee286d-89cb-407d-9479-66b0ae0af452 admin admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:35:19.359226 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fee286d-89cb-407d-9479-66b0ae0af452 admin admin] PATH_INFO: `/v3/roles` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:35:19.360398 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fee286d-89cb-407d-9479-66b0ae0af452 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:35:19.363466 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fee286d-89cb-407d-9479-66b0ae0af452 admin admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:35:19.374839 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 1433/2874] 10.222.0.18 () {58 vars in 1122 bytes} [Tue Sep 13 03:35:19 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 03:35:19.378928 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d133d38-900e-4be3-b061-58e1254160b2 admin admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:35:19.389537 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d133d38-900e-4be3-b061-58e1254160b2 admin admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:35:19.394360 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b20efeb-fd02-450a-89ea-a0ea33b4b167 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76c5181fc7be45279ac00017bb4640b3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:35:19.395010 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b20efeb-fd02-450a-89ea-a0ea33b4b167 admin admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:35:19.395286 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b20efeb-fd02-450a-89ea-a0ea33b4b167 admin admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:35:19.395560 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b20efeb-fd02-450a-89ea-a0ea33b4b167 admin admin] PATH_INFO: `/v3/roles` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:35:19.396247 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b20efeb-fd02-450a-89ea-a0ea33b4b167 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:35:19.397780 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b20efeb-fd02-450a-89ea-a0ea33b4b167 admin admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:35:19.408399 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 1442/2875] 10.222.0.18 () {58 vars in 1122 bytes} [Tue Sep 13 03:35:19 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 13 03:35:19.413211 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fee286d-89cb-407d-9479-66b0ae0af452 admin admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:35:19.423189 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fee286d-89cb-407d-9479-66b0ae0af452 admin admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:35:19.428333 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c325457-0fd8-4725-8948-48b60475f667 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76c5181fc7be45279ac00017bb4640b3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:35:19.428969 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c325457-0fd8-4725-8948-48b60475f667 admin admin] REQUEST_METHOD: `PUT` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:35:19.429243 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c325457-0fd8-4725-8948-48b60475f667 admin admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:35:19.429514 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c325457-0fd8-4725-8948-48b60475f667 admin admin] PATH_INFO: `/v3/projects/1acee96791e54960abd4d9d1c735a298/users/79301dbf286e4b7d9fad34272a8002f5/roles/1eb7a5b5203945c29e6d22d653f47847` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:35:19.443727 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c325457-0fd8-4725-8948-48b60475f667 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1acee96791e54960abd4d9d1c735a298, user_id=79301dbf286e4b7d9fad34272a8002f5, role_id=1eb7a5b5203945c29e6d22d653f47847)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:35:19.446488 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c325457-0fd8-4725-8948-48b60475f667 admin admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:35:19.459684 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 1434/2876] 10.222.0.18 () {60 vars in 1483 bytes} [Tue Sep 13 03:35:19 2022] PUT /identity/v3/projects/1acee96791e54960abd4d9d1c735a298/users/79301dbf286e4b7d9fad34272a8002f5/roles/1eb7a5b5203945c29e6d22d653f47847 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 03:35:19.464684 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e81e0ed9-574b-47ff-abee-ecb19ee08921 None None] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:35:19.465144 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e81e0ed9-574b-47ff-abee-ecb19ee08921 None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:35:19.465249 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e81e0ed9-574b-47ff-abee-ecb19ee08921 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:35:19.484350 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.auth.core [None req-e81e0ed9-574b-47ff-abee-ecb19ee08921 None None] MFA Rules not processed for user `49f280ab086b4610855c4ffa378517d9`. Rule list: `[]` (Enabled: `True`). {{(pid=81399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 03:35:19.485526 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-e81e0ed9-574b-47ff-abee-ecb19ee08921 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:35:19.505340 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:DELETE=4,keystone:SELECT=92,keystone:INSERT=4 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:35:19.554784 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 1443/2877] 10.222.0.18 () {58 vars in 958 bytes} [Tue Sep 13 03:35:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 03:35:19.564527 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c325457-0fd8-4725-8948-48b60475f667 admin admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:35:19.566185 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-7c325457-0fd8-4725-8948-48b60475f667 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:35:19.624542 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c325457-0fd8-4725-8948-48b60475f667 admin admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:35:19.628188 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25df9661-f454-479c-9a8b-7f1fdc133397 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdffbc851086441ebd01022d1e4c2609', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b9d94f6a68f7470d9494f112175620dc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:35:19.628788 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25df9661-f454-479c-9a8b-7f1fdc133397 service neutron] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:35:19.629065 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25df9661-f454-479c-9a8b-7f1fdc133397 service neutron] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:35:19.629332 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25df9661-f454-479c-9a8b-7f1fdc133397 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:35:19.638296 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25df9661-f454-479c-9a8b-7f1fdc133397 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:35:19.639806 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25df9661-f454-479c-9a8b-7f1fdc133397 service neutron] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:35:19.649490 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 1435/2878] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Sep 13 03:35:19 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 03:35:20.260953 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13e5444e-cc9f-4061-844a-063412eef8ea None None] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:35:20.261238 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13e5444e-cc9f-4061-844a-063412eef8ea None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:35:20.261511 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13e5444e-cc9f-4061-844a-063412eef8ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:35:20.289350 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.auth.core [None req-13e5444e-cc9f-4061-844a-063412eef8ea None None] MFA Rules not processed for user `79301dbf286e4b7d9fad34272a8002f5`. Rule list: `[]` (Enabled: `True`). {{(pid=81399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 03:35:20.290550 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-13e5444e-cc9f-4061-844a-063412eef8ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:35:20.355642 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 1444/2879] 10.222.0.18 () {58 vars in 958 bytes} [Tue Sep 13 03:35:20 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 13 03:35:20.366735 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25df9661-f454-479c-9a8b-7f1fdc133397 service neutron] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:35:20.368127 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-25df9661-f454-479c-9a8b-7f1fdc133397 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:35:20.427960 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25df9661-f454-479c-9a8b-7f1fdc133397 service neutron] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:35:20.431767 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-398571ec-f47c-437c-85fc-8428b33e1e9f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '924ee3f608d34a18b0ddc4695f102a56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b9d94f6a68f7470d9494f112175620dc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:35:20.432372 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-398571ec-f47c-437c-85fc-8428b33e1e9f service cinder] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:35:20.432651 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-398571ec-f47c-437c-85fc-8428b33e1e9f service cinder] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:35:20.432919 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-398571ec-f47c-437c-85fc-8428b33e1e9f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:35:20.442743 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-398571ec-f47c-437c-85fc-8428b33e1e9f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:35:20.444185 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-398571ec-f47c-437c-85fc-8428b33e1e9f service cinder] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:35:20.453110 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 1436/2880] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Sep 13 03:35:20 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 03:35:22.856837 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-448297f4-1619-4b82-aea4-59c513defb9e None None] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:35:22.857352 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-448297f4-1619-4b82-aea4-59c513defb9e None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:35:22.857459 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-448297f4-1619-4b82-aea4-59c513defb9e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:35:22.877202 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.auth.core [None req-448297f4-1619-4b82-aea4-59c513defb9e None None] MFA Rules not processed for user `79301dbf286e4b7d9fad34272a8002f5`. Rule list: `[]` (Enabled: `True`). {{(pid=81399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 03:35:22.878401 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-448297f4-1619-4b82-aea4-59c513defb9e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:35:22.886014 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 1445/2881] 10.222.0.18 () {58 vars in 958 bytes} [Tue Sep 13 03:35:22 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 03:35:30.449156 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=70,keystone:INSERT=2 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:35:32.784112 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-856e6446-212b-48a3-9d96-676ce80de68e None None] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:35:32.784744 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-856e6446-212b-48a3-9d96-676ce80de68e None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:35:32.784744 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-856e6446-212b-48a3-9d96-676ce80de68e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:35:32.806149 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.auth.core [None req-856e6446-212b-48a3-9d96-676ce80de68e None None] MFA Rules not processed for user `4c2d20fb54a54ecc99b47f237be4917c`. Rule list: `[]` (Enabled: `True`). {{(pid=81400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 03:35:32.807529 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-856e6446-212b-48a3-9d96-676ce80de68e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:35:32.864040 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 1437/2882] 10.222.0.18 () {58 vars in 967 bytes} [Tue Sep 13 03:35:32 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 03:35:32.871719 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=50 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:35:32.915817 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-448297f4-1619-4b82-aea4-59c513defb9e None None] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:35:32.917229 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-448297f4-1619-4b82-aea4-59c513defb9e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:35:33.003944 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-448297f4-1619-4b82-aea4-59c513defb9e None None] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:35:33.009093 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bad2f579-5f9c-4ddb-b619-43e15892980a service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '893b0c0d85964b7dbc9705c0a091682c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b9d94f6a68f7470d9494f112175620dc', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:35:33.009702 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bad2f579-5f9c-4ddb-b619-43e15892980a service swift] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:35:33.009992 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bad2f579-5f9c-4ddb-b619-43e15892980a service swift] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:35:33.010254 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bad2f579-5f9c-4ddb-b619-43e15892980a service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:35:33.019852 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bad2f579-5f9c-4ddb-b619-43e15892980a service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:35:33.021510 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bad2f579-5f9c-4ddb-b619-43e15892980a service swift] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:35:33.030134 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 1446/2883] 10.222.0.18 () {60 vars in 1384 bytes} [Tue Sep 13 03:35:32 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 13 03:35:33.376386 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-856e6446-212b-48a3-9d96-676ce80de68e None None] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:35:33.377803 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-856e6446-212b-48a3-9d96-676ce80de68e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:35:33.389170 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-856e6446-212b-48a3-9d96-676ce80de68e None None] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:35:33.392848 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dd3b8dc-1a7a-46b2-a4ff-8c7d7c52a0cf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdffbc851086441ebd01022d1e4c2609', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b9d94f6a68f7470d9494f112175620dc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:35:33.393287 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd3b8dc-1a7a-46b2-a4ff-8c7d7c52a0cf service neutron] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:35:33.393555 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd3b8dc-1a7a-46b2-a4ff-8c7d7c52a0cf service neutron] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:35:33.393808 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd3b8dc-1a7a-46b2-a4ff-8c7d7c52a0cf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:35:33.395476 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-5dd3b8dc-1a7a-46b2-a4ff-8c7d7c52a0cf service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:35:33.407704 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dd3b8dc-1a7a-46b2-a4ff-8c7d7c52a0cf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:35:33.409159 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dd3b8dc-1a7a-46b2-a4ff-8c7d7c52a0cf service neutron] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:35:33.419689 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 1438/2884] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Sep 13 03:35:33 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 03:35:35.148730 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bad2f579-5f9c-4ddb-b619-43e15892980a service swift] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:35:35.150870 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-bad2f579-5f9c-4ddb-b619-43e15892980a service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:35:35.167779 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bad2f579-5f9c-4ddb-b619-43e15892980a service swift] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:35:35.174178 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-838c8cf2-76e4-4ec6-ac34-dbcbbcec4309 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76c5181fc7be45279ac00017bb4640b3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:35:35.174840 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-838c8cf2-76e4-4ec6-ac34-dbcbbcec4309 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:35:35.175697 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-838c8cf2-76e4-4ec6-ac34-dbcbbcec4309 admin admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:35:35.175982 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-838c8cf2-76e4-4ec6-ac34-dbcbbcec4309 admin admin] PATH_INFO: `/v3/users/452bed8c90ea4ab8957a8da46f9acc4a` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:35:35.177140 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-838c8cf2-76e4-4ec6-ac34-dbcbbcec4309 admin admin] RBAC: Authorizing `identity:delete_user(user_id=452bed8c90ea4ab8957a8da46f9acc4a)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:35:35.179489 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-838c8cf2-76e4-4ec6-ac34-dbcbbcec4309 admin admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:35:35.237942 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-838c8cf2-76e4-4ec6-ac34-dbcbbcec4309 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '452bed8c90ea4ab8957a8da46f9acc4a'} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:35:35.247977 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-838c8cf2-76e4-4ec6-ac34-dbcbbcec4309 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '452bed8c90ea4ab8957a8da46f9acc4a'} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:35:35.263465 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-838c8cf2-76e4-4ec6-ac34-dbcbbcec4309 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '452bed8c90ea4ab8957a8da46f9acc4a'} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:35:35.264228 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-838c8cf2-76e4-4ec6-ac34-dbcbbcec4309 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '452bed8c90ea4ab8957a8da46f9acc4a'} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:35:35.271573 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-838c8cf2-76e4-4ec6-ac34-dbcbbcec4309 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '452bed8c90ea4ab8957a8da46f9acc4a'} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:35:35.282386 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 1447/2885] 10.222.0.18 () {58 vars in 1224 bytes} [Tue Sep 13 03:35:35 2022] DELETE /identity/v3/users/452bed8c90ea4ab8957a8da46f9acc4a => generated 0 bytes in 131 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 03:35:35.284153 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c035cb-eeb0-43c2-9700-6c8a05ca049a None None] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:35:35.284677 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c035cb-eeb0-43c2-9700-6c8a05ca049a None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:35:35.284786 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c035cb-eeb0-43c2-9700-6c8a05ca049a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:35:35.304701 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.auth.core [None req-67c035cb-eeb0-43c2-9700-6c8a05ca049a None None] MFA Rules not processed for user `49f280ab086b4610855c4ffa378517d9`. Rule list: `[]` (Enabled: `True`). {{(pid=81400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 03:35:35.306195 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-67c035cb-eeb0-43c2-9700-6c8a05ca049a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:35:35.377408 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 1439/2886] 10.222.0.18 () {58 vars in 958 bytes} [Tue Sep 13 03:35:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 03:35:35.384983 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-838c8cf2-76e4-4ec6-ac34-dbcbbcec4309 admin admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:35:35.386390 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-838c8cf2-76e4-4ec6-ac34-dbcbbcec4309 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:35:35.479802 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-838c8cf2-76e4-4ec6-ac34-dbcbbcec4309 admin admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:35:35.487606 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5b1f232-cf63-4c2e-9e44-60007abd4cfa service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdffbc851086441ebd01022d1e4c2609', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b9d94f6a68f7470d9494f112175620dc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:35:35.488772 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b1f232-cf63-4c2e-9e44-60007abd4cfa service neutron] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:35:35.489123 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b1f232-cf63-4c2e-9e44-60007abd4cfa service neutron] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:35:35.489464 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b1f232-cf63-4c2e-9e44-60007abd4cfa service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:35:35.497376 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:35:35.501590 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5b1f232-cf63-4c2e-9e44-60007abd4cfa service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:35:35.503952 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5b1f232-cf63-4c2e-9e44-60007abd4cfa service neutron] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:35:35.520162 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 1448/2887] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Sep 13 03:35:35 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 135 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 03:35:36.151101 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67c035cb-eeb0-43c2-9700-6c8a05ca049a None None] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:35:36.153129 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-67c035cb-eeb0-43c2-9700-6c8a05ca049a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:35:36.167867 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67c035cb-eeb0-43c2-9700-6c8a05ca049a None None] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:35:36.186228 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-685cda80-ac7f-452f-8b5b-aaa145c50324 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76c5181fc7be45279ac00017bb4640b3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:35:36.187048 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-685cda80-ac7f-452f-8b5b-aaa145c50324 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:35:36.187485 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-685cda80-ac7f-452f-8b5b-aaa145c50324 admin admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:35:36.187755 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-685cda80-ac7f-452f-8b5b-aaa145c50324 admin admin] PATH_INFO: `/v3/projects/3903eed3df354b8a8e84c4f12d740dba` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:35:36.189879 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-685cda80-ac7f-452f-8b5b-aaa145c50324 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3903eed3df354b8a8e84c4f12d740dba)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:35:36.193572 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-685cda80-ac7f-452f-8b5b-aaa145c50324 admin admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:35:36.247963 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-685cda80-ac7f-452f-8b5b-aaa145c50324 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3903eed3df354b8a8e84c4f12d740dba', 'request_id': 'req-685cda80-ac7f-452f-8b5b-aaa145c50324'} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:35:36.254403 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-685cda80-ac7f-452f-8b5b-aaa145c50324 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3903eed3df354b8a8e84c4f12d740dba', 'request_id': 'req-685cda80-ac7f-452f-8b5b-aaa145c50324'} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:35:36.261277 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-685cda80-ac7f-452f-8b5b-aaa145c50324 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3903eed3df354b8a8e84c4f12d740dba', 'request_id': 'req-685cda80-ac7f-452f-8b5b-aaa145c50324'} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:35:36.277221 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-685cda80-ac7f-452f-8b5b-aaa145c50324 admin admin] The token cache is being invalidate because project 3903eed3df354b8a8e84c4f12d740dba was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81400) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 03:35:36.277221 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-685cda80-ac7f-452f-8b5b-aaa145c50324 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:35:36.278005 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-685cda80-ac7f-452f-8b5b-aaa145c50324 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:35:36.280682 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 1440/2888] 10.222.0.18 () {58 vars in 1233 bytes} [Tue Sep 13 03:35:36 2022] DELETE /identity/v3/projects/3903eed3df354b8a8e84c4f12d740dba => generated 0 bytes in 130 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 03:35:36.286412 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-732d1746-538b-47f1-9a4b-e9b9cc56a0a3 None None] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:35:36.286911 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-732d1746-538b-47f1-9a4b-e9b9cc56a0a3 None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:35:36.287222 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-732d1746-538b-47f1-9a4b-e9b9cc56a0a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:35:36.315289 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.auth.core [None req-732d1746-538b-47f1-9a4b-e9b9cc56a0a3 None None] MFA Rules not processed for user `49f280ab086b4610855c4ffa378517d9`. Rule list: `[]` (Enabled: `True`). {{(pid=81399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 03:35:36.316519 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-732d1746-538b-47f1-9a4b-e9b9cc56a0a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:35:36.403928 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee4abaf8-7653-47a7-8ac7-7260f94e33d8 None None] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:35:36.404477 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee4abaf8-7653-47a7-8ac7-7260f94e33d8 None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:35:36.404477 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee4abaf8-7653-47a7-8ac7-7260f94e33d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:35:36.406912 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 1449/2889] 10.222.0.18 () {58 vars in 958 bytes} [Tue Sep 13 03:35:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 03:35:36.437213 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.auth.core [None req-ee4abaf8-7653-47a7-8ac7-7260f94e33d8 None None] MFA Rules not processed for user `49f280ab086b4610855c4ffa378517d9`. Rule list: `[]` (Enabled: `True`). {{(pid=81400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 03:35:36.439179 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-ee4abaf8-7653-47a7-8ac7-7260f94e33d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:35:36.448606 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 1441/2890] 10.222.0.18 () {58 vars in 958 bytes} [Tue Sep 13 03:35:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 46 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 03:35:36.452479 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-732d1746-538b-47f1-9a4b-e9b9cc56a0a3 None None] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:35:36.471567 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-732d1746-538b-47f1-9a4b-e9b9cc56a0a3 None None] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:35:36.482637 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c6727a8-6469-4abc-84f6-776ec30e0e14 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76c5181fc7be45279ac00017bb4640b3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:35:36.483505 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c6727a8-6469-4abc-84f6-776ec30e0e14 admin admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:35:36.483910 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c6727a8-6469-4abc-84f6-776ec30e0e14 admin admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:35:36.484247 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c6727a8-6469-4abc-84f6-776ec30e0e14 admin admin] PATH_INFO: `/v3/domains` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:35:36.485072 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c6727a8-6469-4abc-84f6-776ec30e0e14 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 03:35:36.485330 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c6727a8-6469-4abc-84f6-776ec30e0e14 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:35:36.487503 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c6727a8-6469-4abc-84f6-776ec30e0e14 admin admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:35:36.504148 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 1450/2891] 10.222.0.18 () {58 vars in 1153 bytes} [Tue Sep 13 03:35:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 03:35:36.508306 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee4abaf8-7653-47a7-8ac7-7260f94e33d8 None None] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:35:36.522846 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee4abaf8-7653-47a7-8ac7-7260f94e33d8 None None] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:35:36.529474 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62ea6501-e223-44bc-b598-062cbdda954e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76c5181fc7be45279ac00017bb4640b3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:35:36.530095 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ea6501-e223-44bc-b598-062cbdda954e admin admin] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:35:36.530375 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ea6501-e223-44bc-b598-062cbdda954e admin admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:35:36.530771 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ea6501-e223-44bc-b598-062cbdda954e admin admin] PATH_INFO: `/v3/projects` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:35:36.531353 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62ea6501-e223-44bc-b598-062cbdda954e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:35:36.533284 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62ea6501-e223-44bc-b598-062cbdda954e admin admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:35:36.540717 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:35:36.573748 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 1442/2892] 10.222.0.18 () {60 vars in 1153 bytes} [Tue Sep 13 03:35:36 2022] POST /identity/v3/projects => generated 375 bytes in 63 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 03:35:36.575641 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c6727a8-6469-4abc-84f6-776ec30e0e14 admin admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:35:36.595716 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c6727a8-6469-4abc-84f6-776ec30e0e14 admin admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:35:36.702713 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64ff8a19-8917-486a-9f78-8137c00039c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76c5181fc7be45279ac00017bb4640b3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:35:36.702713 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64ff8a19-8917-486a-9f78-8137c00039c3 admin admin] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:35:36.702713 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64ff8a19-8917-486a-9f78-8137c00039c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:35:36.702713 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64ff8a19-8917-486a-9f78-8137c00039c3 admin admin] PATH_INFO: `/v3/users` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:35:36.703221 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64ff8a19-8917-486a-9f78-8137c00039c3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:35:36.703221 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64ff8a19-8917-486a-9f78-8137c00039c3 admin admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:35:36.733461 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 1451/2893] 10.222.0.18 () {60 vars in 1144 bytes} [Tue Sep 13 03:35:36 2022] POST /identity/v3/users => generated 288 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 03:35:36.734903 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62ea6501-e223-44bc-b598-062cbdda954e admin admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:35:36.746939 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62ea6501-e223-44bc-b598-062cbdda954e admin admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:35:36.755351 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-154cf845-9dcf-4f88-b950-235bcae6ca7f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76c5181fc7be45279ac00017bb4640b3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:35:36.756057 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-154cf845-9dcf-4f88-b950-235bcae6ca7f admin admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:35:36.756405 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-154cf845-9dcf-4f88-b950-235bcae6ca7f admin admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:35:36.756747 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-154cf845-9dcf-4f88-b950-235bcae6ca7f admin admin] PATH_INFO: `/v3/roles` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:35:36.757465 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-154cf845-9dcf-4f88-b950-235bcae6ca7f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:35:36.762986 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-154cf845-9dcf-4f88-b950-235bcae6ca7f admin admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:35:36.775511 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 1443/2894] 10.222.0.18 () {58 vars in 1122 bytes} [Tue Sep 13 03:35:36 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 03:35:36.780825 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64ff8a19-8917-486a-9f78-8137c00039c3 admin admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:35:36.811420 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64ff8a19-8917-486a-9f78-8137c00039c3 admin admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:35:36.819514 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4759c383-7702-4bfb-8448-6f51d09d880b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76c5181fc7be45279ac00017bb4640b3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:35:36.820278 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4759c383-7702-4bfb-8448-6f51d09d880b admin admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:35:36.821219 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4759c383-7702-4bfb-8448-6f51d09d880b admin admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:35:36.821994 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4759c383-7702-4bfb-8448-6f51d09d880b admin admin] PATH_INFO: `/v3/roles` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:35:36.822801 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4759c383-7702-4bfb-8448-6f51d09d880b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:35:36.824864 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4759c383-7702-4bfb-8448-6f51d09d880b admin admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:35:36.851151 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 1452/2895] 10.222.0.18 () {58 vars in 1122 bytes} [Tue Sep 13 03:35:36 2022] GET /identity/v3/roles => generated 1420 bytes in 71 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 03:35:36.852636 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-154cf845-9dcf-4f88-b950-235bcae6ca7f admin admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:35:36.869604 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-154cf845-9dcf-4f88-b950-235bcae6ca7f admin admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:35:36.882373 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94f9613f-b6fe-4106-b99b-c1aee55060fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76c5181fc7be45279ac00017bb4640b3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:35:36.883774 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94f9613f-b6fe-4106-b99b-c1aee55060fb admin admin] REQUEST_METHOD: `PUT` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:35:36.884114 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94f9613f-b6fe-4106-b99b-c1aee55060fb admin admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:35:36.884458 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94f9613f-b6fe-4106-b99b-c1aee55060fb admin admin] PATH_INFO: `/v3/projects/46af96b3a7dd4d45a3b379c021debf5b/users/3d0b72be90b94d248e3fd17bd64cf8e3/roles/1eb7a5b5203945c29e6d22d653f47847` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:35:36.904248 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94f9613f-b6fe-4106-b99b-c1aee55060fb admin admin] RBAC: Authorizing `identity:create_grant(project_id=46af96b3a7dd4d45a3b379c021debf5b, user_id=3d0b72be90b94d248e3fd17bd64cf8e3, role_id=1eb7a5b5203945c29e6d22d653f47847)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:35:36.917429 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94f9613f-b6fe-4106-b99b-c1aee55060fb admin admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:35:36.933244 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 1444/2896] 10.222.0.18 () {60 vars in 1483 bytes} [Tue Sep 13 03:35:36 2022] PUT /identity/v3/projects/46af96b3a7dd4d45a3b379c021debf5b/users/3d0b72be90b94d248e3fd17bd64cf8e3/roles/1eb7a5b5203945c29e6d22d653f47847 => generated 0 bytes in 81 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 03:35:36.943426 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a77feb93-6300-44a9-9241-555da21c2b4d None None] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:35:36.943710 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a77feb93-6300-44a9-9241-555da21c2b4d None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:35:36.943980 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a77feb93-6300-44a9-9241-555da21c2b4d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:35:36.980007 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.auth.core [None req-a77feb93-6300-44a9-9241-555da21c2b4d None None] MFA Rules not processed for user `3d0b72be90b94d248e3fd17bd64cf8e3`. Rule list: `[]` (Enabled: `True`). {{(pid=81399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 03:35:36.981654 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-a77feb93-6300-44a9-9241-555da21c2b4d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:35:37.032698 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:35:37.054702 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 1453/2897] 10.222.0.18 () {58 vars in 958 bytes} [Tue Sep 13 03:35:36 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 03:35:37.065506 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94f9613f-b6fe-4106-b99b-c1aee55060fb admin admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:35:37.067457 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-94f9613f-b6fe-4106-b99b-c1aee55060fb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:35:37.128788 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94f9613f-b6fe-4106-b99b-c1aee55060fb admin admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:35:37.132429 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fa7ae70-8949-4fd0-b306-ee143c1df400 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '924ee3f608d34a18b0ddc4695f102a56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b9d94f6a68f7470d9494f112175620dc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:35:37.133032 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa7ae70-8949-4fd0-b306-ee143c1df400 service cinder] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:35:37.133356 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa7ae70-8949-4fd0-b306-ee143c1df400 service cinder] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:35:37.133632 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa7ae70-8949-4fd0-b306-ee143c1df400 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:35:37.142121 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fa7ae70-8949-4fd0-b306-ee143c1df400 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:35:37.149720 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fa7ae70-8949-4fd0-b306-ee143c1df400 service cinder] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:35:37.154879 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 1445/2898] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Sep 13 03:35:37 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 03:35:40.374068 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a77feb93-6300-44a9-9241-555da21c2b4d None None] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:35:40.440233 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a77feb93-6300-44a9-9241-555da21c2b4d None None] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:35:40.444924 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f6bc78d-c52e-4028-9a3f-8d390f6c3bda admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76c5181fc7be45279ac00017bb4640b3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:35:40.446251 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f6bc78d-c52e-4028-9a3f-8d390f6c3bda admin admin] REQUEST_METHOD: `DELETE` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:35:40.446386 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f6bc78d-c52e-4028-9a3f-8d390f6c3bda admin admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:35:40.446586 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f6bc78d-c52e-4028-9a3f-8d390f6c3bda admin admin] PATH_INFO: `/v3/users/3d0b72be90b94d248e3fd17bd64cf8e3` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:35:40.447912 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f6bc78d-c52e-4028-9a3f-8d390f6c3bda admin admin] RBAC: Authorizing `identity:delete_user(user_id=3d0b72be90b94d248e3fd17bd64cf8e3)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:35:40.449840 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f6bc78d-c52e-4028-9a3f-8d390f6c3bda admin admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:35:40.499641 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-3f6bc78d-c52e-4028-9a3f-8d390f6c3bda admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3d0b72be90b94d248e3fd17bd64cf8e3'} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:35:40.509272 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-3f6bc78d-c52e-4028-9a3f-8d390f6c3bda admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3d0b72be90b94d248e3fd17bd64cf8e3'} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:35:40.524775 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-3f6bc78d-c52e-4028-9a3f-8d390f6c3bda admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3d0b72be90b94d248e3fd17bd64cf8e3'} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:35:40.525482 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-3f6bc78d-c52e-4028-9a3f-8d390f6c3bda admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3d0b72be90b94d248e3fd17bd64cf8e3'} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:35:40.533616 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-3f6bc78d-c52e-4028-9a3f-8d390f6c3bda admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3d0b72be90b94d248e3fd17bd64cf8e3'} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:35:40.544756 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 1454/2899] 10.222.0.18 () {58 vars in 1224 bytes} [Tue Sep 13 03:35:40 2022] DELETE /identity/v3/users/3d0b72be90b94d248e3fd17bd64cf8e3 => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 03:35:40.546395 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a93ad20-6b75-4599-972c-d5bc9fdd5aae None None] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:35:40.546832 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a93ad20-6b75-4599-972c-d5bc9fdd5aae None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:35:40.547017 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a93ad20-6b75-4599-972c-d5bc9fdd5aae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:35:40.554499 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f6bc78d-c52e-4028-9a3f-8d390f6c3bda admin admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:35:40.555958 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-3f6bc78d-c52e-4028-9a3f-8d390f6c3bda admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:35:40.584456 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.auth.core [None req-4a93ad20-6b75-4599-972c-d5bc9fdd5aae None None] MFA Rules not processed for user `49f280ab086b4610855c4ffa378517d9`. Rule list: `[]` (Enabled: `True`). {{(pid=81400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 03:35:40.585650 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-4a93ad20-6b75-4599-972c-d5bc9fdd5aae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:35:40.632566 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f6bc78d-c52e-4028-9a3f-8d390f6c3bda admin admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:35:40.637993 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2932cbf-db02-452e-8d6c-6175a841f4bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76c5181fc7be45279ac00017bb4640b3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:35:40.638810 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2932cbf-db02-452e-8d6c-6175a841f4bc admin admin] REQUEST_METHOD: `DELETE` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:35:40.639184 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2932cbf-db02-452e-8d6c-6175a841f4bc admin admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:35:40.639463 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2932cbf-db02-452e-8d6c-6175a841f4bc admin admin] PATH_INFO: `/v3/users/92bba932a70f4a75b1bb1810d8826447` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:35:40.640572 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2932cbf-db02-452e-8d6c-6175a841f4bc admin admin] RBAC: Authorizing `identity:delete_user(user_id=92bba932a70f4a75b1bb1810d8826447)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:35:40.642879 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2932cbf-db02-452e-8d6c-6175a841f4bc admin admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:35:40.646513 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 1446/2900] 10.222.0.18 () {58 vars in 958 bytes} [Tue Sep 13 03:35:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 03:35:40.654434 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:35:40.656693 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a93ad20-6b75-4599-972c-d5bc9fdd5aae None None] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:35:40.658254 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-4a93ad20-6b75-4599-972c-d5bc9fdd5aae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:35:40.707894 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-c2932cbf-db02-452e-8d6c-6175a841f4bc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '92bba932a70f4a75b1bb1810d8826447'} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:35:40.716414 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-c2932cbf-db02-452e-8d6c-6175a841f4bc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '92bba932a70f4a75b1bb1810d8826447'} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:35:40.730372 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-c2932cbf-db02-452e-8d6c-6175a841f4bc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '92bba932a70f4a75b1bb1810d8826447'} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:35:40.731184 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-c2932cbf-db02-452e-8d6c-6175a841f4bc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '92bba932a70f4a75b1bb1810d8826447'} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:35:40.738739 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-c2932cbf-db02-452e-8d6c-6175a841f4bc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '92bba932a70f4a75b1bb1810d8826447'} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:35:40.746416 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 1455/2901] 10.222.0.18 () {58 vars in 1224 bytes} [Tue Sep 13 03:35:40 2022] DELETE /identity/v3/users/92bba932a70f4a75b1bb1810d8826447 => generated 0 bytes in 193 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 03:35:40.838279 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a93ad20-6b75-4599-972c-d5bc9fdd5aae None None] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:35:40.845914 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:35:40.903441 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a0a6680-4846-45db-aefc-5f949fc11c5c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdffbc851086441ebd01022d1e4c2609', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b9d94f6a68f7470d9494f112175620dc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:35:40.903970 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a0a6680-4846-45db-aefc-5f949fc11c5c service neutron] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:35:40.904245 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a0a6680-4846-45db-aefc-5f949fc11c5c service neutron] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:35:40.904518 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a0a6680-4846-45db-aefc-5f949fc11c5c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:35:40.907886 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-7a0a6680-4846-45db-aefc-5f949fc11c5c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:35:40.933377 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2932cbf-db02-452e-8d6c-6175a841f4bc admin admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:35:40.934859 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-c2932cbf-db02-452e-8d6c-6175a841f4bc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:35:40.981128 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a0a6680-4846-45db-aefc-5f949fc11c5c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:35:40.982571 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a0a6680-4846-45db-aefc-5f949fc11c5c service neutron] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:35:40.993620 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 1447/2902] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Sep 13 03:35:40 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 337 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 03:35:41.010715 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2932cbf-db02-452e-8d6c-6175a841f4bc admin admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:35:41.015312 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7579b31-2123-4048-a3e5-1954f4d30701 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76c5181fc7be45279ac00017bb4640b3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:35:41.015949 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7579b31-2123-4048-a3e5-1954f4d30701 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:35:41.016224 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7579b31-2123-4048-a3e5-1954f4d30701 admin admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:35:41.016492 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7579b31-2123-4048-a3e5-1954f4d30701 admin admin] PATH_INFO: `/v3/projects/35d0f9365a60474590ea1bb78b1654b2` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:35:41.017599 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7579b31-2123-4048-a3e5-1954f4d30701 admin admin] RBAC: Authorizing `identity:delete_project(project_id=35d0f9365a60474590ea1bb78b1654b2)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:35:41.019583 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7579b31-2123-4048-a3e5-1954f4d30701 admin admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:35:41.070011 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-e7579b31-2123-4048-a3e5-1954f4d30701 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '35d0f9365a60474590ea1bb78b1654b2', 'request_id': 'req-e7579b31-2123-4048-a3e5-1954f4d30701'} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:35:41.075987 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-e7579b31-2123-4048-a3e5-1954f4d30701 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '35d0f9365a60474590ea1bb78b1654b2', 'request_id': 'req-e7579b31-2123-4048-a3e5-1954f4d30701'} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:35:41.082913 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-e7579b31-2123-4048-a3e5-1954f4d30701 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '35d0f9365a60474590ea1bb78b1654b2', 'request_id': 'req-e7579b31-2123-4048-a3e5-1954f4d30701'} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:35:41.096202 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-e7579b31-2123-4048-a3e5-1954f4d30701 admin admin] The token cache is being invalidate because project 35d0f9365a60474590ea1bb78b1654b2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81399) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 03:35:41.096502 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-e7579b31-2123-4048-a3e5-1954f4d30701 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:35:41.097158 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-e7579b31-2123-4048-a3e5-1954f4d30701 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:35:41.098851 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 1456/2903] 10.222.0.18 () {58 vars in 1233 bytes} [Tue Sep 13 03:35:40 2022] DELETE /identity/v3/projects/35d0f9365a60474590ea1bb78b1654b2 => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 13 03:35:41.103487 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a0a6680-4846-45db-aefc-5f949fc11c5c service neutron] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:35:41.113544 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a0a6680-4846-45db-aefc-5f949fc11c5c service neutron] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:35:41.175090 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7d4e3da-5c8e-4b17-aa1a-666da49b1572 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76c5181fc7be45279ac00017bb4640b3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:35:41.175719 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7d4e3da-5c8e-4b17-aa1a-666da49b1572 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:35:41.175993 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7d4e3da-5c8e-4b17-aa1a-666da49b1572 admin admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:35:41.176256 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7d4e3da-5c8e-4b17-aa1a-666da49b1572 admin admin] PATH_INFO: `/v3/users/a4d295248be84eb9901f3415da4b4901` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:35:41.177432 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7d4e3da-5c8e-4b17-aa1a-666da49b1572 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a4d295248be84eb9901f3415da4b4901)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:35:41.179421 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7d4e3da-5c8e-4b17-aa1a-666da49b1572 admin admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:35:41.231549 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-b7d4e3da-5c8e-4b17-aa1a-666da49b1572 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a4d295248be84eb9901f3415da4b4901'} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:35:41.239069 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-b7d4e3da-5c8e-4b17-aa1a-666da49b1572 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a4d295248be84eb9901f3415da4b4901'} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:35:41.239709 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-b7d4e3da-5c8e-4b17-aa1a-666da49b1572 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a4d295248be84eb9901f3415da4b4901'} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:35:41.247654 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:35:41.254372 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-b7d4e3da-5c8e-4b17-aa1a-666da49b1572 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a4d295248be84eb9901f3415da4b4901'} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:35:41.262730 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-b7d4e3da-5c8e-4b17-aa1a-666da49b1572 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a4d295248be84eb9901f3415da4b4901'} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:35:41.268709 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 1448/2904] 10.222.0.18 () {58 vars in 1224 bytes} [Tue Sep 13 03:35:41 2022] DELETE /identity/v3/users/a4d295248be84eb9901f3415da4b4901 => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 03:35:41.277820 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7579b31-2123-4048-a3e5-1954f4d30701 admin admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:35:41.279243 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-e7579b31-2123-4048-a3e5-1954f4d30701 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:35:41.301860 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7579b31-2123-4048-a3e5-1954f4d30701 admin admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:35:41.317353 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:DELETE=17,keystone:SELECT=81,keystone:INSERT=2 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:35:41.365058 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b73037a-b223-4a47-8e58-f2411ba647bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76c5181fc7be45279ac00017bb4640b3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:35:41.365697 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b73037a-b223-4a47-8e58-f2411ba647bc admin admin] REQUEST_METHOD: `DELETE` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:35:41.365976 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b73037a-b223-4a47-8e58-f2411ba647bc admin admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:35:41.366248 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b73037a-b223-4a47-8e58-f2411ba647bc admin admin] PATH_INFO: `/v3/projects/46af96b3a7dd4d45a3b379c021debf5b` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:35:41.367472 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b73037a-b223-4a47-8e58-f2411ba647bc admin admin] RBAC: Authorizing `identity:delete_project(project_id=46af96b3a7dd4d45a3b379c021debf5b)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:35:41.370182 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b73037a-b223-4a47-8e58-f2411ba647bc admin admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:35:41.424081 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-8b73037a-b223-4a47-8e58-f2411ba647bc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '46af96b3a7dd4d45a3b379c021debf5b', 'request_id': 'req-8b73037a-b223-4a47-8e58-f2411ba647bc'} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:35:41.439058 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-8b73037a-b223-4a47-8e58-f2411ba647bc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '46af96b3a7dd4d45a3b379c021debf5b', 'request_id': 'req-8b73037a-b223-4a47-8e58-f2411ba647bc'} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:35:41.446309 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-8b73037a-b223-4a47-8e58-f2411ba647bc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '46af96b3a7dd4d45a3b379c021debf5b', 'request_id': 'req-8b73037a-b223-4a47-8e58-f2411ba647bc'} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:35:41.462226 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7d4e3da-5c8e-4b17-aa1a-666da49b1572 admin admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:35:41.463960 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-b7d4e3da-5c8e-4b17-aa1a-666da49b1572 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:35:41.468253 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-8b73037a-b223-4a47-8e58-f2411ba647bc admin admin] The token cache is being invalidate because project 46af96b3a7dd4d45a3b379c021debf5b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81399) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 03:35:41.468617 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-8b73037a-b223-4a47-8e58-f2411ba647bc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:35:41.469417 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-8b73037a-b223-4a47-8e58-f2411ba647bc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:35:41.482654 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 1457/2905] 10.222.0.18 () {58 vars in 1233 bytes} [Tue Sep 13 03:35:41 2022] DELETE /identity/v3/projects/46af96b3a7dd4d45a3b379c021debf5b => generated 0 bytes in 194 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 13 03:35:41.483342 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7d4e3da-5c8e-4b17-aa1a-666da49b1572 admin admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:35:41.558221 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e568b60-2869-4e78-812c-d41f102fa4db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76c5181fc7be45279ac00017bb4640b3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:35:41.559003 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e568b60-2869-4e78-812c-d41f102fa4db admin admin] REQUEST_METHOD: `DELETE` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:35:41.559344 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e568b60-2869-4e78-812c-d41f102fa4db admin admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:35:41.559699 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e568b60-2869-4e78-812c-d41f102fa4db admin admin] PATH_INFO: `/v3/projects/0baadaacd0434d82af969deb277f2194` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:35:41.562675 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e568b60-2869-4e78-812c-d41f102fa4db admin admin] RBAC: Authorizing `identity:delete_project(project_id=0baadaacd0434d82af969deb277f2194)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:35:41.563286 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e568b60-2869-4e78-812c-d41f102fa4db admin admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:35:41.621957 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-9e568b60-2869-4e78-812c-d41f102fa4db admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0baadaacd0434d82af969deb277f2194', 'request_id': 'req-9e568b60-2869-4e78-812c-d41f102fa4db'} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:35:41.627989 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-9e568b60-2869-4e78-812c-d41f102fa4db admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0baadaacd0434d82af969deb277f2194', 'request_id': 'req-9e568b60-2869-4e78-812c-d41f102fa4db'} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:35:41.635443 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-9e568b60-2869-4e78-812c-d41f102fa4db admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0baadaacd0434d82af969deb277f2194', 'request_id': 'req-9e568b60-2869-4e78-812c-d41f102fa4db'} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:35:41.651306 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-9e568b60-2869-4e78-812c-d41f102fa4db admin admin] The token cache is being invalidate because project 0baadaacd0434d82af969deb277f2194 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81400) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 03:35:41.651584 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-9e568b60-2869-4e78-812c-d41f102fa4db admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:35:41.652245 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-9e568b60-2869-4e78-812c-d41f102fa4db admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:35:41.654035 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 1449/2906] 10.222.0.18 () {58 vars in 1233 bytes} [Tue Sep 13 03:35:41 2022] DELETE /identity/v3/projects/0baadaacd0434d82af969deb277f2194 => generated 0 bytes in 193 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 03:35:45.849027 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f45111de-12ac-4690-b655-7efae142d4df None None] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:35:45.849506 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f45111de-12ac-4690-b655-7efae142d4df None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:35:45.849617 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f45111de-12ac-4690-b655-7efae142d4df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:35:45.868913 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.auth.core [None req-f45111de-12ac-4690-b655-7efae142d4df None None] MFA Rules not processed for user `79301dbf286e4b7d9fad34272a8002f5`. Rule list: `[]` (Enabled: `True`). {{(pid=81399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 03:35:45.870381 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-f45111de-12ac-4690-b655-7efae142d4df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:35:45.924912 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 1458/2907] 10.222.0.18 () {58 vars in 958 bytes} [Tue Sep 13 03:35:45 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 03:35:45.960951 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e568b60-2869-4e78-812c-d41f102fa4db admin admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:35:45.962320 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-9e568b60-2869-4e78-812c-d41f102fa4db admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:35:46.018575 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e568b60-2869-4e78-812c-d41f102fa4db admin admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:35:46.022191 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-621eb26e-006d-42fe-a5be-582a983045b4 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b5fdc2ff46a14638b409a13e73ba5500', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b9d94f6a68f7470d9494f112175620dc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:35:46.022816 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-621eb26e-006d-42fe-a5be-582a983045b4 service glance] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:35:46.023098 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-621eb26e-006d-42fe-a5be-582a983045b4 service glance] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:35:46.023364 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-621eb26e-006d-42fe-a5be-582a983045b4 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:35:46.025036 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-621eb26e-006d-42fe-a5be-582a983045b4 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:35:46.049677 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:35:46.099052 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-621eb26e-006d-42fe-a5be-582a983045b4 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:35:46.100486 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-621eb26e-006d-42fe-a5be-582a983045b4 service glance] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:35:46.111651 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 1450/2908] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Sep 13 03:35:45 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 151 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 03:35:46.315725 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f45111de-12ac-4690-b655-7efae142d4df None None] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:35:46.317134 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-f45111de-12ac-4690-b655-7efae142d4df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:35:46.372660 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f45111de-12ac-4690-b655-7efae142d4df None None] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:35:46.379018 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02fb923b-a649-486e-aa7d-87f93c20688b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdffbc851086441ebd01022d1e4c2609', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b9d94f6a68f7470d9494f112175620dc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:35:46.379732 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02fb923b-a649-486e-aa7d-87f93c20688b service neutron] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:35:46.380013 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02fb923b-a649-486e-aa7d-87f93c20688b service neutron] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:35:46.380280 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02fb923b-a649-486e-aa7d-87f93c20688b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:35:46.382269 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-02fb923b-a649-486e-aa7d-87f93c20688b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:35:46.390785 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:35:46.393650 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02fb923b-a649-486e-aa7d-87f93c20688b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:35:46.395568 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02fb923b-a649-486e-aa7d-87f93c20688b service neutron] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:35:46.404668 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 1459/2909] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Sep 13 03:35:46 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 13 03:35:48.949513 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-621eb26e-006d-42fe-a5be-582a983045b4 service glance] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:35:48.950979 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-621eb26e-006d-42fe-a5be-582a983045b4 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:35:48.962196 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-621eb26e-006d-42fe-a5be-582a983045b4 service glance] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:35:48.965885 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8bad614-5525-448e-bd60-5f7afc838544 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdffbc851086441ebd01022d1e4c2609', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b9d94f6a68f7470d9494f112175620dc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:35:48.966701 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8bad614-5525-448e-bd60-5f7afc838544 service neutron] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:35:48.967518 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8bad614-5525-448e-bd60-5f7afc838544 service neutron] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:35:48.967834 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8bad614-5525-448e-bd60-5f7afc838544 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:35:48.970055 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-c8bad614-5525-448e-bd60-5f7afc838544 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:35:48.983854 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8bad614-5525-448e-bd60-5f7afc838544 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:35:48.985263 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8bad614-5525-448e-bd60-5f7afc838544 service neutron] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:35:48.996954 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 1451/2910] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Sep 13 03:35:48 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 13 03:35:56.405281 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=2 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:35:58.991402 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=28 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:36:10.170204 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf461c1-e80d-42de-9882-c865829efacb None None] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:36:10.170759 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf461c1-e80d-42de-9882-c865829efacb None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:36:10.170908 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf461c1-e80d-42de-9882-c865829efacb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:36:10.192506 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.auth.core [None req-fdf461c1-e80d-42de-9882-c865829efacb None None] MFA Rules not processed for user `4c2d20fb54a54ecc99b47f237be4917c`. Rule list: `[]` (Enabled: `True`). {{(pid=81399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 03:36:10.193719 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-fdf461c1-e80d-42de-9882-c865829efacb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:36:10.251802 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 1460/2911] 10.222.0.18 () {58 vars in 1009 bytes} [Tue Sep 13 03:36:10 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 03:36:10.263984 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8bad614-5525-448e-bd60-5f7afc838544 service neutron] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:36:10.265327 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-c8bad614-5525-448e-bd60-5f7afc838544 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:36:10.331259 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8bad614-5525-448e-bd60-5f7afc838544 service neutron] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:36:10.336228 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a04c9fb5-97fb-4977-9c75-3dbd216eb5c4 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '893b0c0d85964b7dbc9705c0a091682c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b9d94f6a68f7470d9494f112175620dc', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:36:10.336822 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a04c9fb5-97fb-4977-9c75-3dbd216eb5c4 service swift] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:36:10.337095 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a04c9fb5-97fb-4977-9c75-3dbd216eb5c4 service swift] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:36:10.337361 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a04c9fb5-97fb-4977-9c75-3dbd216eb5c4 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:36:10.345646 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a04c9fb5-97fb-4977-9c75-3dbd216eb5c4 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:36:10.347280 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a04c9fb5-97fb-4977-9c75-3dbd216eb5c4 service swift] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:36:10.355915 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 1452/2912] 10.222.0.18 () {60 vars in 1384 bytes} [Tue Sep 13 03:36:10 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 13 03:36:14.010193 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdf461c1-e80d-42de-9882-c865829efacb None None] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:36:14.020527 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdf461c1-e80d-42de-9882-c865829efacb None None] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:36:14.024119 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd0e18dc-af56-48d9-a98f-ce7e67db5740 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdffbc851086441ebd01022d1e4c2609', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b9d94f6a68f7470d9494f112175620dc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:36:14.024952 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd0e18dc-af56-48d9-a98f-ce7e67db5740 service neutron] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:36:14.027592 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd0e18dc-af56-48d9-a98f-ce7e67db5740 service neutron] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:36:14.027592 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd0e18dc-af56-48d9-a98f-ce7e67db5740 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:36:14.030559 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-fd0e18dc-af56-48d9-a98f-ce7e67db5740 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:36:14.124507 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd0e18dc-af56-48d9-a98f-ce7e67db5740 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:36:14.126071 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd0e18dc-af56-48d9-a98f-ce7e67db5740 service neutron] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:36:14.137104 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 1461/2913] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Sep 13 03:36:14 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 128 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 03:36:17.792177 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a04c9fb5-97fb-4977-9c75-3dbd216eb5c4 service swift] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:36:17.796902 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-a04c9fb5-97fb-4977-9c75-3dbd216eb5c4 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:36:17.958139 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a04c9fb5-97fb-4977-9c75-3dbd216eb5c4 service swift] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:36:17.963799 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2c8f462-066a-4630-b864-091d2503eca4 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b99f1b2374d4d2f8db78306316180d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b9d94f6a68f7470d9494f112175620dc', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:36:17.964651 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c8f462-066a-4630-b864-091d2503eca4 service placement] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:36:17.965100 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c8f462-066a-4630-b864-091d2503eca4 service placement] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:36:17.965583 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c8f462-066a-4630-b864-091d2503eca4 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:36:17.968342 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-d2c8f462-066a-4630-b864-091d2503eca4 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:36:17.983499 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2c8f462-066a-4630-b864-091d2503eca4 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:36:17.985878 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2c8f462-066a-4630-b864-091d2503eca4 service placement] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:36:17.998492 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 1453/2914] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Sep 13 03:36:17 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 207 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 03:36:22.000251 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd0e18dc-af56-48d9-a98f-ce7e67db5740 service neutron] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:36:22.002432 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-fd0e18dc-af56-48d9-a98f-ce7e67db5740 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:36:22.020621 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd0e18dc-af56-48d9-a98f-ce7e67db5740 service neutron] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:36:22.026214 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-092c5bd9-8cef-4889-839b-c68a84d994db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76c5181fc7be45279ac00017bb4640b3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:36:22.027171 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-092c5bd9-8cef-4889-839b-c68a84d994db admin admin] REQUEST_METHOD: `DELETE` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:36:22.027519 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-092c5bd9-8cef-4889-839b-c68a84d994db admin admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:36:22.027851 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-092c5bd9-8cef-4889-839b-c68a84d994db admin admin] PATH_INFO: `/v3/users/47fa1f7ce54145ceb3564b9ba7b4964f` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:36:22.044654 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-092c5bd9-8cef-4889-839b-c68a84d994db admin admin] RBAC: Authorizing `identity:delete_user(user_id=47fa1f7ce54145ceb3564b9ba7b4964f)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:36:22.046733 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-092c5bd9-8cef-4889-839b-c68a84d994db admin admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:36:22.112595 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-092c5bd9-8cef-4889-839b-c68a84d994db admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '47fa1f7ce54145ceb3564b9ba7b4964f'} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:36:22.121175 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-092c5bd9-8cef-4889-839b-c68a84d994db admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '47fa1f7ce54145ceb3564b9ba7b4964f'} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:36:22.137145 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-092c5bd9-8cef-4889-839b-c68a84d994db admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '47fa1f7ce54145ceb3564b9ba7b4964f'} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:36:22.137856 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-092c5bd9-8cef-4889-839b-c68a84d994db admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '47fa1f7ce54145ceb3564b9ba7b4964f'} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:36:22.144933 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-092c5bd9-8cef-4889-839b-c68a84d994db admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '47fa1f7ce54145ceb3564b9ba7b4964f'} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:36:22.147492 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:36:22.153005 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 1462/2915] 10.222.0.18 () {58 vars in 1224 bytes} [Tue Sep 13 03:36:21 2022] DELETE /identity/v3/users/47fa1f7ce54145ceb3564b9ba7b4964f => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 03:36:22.354424 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2c8f462-066a-4630-b864-091d2503eca4 service placement] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:36:22.374307 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2c8f462-066a-4630-b864-091d2503eca4 service placement] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:36:22.451594 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cbdd396-ffcf-42d9-883e-25ef3983854f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76c5181fc7be45279ac00017bb4640b3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:36:22.452350 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cbdd396-ffcf-42d9-883e-25ef3983854f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:36:22.452628 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cbdd396-ffcf-42d9-883e-25ef3983854f admin admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:36:22.452897 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cbdd396-ffcf-42d9-883e-25ef3983854f admin admin] PATH_INFO: `/v3/projects/ab6c92ba81254caaad368fd0ef382b0e` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:36:22.469292 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cbdd396-ffcf-42d9-883e-25ef3983854f admin admin] RBAC: Authorizing `identity:delete_project(project_id=ab6c92ba81254caaad368fd0ef382b0e)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:36:22.471262 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cbdd396-ffcf-42d9-883e-25ef3983854f admin admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:36:22.485725 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=100 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:36:22.539453 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-4cbdd396-ffcf-42d9-883e-25ef3983854f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ab6c92ba81254caaad368fd0ef382b0e', 'request_id': 'req-4cbdd396-ffcf-42d9-883e-25ef3983854f'} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:36:22.545604 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-4cbdd396-ffcf-42d9-883e-25ef3983854f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ab6c92ba81254caaad368fd0ef382b0e', 'request_id': 'req-4cbdd396-ffcf-42d9-883e-25ef3983854f'} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:36:22.553621 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-4cbdd396-ffcf-42d9-883e-25ef3983854f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ab6c92ba81254caaad368fd0ef382b0e', 'request_id': 'req-4cbdd396-ffcf-42d9-883e-25ef3983854f'} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:36:22.574075 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-4cbdd396-ffcf-42d9-883e-25ef3983854f admin admin] The token cache is being invalidate because project ab6c92ba81254caaad368fd0ef382b0e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81400) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 03:36:22.574368 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-4cbdd396-ffcf-42d9-883e-25ef3983854f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:36:22.575040 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-4cbdd396-ffcf-42d9-883e-25ef3983854f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:36:22.579563 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 1454/2916] 10.222.0.18 () {58 vars in 1233 bytes} [Tue Sep 13 03:36:22 2022] DELETE /identity/v3/projects/ab6c92ba81254caaad368fd0ef382b0e => generated 0 bytes in 223 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 03:36:22.580759 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-092c5bd9-8cef-4889-839b-c68a84d994db admin admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:36:22.582153 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-092c5bd9-8cef-4889-839b-c68a84d994db admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:36:22.595960 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-092c5bd9-8cef-4889-839b-c68a84d994db admin admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:36:22.659867 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f885645-75c9-4567-82db-135444a0fe86 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76c5181fc7be45279ac00017bb4640b3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:36:22.660509 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f885645-75c9-4567-82db-135444a0fe86 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:36:22.660783 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f885645-75c9-4567-82db-135444a0fe86 admin admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:36:22.661052 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f885645-75c9-4567-82db-135444a0fe86 admin admin] PATH_INFO: `/v3/users/fae11259c5894e94b8b1db60c00089c4` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:36:22.677744 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f885645-75c9-4567-82db-135444a0fe86 admin admin] RBAC: Authorizing `identity:delete_user(user_id=fae11259c5894e94b8b1db60c00089c4)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:36:22.679913 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f885645-75c9-4567-82db-135444a0fe86 admin admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:36:22.747847 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-1f885645-75c9-4567-82db-135444a0fe86 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fae11259c5894e94b8b1db60c00089c4'} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:36:22.763189 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-1f885645-75c9-4567-82db-135444a0fe86 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fae11259c5894e94b8b1db60c00089c4'} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:36:22.778666 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-1f885645-75c9-4567-82db-135444a0fe86 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fae11259c5894e94b8b1db60c00089c4'} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:36:22.779498 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-1f885645-75c9-4567-82db-135444a0fe86 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fae11259c5894e94b8b1db60c00089c4'} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:36:22.787674 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-1f885645-75c9-4567-82db-135444a0fe86 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fae11259c5894e94b8b1db60c00089c4'} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:36:22.794589 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 1463/2917] 10.222.0.18 () {58 vars in 1224 bytes} [Tue Sep 13 03:36:22 2022] DELETE /identity/v3/users/fae11259c5894e94b8b1db60c00089c4 => generated 0 bytes in 214 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 03:36:22.983342 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cbdd396-ffcf-42d9-883e-25ef3983854f admin admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:36:22.984772 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-4cbdd396-ffcf-42d9-883e-25ef3983854f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:36:23.000345 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cbdd396-ffcf-42d9-883e-25ef3983854f admin admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:36:23.073841 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcaca560-f3fa-4445-a2b0-d3a2175788af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76c5181fc7be45279ac00017bb4640b3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:36:23.074639 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcaca560-f3fa-4445-a2b0-d3a2175788af admin admin] REQUEST_METHOD: `DELETE` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:36:23.074999 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcaca560-f3fa-4445-a2b0-d3a2175788af admin admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:36:23.075351 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcaca560-f3fa-4445-a2b0-d3a2175788af admin admin] PATH_INFO: `/v3/projects/279761f67a6c4a89b7cb9358ef5bfd06` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:36:23.088558 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcaca560-f3fa-4445-a2b0-d3a2175788af admin admin] RBAC: Authorizing `identity:delete_project(project_id=279761f67a6c4a89b7cb9358ef5bfd06)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:36:23.090567 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcaca560-f3fa-4445-a2b0-d3a2175788af admin admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:36:23.145575 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-fcaca560-f3fa-4445-a2b0-d3a2175788af admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '279761f67a6c4a89b7cb9358ef5bfd06', 'request_id': 'req-fcaca560-f3fa-4445-a2b0-d3a2175788af'} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:36:23.155293 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-fcaca560-f3fa-4445-a2b0-d3a2175788af admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '279761f67a6c4a89b7cb9358ef5bfd06', 'request_id': 'req-fcaca560-f3fa-4445-a2b0-d3a2175788af'} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:36:23.162703 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-fcaca560-f3fa-4445-a2b0-d3a2175788af admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '279761f67a6c4a89b7cb9358ef5bfd06', 'request_id': 'req-fcaca560-f3fa-4445-a2b0-d3a2175788af'} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:36:23.180314 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-fcaca560-f3fa-4445-a2b0-d3a2175788af admin admin] The token cache is being invalidate because project 279761f67a6c4a89b7cb9358ef5bfd06 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81400) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 03:36:23.181930 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-fcaca560-f3fa-4445-a2b0-d3a2175788af admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:36:23.183169 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-fcaca560-f3fa-4445-a2b0-d3a2175788af admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:36:23.185705 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 1455/2918] 10.222.0.18 () {58 vars in 1233 bytes} [Tue Sep 13 03:36:22 2022] DELETE /identity/v3/projects/279761f67a6c4a89b7cb9358ef5bfd06 => generated 0 bytes in 203 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 03:36:27.547778 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40907c79-45ce-44c4-825b-0cd30a175464 None None] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:36:27.548265 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40907c79-45ce-44c4-825b-0cd30a175464 None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:36:27.548367 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40907c79-45ce-44c4-825b-0cd30a175464 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:36:27.567282 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.auth.core [None req-40907c79-45ce-44c4-825b-0cd30a175464 None None] MFA Rules not processed for user `4c2d20fb54a54ecc99b47f237be4917c`. Rule list: `[]` (Enabled: `True`). {{(pid=81399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 03:36:27.568463 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-40907c79-45ce-44c4-825b-0cd30a175464 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:36:27.621672 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 1464/2919] 10.222.0.18 () {58 vars in 967 bytes} [Tue Sep 13 03:36:27 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 75 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 13 03:36:27.633682 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcaca560-f3fa-4445-a2b0-d3a2175788af admin admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:36:27.635580 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-fcaca560-f3fa-4445-a2b0-d3a2175788af admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:36:27.661505 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:36:27.705301 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcaca560-f3fa-4445-a2b0-d3a2175788af admin admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:36:27.710250 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c6f8670-0b95-42c5-9ecc-d75605ee1936 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '893b0c0d85964b7dbc9705c0a091682c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b9d94f6a68f7470d9494f112175620dc', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:36:27.710886 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c6f8670-0b95-42c5-9ecc-d75605ee1936 service swift] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:36:27.711162 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c6f8670-0b95-42c5-9ecc-d75605ee1936 service swift] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:36:27.711434 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c6f8670-0b95-42c5-9ecc-d75605ee1936 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:36:27.720067 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c6f8670-0b95-42c5-9ecc-d75605ee1936 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:36:27.721632 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c6f8670-0b95-42c5-9ecc-d75605ee1936 service swift] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:36:27.729565 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 1456/2920] 10.222.0.18 () {60 vars in 1384 bytes} [Tue Sep 13 03:36:27 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 13 03:36:29.552607 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40907c79-45ce-44c4-825b-0cd30a175464 None None] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:36:29.554135 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-40907c79-45ce-44c4-825b-0cd30a175464 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:36:29.568326 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:36:29.615650 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c6f8670-0b95-42c5-9ecc-d75605ee1936 service swift] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:36:29.617069 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-3c6f8670-0b95-42c5-9ecc-d75605ee1936 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:36:29.636618 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40907c79-45ce-44c4-825b-0cd30a175464 None None] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:36:29.641647 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c56effd-c9a9-435d-8144-3819979a7962 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b99f1b2374d4d2f8db78306316180d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b9d94f6a68f7470d9494f112175620dc', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:36:29.642248 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c56effd-c9a9-435d-8144-3819979a7962 service placement] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:36:29.642520 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c56effd-c9a9-435d-8144-3819979a7962 service placement] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:36:29.642816 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c56effd-c9a9-435d-8144-3819979a7962 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:36:29.644551 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-9c56effd-c9a9-435d-8144-3819979a7962 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:36:29.656650 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c56effd-c9a9-435d-8144-3819979a7962 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:36:29.658270 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c56effd-c9a9-435d-8144-3819979a7962 service placement] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:36:29.668576 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 1465/2921] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Sep 13 03:36:29 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 116 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 03:36:29.684407 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c6f8670-0b95-42c5-9ecc-d75605ee1936 service swift] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:36:29.691921 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74db9d4d-0755-495e-b98a-43243a0302a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76c5181fc7be45279ac00017bb4640b3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:36:29.692728 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74db9d4d-0755-495e-b98a-43243a0302a5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:36:29.693007 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74db9d4d-0755-495e-b98a-43243a0302a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:36:29.693275 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74db9d4d-0755-495e-b98a-43243a0302a5 admin admin] PATH_INFO: `/v3/users/79301dbf286e4b7d9fad34272a8002f5` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:36:29.694727 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74db9d4d-0755-495e-b98a-43243a0302a5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=79301dbf286e4b7d9fad34272a8002f5)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:36:29.697293 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74db9d4d-0755-495e-b98a-43243a0302a5 admin admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:36:29.742541 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-74db9d4d-0755-495e-b98a-43243a0302a5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '79301dbf286e4b7d9fad34272a8002f5'} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:36:29.750641 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-74db9d4d-0755-495e-b98a-43243a0302a5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '79301dbf286e4b7d9fad34272a8002f5'} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:36:29.751269 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-74db9d4d-0755-495e-b98a-43243a0302a5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '79301dbf286e4b7d9fad34272a8002f5'} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:36:29.763608 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-74db9d4d-0755-495e-b98a-43243a0302a5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '79301dbf286e4b7d9fad34272a8002f5'} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:36:29.770633 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-74db9d4d-0755-495e-b98a-43243a0302a5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '79301dbf286e4b7d9fad34272a8002f5'} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:36:29.776721 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 1457/2922] 10.222.0.18 () {58 vars in 1224 bytes} [Tue Sep 13 03:36:29 2022] DELETE /identity/v3/users/79301dbf286e4b7d9fad34272a8002f5 => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 13 03:36:29.949749 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c56effd-c9a9-435d-8144-3819979a7962 service placement] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:36:29.960347 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c56effd-c9a9-435d-8144-3819979a7962 service placement] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:36:30.023573 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7b7339c-f66e-464e-b6dd-e4f50616cfc8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76c5181fc7be45279ac00017bb4640b3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:36:30.024232 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b7339c-f66e-464e-b6dd-e4f50616cfc8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:36:30.024518 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b7339c-f66e-464e-b6dd-e4f50616cfc8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:36:30.024802 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b7339c-f66e-464e-b6dd-e4f50616cfc8 admin admin] PATH_INFO: `/v3/projects/1acee96791e54960abd4d9d1c735a298` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:36:30.025941 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7b7339c-f66e-464e-b6dd-e4f50616cfc8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1acee96791e54960abd4d9d1c735a298)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:36:30.027884 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7b7339c-f66e-464e-b6dd-e4f50616cfc8 admin admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:36:30.076043 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-b7b7339c-f66e-464e-b6dd-e4f50616cfc8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1acee96791e54960abd4d9d1c735a298', 'request_id': 'req-b7b7339c-f66e-464e-b6dd-e4f50616cfc8'} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:36:30.081484 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-b7b7339c-f66e-464e-b6dd-e4f50616cfc8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1acee96791e54960abd4d9d1c735a298', 'request_id': 'req-b7b7339c-f66e-464e-b6dd-e4f50616cfc8'} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:36:30.087582 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-b7b7339c-f66e-464e-b6dd-e4f50616cfc8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1acee96791e54960abd4d9d1c735a298', 'request_id': 'req-b7b7339c-f66e-464e-b6dd-e4f50616cfc8'} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:36:30.099916 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-b7b7339c-f66e-464e-b6dd-e4f50616cfc8 admin admin] The token cache is being invalidate because project 1acee96791e54960abd4d9d1c735a298 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81399) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 03:36:30.100212 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-b7b7339c-f66e-464e-b6dd-e4f50616cfc8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:36:30.100841 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-b7b7339c-f66e-464e-b6dd-e4f50616cfc8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:36:30.102332 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 1466/2923] 10.222.0.18 () {58 vars in 1233 bytes} [Tue Sep 13 03:36:29 2022] DELETE /identity/v3/projects/1acee96791e54960abd4d9d1c735a298 => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 03:36:30.107658 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7daa1d76-fb89-4a1b-8f46-276d00686843 None None] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:36:30.107948 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7daa1d76-fb89-4a1b-8f46-276d00686843 None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:36:30.108223 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7daa1d76-fb89-4a1b-8f46-276d00686843 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:36:30.129572 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.auth.core [None req-7daa1d76-fb89-4a1b-8f46-276d00686843 None None] MFA Rules not processed for user `49f280ab086b4610855c4ffa378517d9`. Rule list: `[]` (Enabled: `True`). {{(pid=81400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 03:36:30.130729 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-7daa1d76-fb89-4a1b-8f46-276d00686843 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:36:30.156614 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:36:30.192164 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 1458/2924] 10.222.0.18 () {58 vars in 958 bytes} [Tue Sep 13 03:36:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 03:36:30.199885 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3935c679-eea0-418f-ba90-e1c06e17ce9a None None] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:36:30.200176 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3935c679-eea0-418f-ba90-e1c06e17ce9a None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:36:30.200451 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3935c679-eea0-418f-ba90-e1c06e17ce9a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:36:30.220544 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.auth.core [None req-3935c679-eea0-418f-ba90-e1c06e17ce9a None None] MFA Rules not processed for user `49f280ab086b4610855c4ffa378517d9`. Rule list: `[]` (Enabled: `True`). {{(pid=81399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 03:36:30.221748 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-3935c679-eea0-418f-ba90-e1c06e17ce9a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:36:30.230647 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 1467/2925] 10.222.0.18 () {58 vars in 958 bytes} [Tue Sep 13 03:36:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 03:36:30.234886 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7daa1d76-fb89-4a1b-8f46-276d00686843 None None] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:36:30.244515 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7daa1d76-fb89-4a1b-8f46-276d00686843 None None] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:36:30.249021 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31c8196a-3669-4ed6-a2af-03ba3194169a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76c5181fc7be45279ac00017bb4640b3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:36:30.249609 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31c8196a-3669-4ed6-a2af-03ba3194169a admin admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:36:30.249886 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31c8196a-3669-4ed6-a2af-03ba3194169a admin admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:36:30.250154 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31c8196a-3669-4ed6-a2af-03ba3194169a admin admin] PATH_INFO: `/v3/domains` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:36:30.250824 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31c8196a-3669-4ed6-a2af-03ba3194169a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 03:36:30.251152 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31c8196a-3669-4ed6-a2af-03ba3194169a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:36:30.252744 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31c8196a-3669-4ed6-a2af-03ba3194169a admin admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:36:30.264026 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 1459/2926] 10.222.0.18 () {58 vars in 1153 bytes} [Tue Sep 13 03:36:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 03:36:30.268205 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3935c679-eea0-418f-ba90-e1c06e17ce9a None None] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:36:30.278014 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3935c679-eea0-418f-ba90-e1c06e17ce9a None None] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:36:30.282690 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0734341a-c958-4b33-930e-a9221dc650d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76c5181fc7be45279ac00017bb4640b3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:36:30.283284 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0734341a-c958-4b33-930e-a9221dc650d2 admin admin] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:36:30.283566 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0734341a-c958-4b33-930e-a9221dc650d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:36:30.283841 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0734341a-c958-4b33-930e-a9221dc650d2 admin admin] PATH_INFO: `/v3/projects` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:36:30.284748 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0734341a-c958-4b33-930e-a9221dc650d2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:36:30.286833 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0734341a-c958-4b33-930e-a9221dc650d2 admin admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:36:30.314275 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 1468/2927] 10.222.0.18 () {60 vars in 1153 bytes} [Tue Sep 13 03:36:30 2022] POST /identity/v3/projects => generated 375 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 03:36:30.318436 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31c8196a-3669-4ed6-a2af-03ba3194169a admin admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:36:30.328360 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31c8196a-3669-4ed6-a2af-03ba3194169a admin admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:36:30.384652 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cd7d2e6-9c21-4b7d-bb85-144eee0e7fc8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76c5181fc7be45279ac00017bb4640b3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:36:30.385176 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd7d2e6-9c21-4b7d-bb85-144eee0e7fc8 admin admin] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:36:30.385451 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd7d2e6-9c21-4b7d-bb85-144eee0e7fc8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:36:30.385720 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd7d2e6-9c21-4b7d-bb85-144eee0e7fc8 admin admin] PATH_INFO: `/v3/users` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:36:30.386331 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cd7d2e6-9c21-4b7d-bb85-144eee0e7fc8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:36:30.388284 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cd7d2e6-9c21-4b7d-bb85-144eee0e7fc8 admin admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:36:30.408955 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 1460/2928] 10.222.0.18 () {60 vars in 1144 bytes} [Tue Sep 13 03:36:30 2022] POST /identity/v3/users => generated 288 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 03:36:30.413302 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0734341a-c958-4b33-930e-a9221dc650d2 admin admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:36:30.420481 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:36:30.424575 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0734341a-c958-4b33-930e-a9221dc650d2 admin admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:36:30.430498 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaaf4d69-62a3-4ec3-bd5b-c50961d1f167 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76c5181fc7be45279ac00017bb4640b3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:36:30.431494 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaaf4d69-62a3-4ec3-bd5b-c50961d1f167 admin admin] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:36:30.431965 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaaf4d69-62a3-4ec3-bd5b-c50961d1f167 admin admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:36:30.432459 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaaf4d69-62a3-4ec3-bd5b-c50961d1f167 admin admin] PATH_INFO: `/v3/roles` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:36:30.433104 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaaf4d69-62a3-4ec3-bd5b-c50961d1f167 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:36:30.434749 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaaf4d69-62a3-4ec3-bd5b-c50961d1f167 admin admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:36:30.444643 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 1469/2929] 10.222.0.18 () {58 vars in 1122 bytes} [Tue Sep 13 03:36:30 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 03:36:30.448629 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cd7d2e6-9c21-4b7d-bb85-144eee0e7fc8 admin admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:36:30.458877 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cd7d2e6-9c21-4b7d-bb85-144eee0e7fc8 admin admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:36:30.464055 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b5ad93c-deeb-40c8-bca1-693e5a759eae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76c5181fc7be45279ac00017bb4640b3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:36:30.464663 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b5ad93c-deeb-40c8-bca1-693e5a759eae admin admin] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:36:30.465063 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b5ad93c-deeb-40c8-bca1-693e5a759eae admin admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:36:30.465402 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b5ad93c-deeb-40c8-bca1-693e5a759eae admin admin] PATH_INFO: `/v3/roles` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:36:30.465837 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b5ad93c-deeb-40c8-bca1-693e5a759eae admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:36:30.467467 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b5ad93c-deeb-40c8-bca1-693e5a759eae admin admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:36:30.478938 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 1461/2930] 10.222.0.18 () {58 vars in 1122 bytes} [Tue Sep 13 03:36:30 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 13 03:36:30.483338 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaaf4d69-62a3-4ec3-bd5b-c50961d1f167 admin admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:36:30.493261 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaaf4d69-62a3-4ec3-bd5b-c50961d1f167 admin admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:36:30.497943 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e208dc63-b827-4679-b16f-64d02aa4e9f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76c5181fc7be45279ac00017bb4640b3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:36:30.498588 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e208dc63-b827-4679-b16f-64d02aa4e9f9 admin admin] REQUEST_METHOD: `PUT` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:36:30.498899 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e208dc63-b827-4679-b16f-64d02aa4e9f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:36:30.499175 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e208dc63-b827-4679-b16f-64d02aa4e9f9 admin admin] PATH_INFO: `/v3/projects/708dee9f9959492990bf96006734070f/users/6bbd57e1a98644629c2602996cfe224e/roles/1eb7a5b5203945c29e6d22d653f47847` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:36:30.514046 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e208dc63-b827-4679-b16f-64d02aa4e9f9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=708dee9f9959492990bf96006734070f, user_id=6bbd57e1a98644629c2602996cfe224e, role_id=1eb7a5b5203945c29e6d22d653f47847)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:36:30.516737 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e208dc63-b827-4679-b16f-64d02aa4e9f9 admin admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:36:30.530292 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 1470/2931] 10.222.0.18 () {60 vars in 1483 bytes} [Tue Sep 13 03:36:30 2022] PUT /identity/v3/projects/708dee9f9959492990bf96006734070f/users/6bbd57e1a98644629c2602996cfe224e/roles/1eb7a5b5203945c29e6d22d653f47847 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 13 03:36:30.541369 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3252a6e8-93a0-4d85-a601-78800d98683a None None] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:36:30.541652 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3252a6e8-93a0-4d85-a601-78800d98683a None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:36:30.541961 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3252a6e8-93a0-4d85-a601-78800d98683a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:36:30.568899 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.auth.core [None req-3252a6e8-93a0-4d85-a601-78800d98683a None None] MFA Rules not processed for user `6bbd57e1a98644629c2602996cfe224e`. Rule list: `[]` (Enabled: `True`). {{(pid=81400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 03:36:30.570109 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-3252a6e8-93a0-4d85-a601-78800d98683a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:36:30.632308 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 1462/2932] 10.222.0.18 () {58 vars in 958 bytes} [Tue Sep 13 03:36:30 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 03:36:30.643527 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e208dc63-b827-4679-b16f-64d02aa4e9f9 admin admin] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:36:30.645252 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-e208dc63-b827-4679-b16f-64d02aa4e9f9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:36:30.701488 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e208dc63-b827-4679-b16f-64d02aa4e9f9 admin admin] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:36:30.705149 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea07d6dc-f826-435e-b60f-d59056db3caa service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '924ee3f608d34a18b0ddc4695f102a56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b9d94f6a68f7470d9494f112175620dc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:36:30.705740 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea07d6dc-f826-435e-b60f-d59056db3caa service cinder] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:36:30.706017 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea07d6dc-f826-435e-b60f-d59056db3caa service cinder] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:36:30.706287 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea07d6dc-f826-435e-b60f-d59056db3caa service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:36:30.715079 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea07d6dc-f826-435e-b60f-d59056db3caa service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:36:30.716501 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea07d6dc-f826-435e-b60f-d59056db3caa service cinder] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:36:30.725167 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 1471/2933] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Sep 13 03:36:30 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 03:36:32.076028 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f672f5-b57b-458c-8c8c-7659a4d06e99 None None] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:36:32.076483 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f672f5-b57b-458c-8c8c-7659a4d06e99 None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:36:32.076587 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f672f5-b57b-458c-8c8c-7659a4d06e99 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:36:32.095680 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.auth.core [None req-b9f672f5-b57b-458c-8c8c-7659a4d06e99 None None] MFA Rules not processed for user `6bbd57e1a98644629c2602996cfe224e`. Rule list: `[]` (Enabled: `True`). {{(pid=81400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 03:36:32.096845 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-b9f672f5-b57b-458c-8c8c-7659a4d06e99 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:36:32.103555 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 1463/2934] 10.222.0.18 () {58 vars in 958 bytes} [Tue Sep 13 03:36:32 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 03:36:32.560755 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea07d6dc-f826-435e-b60f-d59056db3caa service cinder] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:36:32.562241 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-ea07d6dc-f826-435e-b60f-d59056db3caa service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:36:32.635313 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea07d6dc-f826-435e-b60f-d59056db3caa service cinder] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:36:32.640241 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-157fdb44-5e2c-4a5e-b705-c1c0db2d18d7 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b99f1b2374d4d2f8db78306316180d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b9d94f6a68f7470d9494f112175620dc', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:36:32.640841 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-157fdb44-5e2c-4a5e-b705-c1c0db2d18d7 service placement] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:36:32.641118 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-157fdb44-5e2c-4a5e-b705-c1c0db2d18d7 service placement] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:36:32.641387 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-157fdb44-5e2c-4a5e-b705-c1c0db2d18d7 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:36:32.643123 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-157fdb44-5e2c-4a5e-b705-c1c0db2d18d7 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:36:32.654448 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-157fdb44-5e2c-4a5e-b705-c1c0db2d18d7 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:36:32.656080 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-157fdb44-5e2c-4a5e-b705-c1c0db2d18d7 service placement] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:36:32.666467 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 1472/2935] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Sep 13 03:36:32 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 03:36:42.090321 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=88,keystone:INSERT=3 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:36:42.662071 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=71,keystone:INSERT=1 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:36:48.049313 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c11b74c-8a53-4e2d-8d10-0a135d61b81d None None] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:36:48.049813 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c11b74c-8a53-4e2d-8d10-0a135d61b81d None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:36:48.049934 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c11b74c-8a53-4e2d-8d10-0a135d61b81d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:36:48.073993 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.auth.core [None req-2c11b74c-8a53-4e2d-8d10-0a135d61b81d None None] MFA Rules not processed for user `4c2d20fb54a54ecc99b47f237be4917c`. Rule list: `[]` (Enabled: `True`). {{(pid=81400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 03:36:48.075823 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-2c11b74c-8a53-4e2d-8d10-0a135d61b81d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:36:48.129116 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 1464/2936] 10.222.0.18 () {58 vars in 1009 bytes} [Tue Sep 13 03:36:48 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 03:36:48.150780 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-157fdb44-5e2c-4a5e-b705-c1c0db2d18d7 service placement] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:36:48.152182 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-157fdb44-5e2c-4a5e-b705-c1c0db2d18d7 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:36:48.220388 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-157fdb44-5e2c-4a5e-b705-c1c0db2d18d7 service placement] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:36:48.225709 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ffafb8b-ffa8-4398-b9a7-d3d4d2577f64 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '893b0c0d85964b7dbc9705c0a091682c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b9d94f6a68f7470d9494f112175620dc', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:36:48.226524 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ffafb8b-ffa8-4398-b9a7-d3d4d2577f64 service swift] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:36:48.226833 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ffafb8b-ffa8-4398-b9a7-d3d4d2577f64 service swift] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:36:48.227104 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ffafb8b-ffa8-4398-b9a7-d3d4d2577f64 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:36:48.236445 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ffafb8b-ffa8-4398-b9a7-d3d4d2577f64 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:36:48.238024 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ffafb8b-ffa8-4398-b9a7-d3d4d2577f64 service swift] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:36:48.246058 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 1473/2937] 10.222.0.18 () {60 vars in 1384 bytes} [Tue Sep 13 03:36:48 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 13 03:36:48.744704 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b01e1bee-25ef-489f-a291-bcd2952695f9 None None] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:36:48.745096 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b01e1bee-25ef-489f-a291-bcd2952695f9 None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:36:48.745465 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b01e1bee-25ef-489f-a291-bcd2952695f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:36:48.766251 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.auth.core [None req-b01e1bee-25ef-489f-a291-bcd2952695f9 None None] MFA Rules not processed for user `4c2d20fb54a54ecc99b47f237be4917c`. Rule list: `[]` (Enabled: `True`). {{(pid=81400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 03:36:48.767476 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-b01e1bee-25ef-489f-a291-bcd2952695f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:36:48.774743 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 1465/2938] 10.222.0.18 () {58 vars in 967 bytes} [Tue Sep 13 03:36:48 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 03:36:48.787872 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ffafb8b-ffa8-4398-b9a7-d3d4d2577f64 service swift] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:36:48.798938 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ffafb8b-ffa8-4398-b9a7-d3d4d2577f64 service swift] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:36:48.803954 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dc35898-017c-43d5-9c64-6623076ddabd service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '893b0c0d85964b7dbc9705c0a091682c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b9d94f6a68f7470d9494f112175620dc', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:36:48.804595 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dc35898-017c-43d5-9c64-6623076ddabd service swift] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:36:48.804869 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dc35898-017c-43d5-9c64-6623076ddabd service swift] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:36:48.805136 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dc35898-017c-43d5-9c64-6623076ddabd service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:36:48.813444 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dc35898-017c-43d5-9c64-6623076ddabd service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:36:48.815040 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dc35898-017c-43d5-9c64-6623076ddabd service swift] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:36:48.822913 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 1474/2939] 10.222.0.18 () {60 vars in 1384 bytes} [Tue Sep 13 03:36:48 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 13 03:36:49.042156 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cab6c2e0-0e62-48b3-854d-9d58490791ce None None] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:36:49.042437 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cab6c2e0-0e62-48b3-854d-9d58490791ce None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:36:49.042742 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cab6c2e0-0e62-48b3-854d-9d58490791ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:36:49.062174 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.auth.core [None req-cab6c2e0-0e62-48b3-854d-9d58490791ce None None] MFA Rules not processed for user `4c2d20fb54a54ecc99b47f237be4917c`. Rule list: `[]` (Enabled: `True`). {{(pid=81400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 03:36:49.063446 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-cab6c2e0-0e62-48b3-854d-9d58490791ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:36:49.070716 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 1466/2940] 10.222.0.18 () {58 vars in 1009 bytes} [Tue Sep 13 03:36:49 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 03:36:49.082943 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dc35898-017c-43d5-9c64-6623076ddabd service swift] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:36:49.094076 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dc35898-017c-43d5-9c64-6623076ddabd service swift] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:36:49.099023 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8e405eb-4e63-4566-a8c0-24d4132c12b4 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '893b0c0d85964b7dbc9705c0a091682c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b9d94f6a68f7470d9494f112175620dc', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:36:49.099635 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8e405eb-4e63-4566-a8c0-24d4132c12b4 service swift] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:36:49.099906 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8e405eb-4e63-4566-a8c0-24d4132c12b4 service swift] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:36:49.100171 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8e405eb-4e63-4566-a8c0-24d4132c12b4 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:36:49.108508 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8e405eb-4e63-4566-a8c0-24d4132c12b4 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:36:49.110073 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8e405eb-4e63-4566-a8c0-24d4132c12b4 service swift] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:36:49.117913 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 1475/2941] 10.222.0.18 () {60 vars in 1384 bytes} [Tue Sep 13 03:36:49 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Sep 13 03:36:49.430841 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a20c02f-5ac1-4995-9a6b-782736e3e2ec None None] REQUEST_METHOD: `POST` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:36:49.431125 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a20c02f-5ac1-4995-9a6b-782736e3e2ec None None] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:36:49.431404 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a20c02f-5ac1-4995-9a6b-782736e3e2ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:36:49.450875 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.auth.core [None req-0a20c02f-5ac1-4995-9a6b-782736e3e2ec None None] MFA Rules not processed for user `4c2d20fb54a54ecc99b47f237be4917c`. Rule list: `[]` (Enabled: `True`). {{(pid=81400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 03:36:49.452101 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-0a20c02f-5ac1-4995-9a6b-782736e3e2ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:36:49.459274 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 1467/2942] 10.222.0.18 () {58 vars in 967 bytes} [Tue Sep 13 03:36:49 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 03:36:49.471316 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8e405eb-4e63-4566-a8c0-24d4132c12b4 service swift] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:36:49.482328 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8e405eb-4e63-4566-a8c0-24d4132c12b4 service swift] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:36:49.487436 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-253d6f5a-2dea-4a92-9b1e-bef34f20af87 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '893b0c0d85964b7dbc9705c0a091682c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b9d94f6a68f7470d9494f112175620dc', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:36:49.487972 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-253d6f5a-2dea-4a92-9b1e-bef34f20af87 service swift] REQUEST_METHOD: `GET` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:36:49.488252 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-253d6f5a-2dea-4a92-9b1e-bef34f20af87 service swift] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:36:49.488523 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-253d6f5a-2dea-4a92-9b1e-bef34f20af87 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:36:49.496353 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-253d6f5a-2dea-4a92-9b1e-bef34f20af87 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:36:49.498511 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-253d6f5a-2dea-4a92-9b1e-bef34f20af87 service swift] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:36:49.506406 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 1476/2943] 10.222.0.18 () {60 vars in 1384 bytes} [Tue Sep 13 03:36:49 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 13 03:36:51.579065 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a20c02f-5ac1-4995-9a6b-782736e3e2ec None None] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:36:51.641250 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a20c02f-5ac1-4995-9a6b-782736e3e2ec None None] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:36:51.645717 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78e202bb-8107-4885-8441-9e43de1fab0e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76c5181fc7be45279ac00017bb4640b3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:36:51.646336 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e202bb-8107-4885-8441-9e43de1fab0e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:36:51.646621 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e202bb-8107-4885-8441-9e43de1fab0e admin admin] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:36:51.646912 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e202bb-8107-4885-8441-9e43de1fab0e admin admin] PATH_INFO: `/v3/users/6bbd57e1a98644629c2602996cfe224e` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:36:51.648037 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78e202bb-8107-4885-8441-9e43de1fab0e admin admin] RBAC: Authorizing `identity:delete_user(user_id=6bbd57e1a98644629c2602996cfe224e)` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:36:51.649941 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78e202bb-8107-4885-8441-9e43de1fab0e admin admin] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:36:51.695648 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-78e202bb-8107-4885-8441-9e43de1fab0e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6bbd57e1a98644629c2602996cfe224e'} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:36:51.702633 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-78e202bb-8107-4885-8441-9e43de1fab0e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6bbd57e1a98644629c2602996cfe224e'} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:36:51.703260 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-78e202bb-8107-4885-8441-9e43de1fab0e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6bbd57e1a98644629c2602996cfe224e'} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:36:51.715261 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-78e202bb-8107-4885-8441-9e43de1fab0e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6bbd57e1a98644629c2602996cfe224e'} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:36:51.722130 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.notifications [None req-78e202bb-8107-4885-8441-9e43de1fab0e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6bbd57e1a98644629c2602996cfe224e'} {{(pid=81400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:36:51.727924 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 1468/2944] 10.222.0.18 () {58 vars in 1224 bytes} [Tue Sep 13 03:36:51 2022] DELETE /identity/v3/users/6bbd57e1a98644629c2602996cfe224e => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 03:36:51.733045 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c1b7adb-11c8-4df8-953d-d7f7a6b2577c None None] REQUEST_METHOD: `POST` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:36:51.733533 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c1b7adb-11c8-4df8-953d-d7f7a6b2577c None None] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:36:51.733652 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c1b7adb-11c8-4df8-953d-d7f7a6b2577c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:36:51.753133 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.auth.core [None req-8c1b7adb-11c8-4df8-953d-d7f7a6b2577c None None] MFA Rules not processed for user `49f280ab086b4610855c4ffa378517d9`. Rule list: `[]` (Enabled: `True`). {{(pid=81399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 03:36:51.754332 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-8c1b7adb-11c8-4df8-953d-d7f7a6b2577c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:36:51.812859 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 1477/2945] 10.222.0.18 () {58 vars in 958 bytes} [Tue Sep 13 03:36:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 03:36:51.822798 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78e202bb-8107-4885-8441-9e43de1fab0e admin admin] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:36:51.824205 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-78e202bb-8107-4885-8441-9e43de1fab0e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:36:51.830660 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:36:51.882460 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78e202bb-8107-4885-8441-9e43de1fab0e admin admin] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:36:51.886093 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89006d76-ba55-4427-90a7-0814503ecacd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdffbc851086441ebd01022d1e4c2609', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b9d94f6a68f7470d9494f112175620dc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:36:51.886790 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89006d76-ba55-4427-90a7-0814503ecacd service neutron] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:36:51.887066 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89006d76-ba55-4427-90a7-0814503ecacd service neutron] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:36:51.887334 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89006d76-ba55-4427-90a7-0814503ecacd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:36:51.896359 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89006d76-ba55-4427-90a7-0814503ecacd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:36:51.897775 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89006d76-ba55-4427-90a7-0814503ecacd service neutron] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:36:51.907535 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 1469/2946] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Sep 13 03:36:51 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 03:36:52.075636 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c1b7adb-11c8-4df8-953d-d7f7a6b2577c None None] Authenticating user token {{(pid=81399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:36:52.077066 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.fernet_utils [None req-8c1b7adb-11c8-4df8-953d-d7f7a6b2577c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:36:52.089782 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c1b7adb-11c8-4df8-953d-d7f7a6b2577c None None] Validating token access rules against request {{(pid=81399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:36:52.094936 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ef9ba69-057f-485d-b44b-6973ef38248a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f280ab086b4610855c4ffa378517d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76c5181fc7be45279ac00017bb4640b3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:36:52.095579 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef9ba69-057f-485d-b44b-6973ef38248a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:36:52.095841 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef9ba69-057f-485d-b44b-6973ef38248a admin admin] SCRIPT_NAME: `/identity` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:36:52.096117 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef9ba69-057f-485d-b44b-6973ef38248a admin admin] PATH_INFO: `/v3/projects/708dee9f9959492990bf96006734070f` {{(pid=81399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:36:52.097879 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ef9ba69-057f-485d-b44b-6973ef38248a admin admin] RBAC: Authorizing `identity:delete_project(project_id=708dee9f9959492990bf96006734070f)` {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:36:52.099839 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ef9ba69-057f-485d-b44b-6973ef38248a admin admin] RBAC: Authorization granted {{(pid=81399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:36:52.143693 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:36:52.146747 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-3ef9ba69-057f-485d-b44b-6973ef38248a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '708dee9f9959492990bf96006734070f', 'request_id': 'req-3ef9ba69-057f-485d-b44b-6973ef38248a'} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:36:52.152728 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-3ef9ba69-057f-485d-b44b-6973ef38248a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '708dee9f9959492990bf96006734070f', 'request_id': 'req-3ef9ba69-057f-485d-b44b-6973ef38248a'} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:36:52.159701 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-3ef9ba69-057f-485d-b44b-6973ef38248a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '708dee9f9959492990bf96006734070f', 'request_id': 'req-3ef9ba69-057f-485d-b44b-6973ef38248a'} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:36:52.172467 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-3ef9ba69-057f-485d-b44b-6973ef38248a admin admin] The token cache is being invalidate because project 708dee9f9959492990bf96006734070f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81399) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 03:36:52.172753 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-3ef9ba69-057f-485d-b44b-6973ef38248a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:36:52.173359 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG keystone.notifications [None req-3ef9ba69-057f-485d-b44b-6973ef38248a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 03:36:52.174853 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: [pid: 81399|app: 0|req: 1478/2947] 10.222.0.18 () {58 vars in 1233 bytes} [Tue Sep 13 03:36:52 2022] DELETE /identity/v3/projects/708dee9f9959492990bf96006734070f => generated 0 bytes in 100 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 03:37:01.903616 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=24 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:37:02.169498 dv-cinder-857075-2-iscsi devstack@keystone.service[81399]: DEBUG dbcounter [-] [81399] Writing DB stats keystone:DELETE=4,keystone:SELECT=7,keystone:INSERT=1 {{(pid=81399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 03:39:13.557028 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89006d76-ba55-4427-90a7-0814503ecacd service neutron] Authenticating user token {{(pid=81400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 03:39:13.558427 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-89006d76-ba55-4427-90a7-0814503ecacd service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:39:13.694309 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89006d76-ba55-4427-90a7-0814503ecacd service neutron] Validating token access rules against request {{(pid=81400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 03:39:13.751947 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56518fdc-2823-4782-8b82-d1958166a7ed service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b99f1b2374d4d2f8db78306316180d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b9d94f6a68f7470d9494f112175620dc', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 03:39:13.752548 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56518fdc-2823-4782-8b82-d1958166a7ed service placement] REQUEST_METHOD: `GET` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 03:39:13.752825 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56518fdc-2823-4782-8b82-d1958166a7ed service placement] SCRIPT_NAME: `/identity` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 03:39:13.753096 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56518fdc-2823-4782-8b82-d1958166a7ed service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 03:39:13.754871 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.fernet_utils [None req-56518fdc-2823-4782-8b82-d1958166a7ed service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 03:39:13.766535 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56518fdc-2823-4782-8b82-d1958166a7ed service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 03:39:13.768124 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56518fdc-2823-4782-8b82-d1958166a7ed service placement] RBAC: Authorization granted {{(pid=81400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 03:39:13.779518 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: [pid: 81400|app: 0|req: 1470/2948] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Sep 13 03:39:13 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 223 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 03:39:23.774184 dv-cinder-857075-2-iscsi devstack@keystone.service[81400]: DEBUG dbcounter [-] [81400] Writing DB stats keystone:SELECT=89 {{(pid=81400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}}